Estimating Resource Utilization in Congenital Heart Surgery

Abstract

Background:

Optimal methods to assess resource utilization in congenital heart surgery remain unclear. We compared traditional cost-to-charge ratio methods with newer standardized cost methods which aim to more directly assess resources consumed.

Methods:

Clinical data from the STS Database were linked with resource use data from the Pediatric Health Information Systems Database (2010–2015). Standardized cost methods specific to the congenital heart surgery population were developed and compared to cost-to-charge ratio methods. Resource use in the overall population and variability across hospitals were described using hierarchical mixed effect models adjusting for case-mix.

Results:

Overall 43 hospitals (65,331 patients) were included. There were minimal population-level differences in the distribution of resource use as estimated by the two methods. At the hospital-level, there was less apparent variability in resource use across centers with the standardized cost vs. cost-to-charge ratio method, overall (coefficient of variation 20% vs. 25%) and across complexity (STAT) categories. When hospitals were categorized into tertiles by resource use, 33% changed classification depending on which resource use method was used (26% by one tertile and 7% by two tertiles).

Conclusions:

In this first evaluation of standardized cost methodology in the congenital heart population, we found minimal differences vs. traditional methods at the population-level. At the hospital-level, the magnitude of variation in resource use was less with standardized cost methods and ~1/3 of centers changed resource use categories depending on the methodology used. Because of these differences, care should be taken in future studies and benchmarking/reporting efforts in selecting optimal methodology.

INTRODUCTION

Congenital heart defects accounts for the highest costs of any birth defect and are in the top tier of resource utilization across children’s hospitals (1,2). Multiple ongoing initiatives in the field aim to both improve clinical outcomes for these patients and reduce costs of care, thereby optimizing healthcare “value” (3). These include various quality improvement activities, research projects, and health policy initiatives.

However, ideal methods to measure costs, or the resources consumed in providing care, in this population and others remain unclear. Concern has been raised that traditional methods for estimating costs from charges on the hospital bill (using a “cost-to-charge ratio”) are reflective of not only resources consumed but also other hospital-related factors (1). For example, using this method, differences across hospitals in costs associated with laboratory tests could reflect both the number of tests ordered, as well as differences in the costaccounting structure of the hospital with regard to how charges and costs are calculated and assigned. Newer “standardized cost” methodology has been developed aiming to focus more directly on the component specific to actual resources consumed, but has not been studied to date or assessed in relation to traditional methods in the congenital heart disease population (1). Differences between the two methods with regard to characterization of patient or hospital level costs remain unclear, and could have implications for current reporting and benchmarking efforts.

The purpose of this study was to apply and evaluate previously developed standardized cost methodology from the general pediatric population to a large multi-center cohort of patients undergoing congenital heart surgery. We compared these methods to traditional cost-to-charge ratio methods with regard to both patient- and hospital-level assessments of resource utilization, and characterization of variability in resource use across institutions.

METHODS

Data Source

Linked data from the Society of Thoracic Surgeons Congenital Heart Surgery Database (STS-CHSD) and Pediatric Health Information Systems (PHIS) Database were used for this analysis. As described previously, this approach capitalizes on the detailed clinical information in the STS-CHSD and resource utilization information captured in the PHIS Database (4). The STS-CHSD collects standard peri-operative data on all patients undergoing pediatric and congenital heart surgery at participating hospitals. Data quality is optimized through data checks, site visits, and audits. The PHIS Database is a large administrative database that collects inpatient resource utilization information from US children’s hospitals. Systematic monitoring in the PHIS Database includes coding consensus meetings, consistency reviews, and quarterly data quality reports (4). This study was not considered human subjects research by the Duke University and University of Michigan Institutional Review Boards in accordance with the Common Rule (45 CFR 46.102(f)).

Study Population and Data linkage

As previously described and validated, STS-CHS and PHIS data were linked at the patient level for children 0–18 years undergoing congenital heart surgery by matching on indirect identifiers (hospital, sex, date of birth, date of admission, date of discharge) (4). Hospitals participating in both the STS-CHSD and PHIS from 2010–2015 (the time period for which linked data were available) were eligible for inclusion (n=44 of 124 STS-CHSD hospitals). From these hospitals, patients identified in the STS-CHSD undergoing any cardiovascular operation (with or without cardiopulmonary bypass) classified in any of the Society of Thoracic Surgeons-European Association for Cardiothoracic Surgery (STAT) categories were included (5). We excluded 1 hospital with data quality issues, and 73 quarters of data from 8 hospitals (n=4,005) who were not participating in PHIS at the time or had incomplete PHIS data submission. Of 69,305 eligible patients from 43 hospitals, 95.1% (65,890) STS-CHSD records were matched to PHIS, with a range of 88–99% across hospitals. These linkage rates were similar to, or exceeded, rates from prior studies in this population (6–8). After linkage, an additional 559 records were removed due to missing cost or STAT category for a final sample size of 65,331.

Data Collection

Data collected from the STS-CHSD included all clinical variables for the study: demographics, standard STS-defined pre-operative risk factors, non-cardiac/genetic abnormalities, prematurity, previous cardiothoracic surgery, and STAT category (9).

Data collected from the PHIS Database included payer type and resource utilization information. Of note, professional fees are not included in most administrative datasets, including the PHIS Database, and thus are not included in this analysis.

Resource Use

Resource utilization during the admission was estimated using two methods. To briefly review distinct concepts in this area that are often confused: Cost refers to the resources consumed to deliver care. Charges are the fee assigned by the provider for the service. Payment is the reimbursement to the provider by the payer. Charges are not necessarily equal to any measure of cost, and usually charges exceed payments due to negotiated discounts between providers and payers (10). In this study we are focused on resources consumed, or cost, and different methods that can be used to estimate this variable.

Cost-to-Charge Ratio Method:

We first used traditional methods based on cost-to-charge ratios. As described previously, this methodology estimates costs from charges using each hospital’s annual 29 department-specific cost-to-charge ratios reported to the Centers for Medicare and Medicaid Services (CMS) Healthcare Cost Report Information System (1). Regional differences are adjusted for using the CMS price-wage index (1). Values were indexed to 2015 US dollars to account for inflation.

Standardized Cost Method:

As described in prior reports, this methodology standardizes the cost assigned to individual items across hospitals to remove inter-hospital variability in unit costs and allow a more direct understanding of resources consumed (1). The number of units of a particular item is multiplied by a standardized cost for that item, and the same standardized cost is used across hospitals. For example, this method results in all complete blood counts (CBC) being assigned the same standardized cost across hospitals, so any differences across hospitals in CBC standardized costs are related to differences in the number of CBC’s ordered.

For this analysis, we used the Cost Master Index developed previously within the PHIS Database (1). The Cost Master Index assigns a standardized cost to every billed item in the PHIS database. The unit standardized cost for each item for a calendar year is based on the median of the hospital medians of unit costs for that item, calculated from charges and cost-to-charge ratios as described in the preceding section. To calculate standardized cost for the entire hospital visit, the number of units of each item billed for the admission is multiplied by its standardized cost for the relevant calendar year and then summed. All standardized costs were inflated to 2015 dollars as described above.

Several strategies were used to address data discrepancies, missing data, and extreme values. If the unit standardized cost from the Cost Master Index was missing for a particular year, this was imputed with the median from available years. Items with zero charge were assigned zero cost regardless of the number of units indicated as these were likely items that were ordered but not utilized. There was a small proportion of items with extreme units and/or charges, exaggerating the standardized costs. In this instance, the standardized cost was replaced with the estimate from the cost-to-charge ratio method. In choosing the threshold for what was considered “extreme”, a prior study utilized a >3-fold difference between the standardized cost and cost-to-charge ratio method (1). We examined varying thresholds including 3-, 5-, 10-, and 20-fold, and felt that a 3-fold threshold was too stringent, impacting a high proportion of billed items in this cohort (23% overall) that may not represent true coding errors or other coding idiosyncrasies at certain hospitals but rather actual clinically plausible values. Our final methods used a 10-fold threshold, which impacted 6% of total billed items and appeared to address obvious extreme outliers without being too aggressive. In sensitivity analyses, we compared the 3-fold to the 10-fold threshold, and the choice of threshold did not appear to have a meaningful impact on our results (Appendix).

Statistical Analysis

Standard summary statistics were used to describe the study population. We described and contrasted the two methods for estimating resource use in several ways in our cohort. It is important to note, as described previously, that the dollar values generated from these two methods cannot be directly compared. In particular, the data generated from the standardized cost method does not represent a specific dollar amount, but rather is a marker of the sum total of resources utilized that can be compared in a standard way across hospitals. Thus, the comparative data and analyses are presented for illustrative purposes only, in order to describe how the data generated using the standardized cost method relate to values from more traditional calculations using the cost-to-charge ratio methods. Consequently, formal statistical comparisons of the values from the two methods are not made.

We described the distributions of resource use in the overall cohort and by STAT category using both methods. We also performed hospital-level analyses and evaluated how the use of either method impacted the assessment of resource use across hospitals, and in particular the magnitude of variability across hospitals. For this portion of the analysis we fit a hierarchical mixed effects model within the overall cohort, and within each STAT category. To account for the skewed distribution of resource measures, the model assumed a lognormal distribution, and values more extreme than the 1^st or 99^th percentiles within the relevant analysis subgroup were excluded to limit the influence of extreme values. Alternative distributions for cost (gamma, inverse Gaussian, and negative binomial) were evaluated with similar results (Appendix). The model included covariates to account for differences in case-mix across hospitals using the previously published variables from the STS-CHSD risk model (and summarized in the preceding data collection section), along with payer type, race, and year (11). Models also included a random hospital-specific effect to capture between-hospital variability. For each hospital, we calculated an observed to expected ratio. To understand the magnitude of between-hospital variation in case-mix adjusted resource use, we used the model-based estimate of the coefficient of variation (larger values indicating greater variability between hospitals). In addition, hospital classification in resource use tertiles (defined by the 25^th and 75^th percentiles) by either method was assessed. All analyses were performed using SAS v9.4 and R v3.5.2.

RESULTS

Study Population

A total of 43 hospitals (n=65,331) were included. The average annual case volume ranged from 65–985 cases/year (median 316 cases/year). Patient characteristics are described in Table 1. The study population included 20.7% neonates and 26.7% in STAT categories 4 or 5

Table 1.

Study Population Characteristics

Characteristic	Total N=65331
Age at surgery	7.6 mo (2.1 mo – 4.2 yr)
% neonates	13489 (20.7%)
Weight at surgery	7.1 kg (4.0–15.6)
Prematurity*	6677 (18.5%)
Race, white	41178 (67.7%)
Any prior cardiothoracic operation	20776 (32.1%)
Any non-cardiac abnormality	7278 (11.6%)
Any chromosomal anomaly	7992 (12.8%)
Any syndrome	12354 (19.7%)
Any STS-CHSD pre-operative risk factor	22185 (35.1%)
STAT Category
1	18894 (28.9%)
2	21341 (32.7%)
3	7639 (11.7%)
4	14398 (22.0%)
5	3059 (4.7%)
Payer type
Government (Medicaid)	30407 (46.6%)
Commercial	27858 (42.6%)
Other**	7050 (10.8%)

Data are displayed as median (interquartile range) or n (%)

^*Neonates/infants only

^**Includes, Tricare, Medicare, self pay, etc.

Resource Use in the Overall Cohort

We compared values generated from the two resource use methods and distribution in the overall cohort (Table 2). Median values as well as those at the 25^th and 75^th percentiles were all slightly lower with the standardized cost method vs. the cost-to-charge ratio method. This was also true for each STAT category as displayed in Table 2 and Figure 1.

Table 2.

Distribution of Resource Use in the Study Population by the Two Methods

Group	Median ($)	25th – 75th percentile ($)
Overall
Cost-to charge ratio	$57,392	36,618 – 116,962
Standardized cost	$48,696	31,340 – 100,493
STAT 1
Cost-to charge ratio	$37,246	27,474 – 51,852
Standardized cost	$31,739	23,751 – 44,183
STAT 2
Cost-to charge ratio	$52,046	36,262 – 82,686
Standardized cost	$44,072	30,884 – 70,537
STAT 3
Cost-to charge ratio	$71,343	47,551 – 121,519
Standardized cost	$60,732	40,874 – 103,385
STAT 4
Cost-to charge ratio	$119,940	68,347 – 236,067
Standardized cost	$102,739	58,595 – 202,698
STAT 5
Cost-to charge ratio	$196,199	127,569 – 317,451
Standardized cost	$166,755	107,607 – 278,271

Median, 25^th, and 75^th percentiles were all slightly lower with the standardized cost method vs. the cost-to-charge ratio method, both overall and across STAT categories.

Figure 1. Resource Use Across STAT Categories by the Two Methods.

Box and whisker plots showing distribution of resource use by the two methods. The line represents the median, the box the interquartile range, and the upper and lower bars the range, excluding outliers.

Variability across Hospitals

We also calculated hospital-level estimates of resource use adjusted for case-mix using both methods. The distribution across hospitals is shown in Table 3. Variability across hospitals was slightly less using the standardized cost method vs. the cost-to-charge ratio method (coefficient of variation 20% vs. 25%). This pattern was also similar when we examined each individual STAT category (Table 3).

Table 3.

Variation in Case-Mix Adjusted Resource Use Across Hospitals by the Two Methods

Group	Hospital Median ($)	Hospital 25th–75th percentile ($)	Coefficient of Variation (%)
Overall
Cost-to charge ratio	$57,392	48,628 – 67,736	24.9%
Standardized cost	$48,696	42,642 – 55,610	19.9%
STAT 1
Cost-to charge ratio	$37,246	31,888 – 43,505	23.3%
Standardized cost	$31,739	27,834 – 36,191	19.7%
STAT 2
Cost-to charge ratio	$52,046	43,801 – 61,843	26.0%
Standardized cost	$44,072	38,505 – 50,444	20.2%
STAT 3
Cost-to charge ratio	$71,343	60,177 – 84,581	25.6%
Standardized cost	$60,732	53,675 – 68,716	18.5%
STAT 4
Cost-to charge ratio	$119,940	100,646 – 142,932	26.5%
Standardized cost	$102,739	88,932 – 118,689	21.6%
STAT 5
Cost-to charge ratio	$196,199	163,936 – 234,811	27.1%
Standardized cost	$166,755	140,966 – 197,261	25.3%

The coefficient of variation (%) quantifies the magnitude of between-hospital variability in resource use, with a higher value indicating greater variation (see methods for details). Variability across hospitals appeared slightly less using the standardized cost method vs. the cost-to-charge ratio method both overall and across STAT categories.

In addition to assessing overall variability across the cohort, we also sought to understand how the two methods compared for individual hospitals. As shown in Figure 2, case-mix adjusted estimates of resource use for the two methods tended to diverge more frequently at the extremes, most prominently at the higher end of the resource use spectrum. Here, the standardized cost method for many, but not all, hospitals produced lower estimates compared to traditional cost-to-charge ratio methods.

Figure 2. Case-mix Adjusted Resource Use Across Individual Hospitals by the Two Methods.

Observed-to-expected ratios and 95% confidence interval characterizing case-mix adjusted resource use are displayed. Two values are shown for each hospital, one calculated using the cost-to-charge ratio method (black) and the other using the standardized cost method (grey). Hospitals are ordered by increasing case-mix adjusted resource use as assessed by the traditional cost-to-charge ratio method. The two methods tended to diverge more frequently at the extremes, most prominently at the higher end of the resource use spectrum – here the standardized cost method (black) for many, but not all, hospitals produced lower estimates compared to traditional cost-to-charge ratio methods (grey).

We further explored this by categorizing hospitals into resource use groups based on either method. The groups consisted of tertiles defined by the 25^th and 75^th percentiles calculated based on either the standardized cost or cost-to-charge ratio method (Figure 3). One third of hospitals (n=14, 33%) changed resource use classification depending on which method was used. Eleven hospitals (26%) changed by one tertile with some moving to a higher and some moving to a lower tertile. Three hospitals (7%) changed by 2 tertiles and all three moved from the highest resource use tertile when using the cost-to-charge ratio method to the lowest resource use tertile when using the standardized cost method. In evaluating hospitals that changed vs. did not change resource use tertiles we found no significant differences in the characteristics examined including free-standing (vs. children’s hospital within a hospital), U.S. region, surgical volume, and payer mix (Appendix).

Figure 3. Hospital Tertiles for Case-mix Adjusted Resource Use as Assessed by Either Method.

Hospitals were divided into tertiles based on case-mix adjusted resource use, using both methods. Hospitals in bold along the diagonal (n=29, 67%) did not change tertiles regardless of which method was used. Fourteen hospitals (33%) did change: Eleven hospitals (26%) changed by one tertile (some moving to a higher and some moving to a lower tertile), and three hospitals (7%) changed by 2 tertiles (all three moving from the highest resource use tertile when using the cost-to-charge ratio method to the lowest resource use tertile when using the standardized cost method).

DISCUSSION

This study describes the first evaluation of standardized cost methodology in the congenital heart surgery population and impact on characterization of patient- and hospital-level resource utilization. At the patient-level, we found minimal differences in the distribution of resource utilization regardless of which methodology was used (standardized cost vs. traditional cost-to-charge ratios), with slightly lower values with the standardized cost method across the board. When assessing hospitals, the use of standardized cost methodology was associated with reductions in the overall magnitude of resource use variability across institutions, and differences in how approximately one third of individual hospitals were categorized with regard to resource use. These data provide a foundation for better understanding of optimal resource utilization methodology that can be employed in future studies in the congenital heart population and others, and also suggest important implications for future reporting and benchmarking efforts.

While numerous previous studies across a variety of pediatric subspecialties have used the standardized cost methods developed within the PHIS database, ours is the first analysis to date to compare and contrast these methods with traditional methodology to understand the implications (1,12–14). Several strengths and limitations are apparent. First, one of the proposed advantages of the standardized cost methodology is its ability to hone in on resources consumed in providing care through standardizing the cost of items across hospitals, which lessens the influence of other factors that may influence costs estimated through other methods, such as differences across hospitals related to cost-accounting structure, or other costs associated with care delivery beyond simply the number of units or items consumed (1). Our results supported this speculation in that we observed less apparent variability in resource utilization across hospitals when using the standardized methods. Prior studies in the adult population focused on Medicare patients have shown similar findings. Miller and colleagues evaluated Medicare beneficiaries undergoing several different types of common surgery, including hip replacement, coronary artery bypass grafting, back surgery, and colectomy (15). They found that the use of standardized methods similar to those used in our study (but focused on Medicare payments) resulted in less variation across hospitals for all procedures examined. For example, for coronary artery bypass grafting, there was a 70% difference in payments between the upper and lower quintile of hospitals in actual payments, which was reduced to 26% after standardization, indicating less variability across hospitals (15).

In the present study, we further evaluated potential differences in how individual hospitals would be characterized depending on the methodology utilized. We found that approximately one third of hospitals changed categories of resource use, with substantial changes seen in some cases – e.g. moving from being characterized in the highest tertile of resource use with one method to the lowest tertile with the other. These findings emphasize the importance of understanding and selecting the most appropriate method for benchmarking and reporting efforts related to resource use.

Several caveats and potential limitations regarding the standardized cost methodology deserve comment. First, computation and programming were more time intensive and the utility of these efforts may be questioned if the goal is simply population level estimates of distribution, as our estimates using either method in this case were quite similar. Part of the computational work around the standardized method is associated with evaluation of coding discrepancies and outliers. Some of these may be subspecialty specific and clinical input was critical in discerning clinically plausible outlier values. In the case of the congenital heart surgery population, using the initial methods developed across PHIS for standardized costs in the wider general pediatric population resulted in reverting back to the traditional methods in 23% of cases, so that the result was more of a hybrid between the two approaches (1). Further refinement produced a method more consistent with the intended standardized cost approach. This also highlights that the standardized cost methods are not necessarily immune to challenges related to hospitalspecific coding practices and other factors, and that care must be taken with either approach to mitigate these factors. The methods developed in our study may be applied to other studies which could lessen the computational burden for future analyses in this population. Finally, as described in the methods it is important to note that due to the standardization process, the dollar values generated from the standardized method do not represent actual dollars and cannot be directly compared to the traditional values generated from the cost-to-charge ratio method. Instead these represent a different construct and standardized values that allow more direct comparison across hospitals of resources consumed or resource intensity using a similar scale. The standardized cost information may be most useful in studies examining changes in specific resource use over time or across centers. The standardized methods, however, do not include real differences that do exist across hospitals in the costs to deliver care beyond simply the number of units or items consumed. For example, studies conducted from the perspective of society that aim to be inclusive all of these different aspects that influence resource use and dollars spent may not be amenable to the standardized cost methods. Particularly when performing hospital level analyses of variability or characterization of resource use for benchmarking purposes, it is important to understand these nuances select which method is best for the question at hand as this is where we found the greatest differences between the two methods vs. overall population level analyses.

Limitations

The primary limitation of our analysis was that we focused on the subset of US centers that submitted data to both the STS-CHSD and PHIS Database during the study period. No existing dataset captures similar resource use data on the entire cohort of US hospitals performing congenital heart surgery. It is likely that data in our study underestimates the true degree of national variability in resource use as PHIS centers are generally moderate to large academic centers, and that the variability we identified and differences between the two methods analyzed would likely be even greater if inclusive of all centers.

Conclusions

In the first evaluation of standardized cost vs. traditional cost-to-charge ratio methodology in the congenital heart surgery population, we found minimal differences when characterizing the distribution of resource use at the population level. When assessing hospitals, the use of standardized cost methodology was associated with less overall variability across the cohort and impacted the characterization of resource use for approximately one third of centers. Because of these differences, care should be taken in selecting the optimal methodology depending on the question at hand, and when standardized cost methods are utilized our methodology can be applied and adapted in future studies and reporting efforts to reduce computational burden. Future efforts will focus on evaluating these metrics of resource utilization in relation to outcomes and quality measures, and understanding the mechanisms underlying the variability in resource utilization across hospitals.