Measuring the Value of a Clinical Practice Guideline for Children with Perforated Appendicitis

Abstract

Objective:

To determine the incremental cost-effectiveness of a clinical practice guideline (CPG) compared to “usual care” for treatment of perforated appendicitis in children. Secondary objective was to compare cost analyses using hospital accounting system data versus data in the Pediatric Health Information System (PHIS).

Summary Background Data:

Value-based surgical care (outcomes relative to costs) is frequently touted, but outcomes and costs are rarely measured together.

Methods:

During an 18-month period, 122 children with perforated appendicitis at a tertiary-referral children’s hospital were treated using an evidence-based CPG. Clinical outcomes and costs for the CPG cohort were compared to patients in the 30-month period prior to CPG implementation (n=191 children).

Results:

With CPG-directed care, intra-abdominal abscess rate decreased from 0.24 to 0.10 (aRR, 0.44 [95% CI, 0.26–0.75]). The rate of any adverse event decreased from 0.30 to 0.23 (adjusted risk ratio, 0.82 [95% CI, 0.58–1.17]). Mean total hospital costs per patient (hospital accounting system) decreased from $16,466 to $10,528 (adjusted absolute difference, (−$5,451) [95% CI, (−$7,755) – (−$3,147)]), leading to estimated adjusted total savings of $665,022 during the study period. Costs obtained from the PHIS database also showed reduction with CPG-directed care (−$6,669 [95% CI, (−$8,949) – (−$4,389)] per patient). In Bayesian cost-effectiveness analyses, likelihood that CPG was the dominant strategy was 91%.

Conclusions:

An evidence-based CPG increased the value of surgical care for children with perforated appendicitis by improving outcomes and lowering costs. Hospital cost accounting data and pre-existing cost data within the PHIS database provided similar results.

Mini-Abstract

Our purpose was to determine the incremental cost-effectiveness of a clinical practice guideline (CPG) compared to “usual care” for treatment of perforated appendicitis in children. CPG-directed care was associated with improved patient-centered outcomes, reduced hospital costs, and increased hospital margin for treatment with children with perforated appendicitis.

1. INTRODUCTION

Increasing the value of medical care—i.e., healthcare outcomes relative to cost—has paramount importance with rising costs and variable quality of healthcare.^1–3 Hospitals and healthcare providers are being challenged to deliver higher value care via regulatory requirements and consumer demand. The Affordable Care Act and bundled payments provide incentives for better outcomes and decreased cost.^{4, 5} Further, consumers are starting to demand affordability and transparency regarding the value of healthcare treatment options.^{6, 7} However, there has been limited effort to measure the cost-effectiveness of surgical interventions.⁸

Healthcare systems must be able to relate cost of care to clinical outcomes in order to determine value⁵; however, the true cost of care, and thus value, for most medical conditions remains unknown.^{9, 10} There are several barriers to acquiring cost data.¹⁰ Many healthcare organizations lack cost accounting systems to accurately determine patient-level costs. Costs are often aggregated into departments or particular service areas rather than at the patient-level, which prohibits determining the value of care.¹ Furthermore, an appropriate measure of value requires measuring the total costs of the full cycle of care for the patient’s medical condition. To measure these costs, a patient’s care must be followed longitudinally.^{1, 10} There is also disagreement on which costs should be included and uncertainty regarding how to account for costs of care from all providers during the cycle of care.¹⁰ Our experience and anecdotal reports suggest that acquisition of cost data by providers and researchers is difficult in most healthcare systems. Access to these cost data is often limited to personnel in finance departments without responsibility or training to relate costs to outcomes. However, cost data may be available in national administrative datasets that also provide patient and diagnosis specific data.

Formal economic analyses are needed to assess the value of clinical practice guidelines (CPGs), defined by the Institute of Medicine as “systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances.”^11–14 CPGs are known to decrease the variability of care in certain settings although their effects on surgical outcomes and their cost effectiveness have received little study. Skarda et al previously reported that standardization of intraoperative disposable device utilization and post-operative antibiotics decreases costs in surgical treatment of appendcitis.^{15, 16} Our own institution’s CPG-guided treatment for perforated appendicitis resulted in significantly improved clinical outcomes, but evidence is lacking on how CPG-guided treatment for perforated appendicitis affects costs or value of care.¹⁷

While the value of healthcare has been conceptualized as outcomes/cost, formal economic analyses are necessary to quantify and value the consequences of health interventions.¹⁸ Cost-effectiveness analysis (CEA) typically quantitates the incremental cost (savings) of intervention per unit of benefit, e.g., the cost per additional case detected, complication prevented, hospital day avoided, life saved, life-year gained, or disability-free year gained. Cost-utility analysis (CUA) quantitates consequences in terms of preference-based measures of health, such as quality-adjusted life-years gained. In cost-benefit analysis (CBA), consequences are valued in monetary units.¹⁸

The goal of this study was to determine if a CPG is cost-effective for preventing adverse events (AEs) in children undergoing treatment for perforated appendicitis at a tertiary-referral children’s hospital. A secondary goal is to investigate whether two different cost data sources, hospital accounting data and a nationwide administrative database, provide similar estimates of the value of the treatment we are examining. If these two cost sources provide similar estimates, this finding would suggest that publicly available databases could be used by researchers and other stakeholders to more readily estimate costs of other treatments.

2. METHODS

Cost-effectiveness was defined as incremental cost from a health system perspective per averted AE, using two different sources of cost data. The comparison treatment was care as provided prior to implementation of the CPG. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement was utilized to guide consistent and transparent reporting of our economic evaluation.¹⁹

2.1. Clinical Practice Guideline

The CPG was developed for treating children with complicated appendicitis at our institution.¹⁷ The Institute of Medicine guidelines for CPG development and a multi-disciplinary approach were utilized. The treatment protocol emphasized early laparoscopic appendectomy, conversion from intravenous piperacillin-tazobactam to oral ciprofloxacin and metronidazole once a regular diet was tolerated, avoidance of peripherally inserted or other central venous catheters, and administration of antibiotics for 7 days after discharge (Figure 1).¹⁷ On post-operative day 7, if the patient was not improving clinically, the CPG indicated computed tomography (CT) of the abdomen and pelvis should be considered. Importantly, indications for post-operative CT were exactly the same before and after the CPG. To be considered CPG-adherent, a patient had to receive the appropriate antibiotics including pre- and post-operative piperacillin-tazobactam and discharged with 7 days of ciprofloxacin/metronidazole irrespective of total days of inpatient antibiotics, not have a white blood count checked prior to discharge to determine duration of antibiotic therapy, and have a scheduled post-operative clinic appointment within two weeks of discharge. These criteria were thought to be most likely associated with clinical outcomes.

Figure 1.

Clinical Practice Guideline for Complicated Appendicitis.

Represents the guideline implemented during the study period for all patients with gangrenous or perforated appendicitis.

Reference: Willis ZI, Duggan EM, Bucher BT, et al. Effect of a Clinical Practice Guideline for Pediatric Complicated Appendicitis. JAMA Surg 2016:e160194.

2.2. Subjects and Setting

The study population consisted of all children with perforated appendicitis treated at Monroe Carell, Jr. Children’s Hospital, a 271-bed, freestanding, tertiary referral center affiliated with Vanderbilt University Medical Center (VUMC) in Nashville, TN from January 1, 2011 to December 31, 2014. Study time periods were predefined before collecting outcome and cost data and were based on clinical judgment regarding the time periods that would be both adequate and meaningful to compare. Patients prior to CPG implementation (pre-CPG cohort) were treated according to individual surgeon preference (“usual care”). A 30-month period prior to the implementation of a CPG was used as the control period. All records with an ICD-9 diagnosis code of 540.0 (i.e., acute appendicitis with generalized peritonitis) or 540.1 (i.e., acute appendicitis with peritoneal abscess) were ascertained. Charts were reviewed manually by a single reviewer with a second reviewer (MLB) involved to address uncertainty. Children were excluded if there was no mention of appendiceal perforation in the operative report or surgeon’s progress notes. The CPG for the treatment of complicated (gangrenous or perforated) appendicitis was implemented on July 1, 2013, at which point cases and outcomes were prospectively measured for 18-months (post-CPG cohort). Appendiceal perforation was determined intra-operatively by the attending pediatric surgeon as a visible hole in the appendix or extruded fecalith. Due to the subjective nature of gangrenous appendicitis, only patients with perforated appendicitis were included in this analysis.

The study was approved by the institutional review board at VUMC, which waived a requirement for informed consent.

2.3. Assessment of Clinical Outcomes

AEs were predefined as surgical site infections (SSIs), Emergency Department (ED) visits, hospital readmissions, additional operative or interventional radiology (IR) procedures, or other complications of treatment. All AEs documented in the electronic medical record within 30 days of appendectomy were included in analysis.¹⁷ Length of hospital stay was measured as the time from admission order to discharge order in minutes.

2.4. Costs

Total hospital costs from a health system perspective for the entire episode of care for children with perforated appendicitis were calculated in both the pre-CPG and post-CPG cohorts using hospital accounting data (HA) and PHIS data. The episode of care included the initial admission until 30 days after appendectomy similar to prior studies. Inpatient, observation unit, and ED costs for this time period were included. Net profit to the hospital was estimated by subtracting HA costs from the payments received for each episode of care. We inflated all costs and revenue (pre and post-CPG for both HA and PHIS) to 2016 U.S. dollars based on the Consumer Price Index for medical services.²⁰ Physician, clinic, outpatient pharmacy, home health, and CPG development or implementation costs were not included in our analysis.

2.5. Hospital accounting method

Finance departmental personnel (AA) obtained financial data for the episode of care for each patient in the two cohorts from the hospital’s internal cost accounting system. The data were obtained from the institution’s internal costing system, Transition Systems Inc. (TSI), which provides patient-level costs, independent of charge data, integrated into a single database.²¹ The data were stored in an Oracle based Enterprise Data Warehouse and were extracted using SQL Developer. Total costs, revenues, and margin data were acquired. The same cost accounting system was used in all patients.

2.6. PHIS cost analysis method

The Pediatric Health Information System (PHIS) database includes data from 48 United States not-for-profit, freestanding children’s hospitals with inpatient, ED, ambulatory surgery, and observation unit patient encounters.²² In addition to clinical data, resource utilization data including total and departmental charges and/or costs are reported in PHIS. The Children’s Hospital Association manages the database with the goal of quality improvement and comparison across institutions.²³

PHIS data include total RCC (ratio of costs to charge) based costs and adjusted total RCC based costs for each hospital.²⁴ Charges extracted from the database included total billed charges, which were sub-grouped into clinical, imaging, lab, pharmacy, supply, and other (mainly room, nursing, operating room, and ED) charges. For each cost sub-group for the patient encounters, cost was determined using the appropriate RCC provided by the PHIS database, specific for each year and departmental category.²⁴ The use of department-specific RCCs is considered to provide more accurate cost estimates than use of the overall hospital cost-to-charge ratio. RCC costs were also adjusted by the CMS wage/price index for the hospital’s location.^{25, 26}

2.7. Statistical Analyses and Economic Evaluation

All statistical and economic analyses were conducted using multilevel generalized estimating equation (GEE) models with an exchangeable correlation (to account for surgeon clustering) and robust standard errors. Differences in the risk of AEs between pre- and post-CPG cohorts were assessed using a binomial GEE model with log link. Differences in days in the hospital, costs, and revenues before and after CPG implementation were assessed using a GEE model with log-link and gamma distribution. Differences in net profit to the hospital (revenues minus costs) were estimated using a GEE with identity link and Gaussian distribution. All the models were adjusted for age, gender, race, insurance status, and identification of intra-abdominal abscess on pre-operative CT/ultrasound (US).

We used a Bayesian approach to assess the cost-effectiveness of the CPG relative to usual practice. We used a Bayesian bivariate model of total costs and AEs to account for potential correlation between these measures.^{27, 28} The AEs were modeled using a log binomial model adjusting for age, gender, race, insurance status, and identification of intra-abdominal abscess on pre-operative CT/US. A Gamma model with identity link was used for total costs to account for its skewed distribution. The Gamma model included the same covariates, a random intercept for surgeon (to account for within-surgeon correlation), and a term for AEs to model the correlation between cost and effectiveness. We used weakly informative priors for the AEs to exclude large treatment effects.²⁹ We calculated the probability of the intervention being dominant, with dominance defined as resulting in reduction of both AEs and costs.

A p-value of less than .05 was considered statistically significant. All frequentist statistical analyses were performed using an intention-to-treat analysis with Stata version 13.1 (Stata Corp, College Station, TX). The Bayesian cost-effectiveness model was implemented via Markov Chain Monte Carlo (MCMC) methods in OpenBUGS (Bayesian inference Using Gibbs Sampling).³⁰ We used 3 MCMC chains with 20,000 iterations after an initial 4000 iterations. Trace plots of all parameters were monitored for convergence. We additionally calculated the convergence diagnostic of Gelman-Rubin for all parameters.

3.0. RESULTS

3.1. Demographic and Clinical Data

A total of 191 patients (61%) were in the “usual practice” cohort, prior to the start of the CPG. A total of 122 patients (39%) were studied after the start of the CPG. Clinical and demographic characteristics of the two cohorts were similar at baseline (Table 1). The study population predominantly consisted of children who were White (63%), male (59%), and had public insurance (59%) (Table 1). The majority of the patients (87%) underwent laparoscopic appendectomy. A total of 11 surgeons performed the appendectomies included in the study, three of them only in the period prior to the start of the CPG, and two of them only after the start of the CPG. Adherence to the CPG by surgeons, i.e. appropriate antibiotics ordered and follow-up appointment scheduled, was 88%.¹⁷ In the post-CPG cohort, 48/122 patients (39.3%) had a drain placed intra-operatively for an identified abscess. Rate of drain placement prior to implementation of the CPG was not recorded.

Table 1.

Characteristics and Clinical Data of Patients by Treatment Group

	Usual Practice (n = 191)	CPG-Directed Practice (n = 122)	p-value
Age, mean years (SD)	8.8 (4.0)	8.7 (4.1)	0.78
Male sex, No. (%)	111 (58)	74 (61)	0.66
Race/Ethnicity, No. (%)			0.22
Non-Hispanic White	116 (61)	80 (66)
Hispanic	44 (23)	31(25)
African American	23 (12)	7 (6)
Other	3 (2)	4 (3)
Unknown	5 (3)	0 (0)
Insurance Status, No. (%)			0.33
Public insurance	117 (61)	68 (56)
Nonpublic insurance	74 (38)	54 (44)
Intra-abdominal abscess on pre-operative CT/US, No. (%)	45 (24)	26 (21)	0.64
Type of surgery, No. (%)			0.85
Open	11 (6)	8 (7)
Laparoscopic	167 (87)	104 (85)
Laparoscopic converted to open	13 (7)	10 (8)

CPG = Clinical Practice Guideline, CT = Computed Tomography, US = Ultrasound

3.2. Adverse Events

Following the implementation of the CPG, the rate of AEs decreased from 0.30 to 0.23 (adjusted risk ratio [aRR], 0.82 [95% CI, 0.58 – 1.17]) (Table 2). Analyzing each AE separately, SSI and additional operative or IR procedures were significantly reduced (Table 2). With CPG-directed care, the rate of any SSI was reduced from 0.26 to 0.11 (aRR, 0.50 [95% CI, 0.31–0.81]), and more specifically the intra-abdominal abscess rate decreased from 0.24 to 0.10 (aRR, 0.44 [95% CI, 0.26–0.75]). The rate of additional operative or IR procedures decreased from 0.09 to 0.04 (aRR, 0.44 [95% CI, 0.21–0.92]) (Table 2). Similar results (not shown) were obtained in analyses confined to the patients of surgeons on staff throughout the study period.

Table 2.

Clinical Outcome Measures by Treatment Group

	Usual Practice (n = 191) Mean (SD)	CPG-Directed Practice (n = 122) Mean (SD)	Adjusted Risk Ratioa Mean (95% CI)	p-valuea
Any adverse event	0.30 (0.46)	0.23 (0.42)	0.82 (0.58–1.17)	0.27
ED visit	0.14 (0.35)	0.12 (0.33)	0.85 (0.48–1.48)	0.56
Readmission	0.16 (0.37)	0.12 (0.36)	0.76 (0.37–1.5)	0.45
Additional operative or interventional radiology procedure	0.09 (0.29)	0.04 (0.20)	0.44 (0.21–0.92)	0.03
Surgical site infection	0.26 (0.44)	0.11 (0.32)	0.50 (0.31–0.81)	<0.01
Organ-space (intra-abdominal abscess)	0.24 (0.43)	0.10 (0.30)	0.44 (0.26–0.75)	<0.01
Incisional (superficial or deep)	0.02 (0.13)	0.02 (0.13)	1.02 (0.27–3.89)	0.98
Others	0.02 (0.14)	0.05 (0.22)	2.25 (0.78–6.49)	0.13

SD = Standard Deviation, CPG = Clinical Practice Guideline, ED = Emergency Department

^aRisk Ratio of CPG-based practice vs. usual practice adjusted for age, gender, race, insurance status, identification of intra-abdominal abscess on pre-operative CT/US, and within-surgeon correlation.

3.3. Resource Utilization and Hospital Financial Analysis

The mean length of hospital stay per patient was reduced from 6.5 days in the pre-CPG period to 5.5 days after the implementation of the CPG (adjusted absolute difference [aAD], (−0.87) [95% CI, (−1.64) - (−0.08)]) (Table 3). The estimated cost per hospital day was also reduced from $2,557 in the pre-CPG cohort to $1,958 in the post-CPG cohort (aAD, (-$572) [95% CI, (-$748) - (-$397)]). The mean total hospital costs per patient were reduced from $16,466 in the pre-CPG cohort to $10,528 following the CPG implementation (aAD, (-$5,451) [95% CI, (-$7,755) - (-$3,147)]) (Table 3). In Bayesian cost-effectiveness analyses, the probability that the CPG was the dominant strategy (i.e. led to reduction of adverse events with a reduction of costs) was 91%. Of note, by excluding the 5 surgeons who only treated patients in either the pre- or post-CPG periods (42 patients), the magnitude of the aAD in cost per patient among the post-CPG cohort versus the pre-CPG cohort was similar to the overall reduction ((-$5,870) [95% CI, (-$8,558) - (-$3,182)]) (analysis not shown). With the implementation of the CPG, the revenues per patient decreased to $15,495 from $17,495 (aAD, (-$ $2,554) [95%CI, (-$4,261) - (-$848)] (Table 3), whereas the estimated net profit to the hospital increased to $4,966 per patient in the post-CPG cohort compared to $1,028 per patient in the pre-CPG cohort (aAD, $3,118 [95% CI, $1,131 - $5,106]).

Table 3.

Resource Utilization and Hospital-Derived Financial Data by Treatment Group

	Usual Practice Mean, [Median, (IQR)] (n =191)	CPG-Directed Practice Mean, [Median, (IQR)] (n=122)	Adjusted Absolute Differencea [95% CI]	p-valuea
Hospital days per patient	6.5 [5.7 (4.0–7.8)]	5.5 [4.9 (3.2–6.3)	−0.87 [(−1.64) – (−0.08)]	0.03
Estimated cost per hospital dayb	$2,557 [$2,226 ($2,492 – $2,840)]	$1,958 [$1,902 ($1,732–$ 2,195)	−$572 [(−$748) – (−$397)]	<0.01
Estimated total costs per patientb	$16,466 [$13,553 ($9,983 – $18,235)]	$10,528 [$9,642 ($8,073–$11,615)]	−$5,451 [(−$7,755) – (−$3,147)]	<0.01
Total revenue per patientb	$17,495 [$14,173 ($9,545 – $23,193)]	$ 15,495 [$11,994 ($8,775–$20,113)]	−$2,554 [(−$4,261) – (−$848)]	<0.01
Total net profit per patientb	$1,028 [$(645), ($5,999 – $7,849)]	$ 4,966 [$2,491, ($2,157 – $11,490)]	$3,118 [$1,131 – $5,106]	<0.01

IQR = Interquartile Range

^aAbsolute difference of CPG-based practice vs. usual practice adjusted for age, gender, race, insurance status, identification of intra-abdominal abscess on pre-operative CT/US, and within-surgeon correlation.

^bInflated to 2016 US Dollars.

Using HA data, the only factor found to correlate with total cost of treatment of perforated appendicitis per patient other than CPG status (before or after) was the presence of an abscess on pre-operative imaging, which was associated with a 36% increase in total hospital costs (p = 0.001) (analysis not shown).

3.4. PHIS Cost Analysis

All study patients had records identified in PHIS with administrative data available. Total estimated costs were decreased to an average cost per patient of $14,183 in the post-CPG patient population vs. $21,179 in the pre-CPG cohort (aAD, (-$6,669) [(-$8,949) - (-$4,389)]) (Table 4). Total estimated costs adjusted by the CMS wage/price index for the hospital’s location were also significantly decreased from $22,335 pre-CPG to $15,147 post-CPG intervention (aAD, (-$6,835) [(-$9,272) - (-$4,399)]) (Table 4). In addition, all sub-group analyses revealed a significant decrease in costs except for the categories of “other” and “imaging” costs. The most significant reduction percentagewise was in supply costs, with a 99% reduction in costs per patient post-CPG (p < 0.001) (Table 4). Clinical and lab costs were also reduced by 77% (p < 0.001) and 39% (p < 0.001) per patient, respectively (Table 4).

Table 4.

PHIS-Derived Cost Data by Treatment Group

Costsa, Mean, [Median, (IQR)]	Usual Practice (n=122)	CPG-Directed Practice (n =191)	Adjusted Absolute Differenceb [95% CI]	p-valueb
Total	$21,179 [$17,294 ($12,557–$27,031)]	$14,183 [$12,746 ($9,757–$15,402]	(−$6,669) [(−$8,949) – (−$4,389)]	<0.01
Total Adjusted	$22,335 [$18,250 ($13,241–$28,806)]	$15,147 [$13,585 ($10,409–$16,423)]	(−$6,835) [(−$9,272) – (−$4,399)]	<0.01
Clinical	$735 [$198 ($70–$ 739)]	$125 [$65 ($0–$186)]	(−$564) [(−$785) – (−$344)]	<0.01
Imaging	$764 [$214 ($0–$1,177)]	$507 [$142 ($69–$923)]	(−$221) [(−$451) – $9)]	0.06
Lab	$271 [$153 ($78–$303)]	$134 [$112 ($54–$188)]	(−$106) [(−$134) – (−−$79)]	<0.01
Pharmacy	$2,087 [$1,569 ($965–$2,570)]	$1,383 [$1,140 ($833–$1,656)]	(−$649) [(−$1,055) – (−$243)]	<0.01
Supply	$2,013 [$2,033 ($1,193–$2,816)]	$31 [$0.001 ($0–$0.01)	(−$1,993) [(−$2,241) – (−$1,746)]	<0.01
Other	$12,232 [$10,075 ($7,814–$14,652)]	$10,981 [($9,809 ($8248–$11,801)]	(−$1,063) [(−$2,476) – $350]	0.09

IQR = Interquartile Range

^aInflated to 2016 US Dollars.

^bAbsolute difference of CPG-based practice relative to usual practice adjusted for age, gender, race, insurance status, identification of intra-abdominal abscess on pre-operative CT/US, and within-surgeon correlation.

4.0. DISCUSSION

Implementation of a CPG at our institution increased the value of surgical care for children with perforated appendicitis by improving outcomes and lowering costs. The rates of intra-abdominal abscess and of additional operative or IR procedures decreased significantly by 56%. Among patients with a post operative intra-abdominal abscess, there was no difference in the proportion that had a drainage procedure in the two cohorts (40% overall). There was a decrease in hospital stay by nearly 1.0 day. Overall, total costs decreased by 33%, nearly $5,500 per patient, leading to an estimated total savings of $665,022 during the study period. Resource utilization was also significantly decreased.¹⁷ Our almost 90% surgeon adherence rate was achieved by monthly individual and overall patient outcome and CPG adherence feedback, recognition of surgeons with the highest adherence rates, and presentation of cost savings.¹⁷ Using a CEA, we have demonstrated significant value of our intervention, the CPG. The CPG is determined to be the dominant therapy (better outcomes AND lower costs).^{31, 32}

Our analysis addressed whether changes designed to improve one outcome worsen other outcomes.¹⁷ The encouragement to avoid interval appendectomy with the CPG did not result in more conversions to open procedures, and shorter length of hospital stay in the post-CPG cohort did not result in more ED visits or readmissions.

Cost accounting in healthcare has never been more important, but is often considered too difficult, expensive, or resource intensive.⁵ Reimbursement is evolving towards value-based payment models that reward high quality and lower cost with shared savings.^{4, 33} Valid and reliable methods to easily evaluate the value of care are needed. In our experience, the HA data were more difficult to acquire than the publicly available PHIS data; yet the intervention was found to be dominant by both methods. Although the reductions in both PHIS and HA costs per patient were similar percentagewise, the magnitude of the adjusted absolute difference was substantially larger for PHIS-based costs per patient ($6,669 vs. $5,451 for HA-based total costs). HA data allowed assessment of hospital net profit. Estimated hospital profits tripled, increasing over $3,100 per patient after the CPG was implemented, resulting in approximately $380,000 or more of increased hospital profits during the 18-month study period. Publicly available databases, such as PHIS, do not report payments and thus although the PHIS financial data was more easily accessible, the lack of revenue is a significant limitation. Ability to show an increase in profit incentivizes hospitals to implement future CPG initiatives that both improve outcomes and reduce costs. Although not implemented in our CPG, providers could also be incentivized if afforded reimbursements for increasing overall hospital profits.

Our study has several limitations. This is a single-institution study of a relatively small number of patients; however, the development and implementation of the CPG should be generalizable to larger cohorts. As opposed to parallel group randomized trials in which both known and unknown potential confounders are likely to be distributed evenly between the treatment groups, the before-and-after design is more likely to be biased by unidentified confounders that potentially contributed to the observed reduction in adverse events and costs. Our analyses were adjusted for recognized potential sources of bias (including demographic and clinical factors, as well as within-surgeon correlation) but some unrecognized biases may still remain. However, the CPG is the major factor contributing to our findings. The retrospective review is limited by the lack of reviewer blinding to the study. Furthermore, our CPG did not standardize intra-operative resource utilization and the analysis did not include provider/professional costs. With standardization of intraoperative supplies, cost reduction would likely have been even greater. We did not perform additional analyses to determine impact of specific components of the CPG on outcomes, such as intra-operative drain placement, and this remains a focus of future studies. An additional limitation is that costs associated with CPG development and implementation were not formally calculated. However, costs of CPG development may be considered an inherent cost of optimal medical care. Although this study utilized two cost measures, both have limitations. The HA system can potentially have measurement error introduced at multiple stages of the cost calculation.²¹ The PHIS database consists of cost and charge data reported by healthcare institutions, potentially incorrect and limited in ability to accurately represent actual costs.³⁴ However, both databases showed the CPG to be the dominant treatment strategy, suggesting publicly available databases may be a reliable resource for comparing various treatments.

In conclusion, an evidence-based CPG can increase the value of surgical care as measured by its incremental cost-effectiveness for children with perforated appendicitis as assessed using both internal hospital accounting and PHIS cost data. Patient-level cost data obtained from a publicly available database resulted in similar conclusions as a hospital cost accounting system. Future research should be performed to verify the generalizability of our findings to other centers and to determine which CPGs improve the value of treatment of other surgical or medical conditions.