Development and Performance of an Algorithm to Estimate the Child-Turcotte-Pugh Score From a National Electronic Healthcare Database

Abstract

BACKGROUND & METHODS

The Child-Turcotte-Pugh (CTP) score is a widely used and validated predictor of long-term survival in cirrhosis. The CTP score is a composite of 5 subscores, 3 based on objective clinical laboratory values and 2 subjective variables quantifying the severity of ascites and hepatic encephalopathy. To date, no system to quantify CTP score from administrative databases has been validated. The Veterans Outcomes and Costs Associated with Liver Disease study is a multicenter collaborative study to evaluate the outcomes and costs of hepatocellular carcinoma in the U.S. Veterans Health Administration. We developed and validated an algorithm to calculate electronic CTP (eCTP) scores by using data from the Veterans Health Administration Corporate Data Warehouse.

METHODS

Multiple algorithms for determining each CTP subscore from International Classification of Diseases version 9, Common Procedural Terminology, pharmacy, and laboratory data were devised and tested in 2 patient cohorts. For each cohort, 6 site investigators (Boston, Bronx, Brooklyn, Philadelphia, Minneapolis, and West Haven VA Medical Centers) were provided cases from which to determine validity of diagnosis, laboratory data, and clinical assessment of ascites and encephalopathy. The optimal algorithm (designated eCTP) was then applied to 30,840 cirrhotic patients alive in the first quarter of 2008 for whom 5-year overall and transplant-free survival data were available. The ability of the eCTP score and other disease severity scores (Charlson-Deyo index, Veterans Aging Cohort Study index, Model for End-Stage Liver Disease score, and Cirrhosis Comorbidity) to predict survival was then assessed by Cox proportional hazards regression.

RESULTS

Spearman correlations for administrative and investigator validated laboratory data in the HCC and cirrhotic cohorts, respectively, were 0.85 and 0.92 for bilirubin, 0.92 and 0.87 for albumin, and 0.84 and 0.86 for international normalized ratio. In the HCC cohort, the overall eCTP score matched 96% of patients to within 1 point of the chart-validated CTP score (Spearman correlation, 0.81). In the cirrhosis cohort, 98% were matched to within 1 point of their actual CTP score (Spearman, 0.85). When applied to a cohort of 30,840 patients with cirrhosis, each unit change in eCTP was associated with 39% increase in the relative risk of death or transplantation. The Harrell C statistic for the eCTP (0.678) was numerically higher than those for other disease severity indices for predicting 5-year transplant-free survival. Adding other predictive models to the eCTP resulted in minimal differences in its predictive performance.

CONCLUSION

We developed and validated an algorithm to extrapolate an eCTP score from data in a large administrative database with excellent correlation to actual CTP score on chart review. When applied to an administrative database, this algorithm is a highly useful predictor of survival when compared with multiple other published liver disease severity indices.

The Child-Turcotte-Pugh (CTP) score has been the standard assessment of cirrhosis severity for more than 40 years with minimal modifications.^1–4 The CTP currently used is calculated from 5 subscores, 3 based on objective clinical laboratory values, total bilirubin, serum albumin, and international normalized ratio (INR) and 2 subjective variables quantifying the severity of ascites and hepatic encephalopathy (HE) from none to mild (or medically controlled) to severe (or medically refractory). Although highly predictive of surgical risks,^5–9 hospital mortality,^10,11 post-transcatheter arterial embolization mortality,^12,13 transplantation waitlist mortality,¹⁴ and long-term survival¹⁵ in cirrhosis, its major limitations are the dependence on subjective variables and arbitrary laboratory cut points that have never been formally validated.⁴ Because of the subjective nature of 2 of the CTP variables, it is challenging, time-consuming, and costly to manually extract all data required to calculate the CTP from historical data. An electronic means for extracting these, if accurate, would alleviate this problem, allowing identification of CTP scores from electronic records. Such a method might be useful not only for outcomes research but also for health system–based quality management of chronic liver disease patients.

The objectives of this study were to develop operational definitions for the subjective variables, to validate those definitions, and to demonstrate the value of the electronic CTP (eCTP) score in predicting survival in a large cohort of patients with cirrhosis. Secondary objectives were to compare the eCTP score with other measures of liver disease severity, including the Model for End-Stage Liver Disease (MELD) score,¹⁶ and comorbidity indices such as the Charlson-Deyo Index (CDI),¹⁷ the Veterans Aging Cohort (VACS) Index,¹⁸ and the Cirrhosis Comorbidity (CirCom) score¹⁹ in predicting survival in cirrhosis.²⁰

Methods

Data Sources

Investigators from 7 participating VA centers obtained local institutional review board approval to access data from the Computerized Patient Records System and Veterans Health Administration Corporate Data Warehouse (CDW). For patients with 2 outpatient or 1 inpatient International Classification of Diseases version 9 (ICD9-CM) code for cirrhosis (571.2, 571.5, 571.6)²¹ from the period of January 1, 2008 to December 31, 2010, we obtained inpatient and outpatient ICD9-CM codes, Common Procedural Terminology (CPT) codes, pharmacy fill data, and laboratory values from January 1, 2002 to December 31, 2013. The development of hepatocellular carcinoma (HCC) was identified by using ICD9-CM codes (155.0 [malignant neoplasm of liver primary] or 155.2 [malignant neoplasm of liver NOS], excluding 155.1 [malignant neoplasm of intrahepatic bile ducts]).²² Death was ascertained by using the Vital Status File (censoring as of December 31, 2013).²³ Liver transplantation status was obtained by cross-referencing United Network for Organ Sharing and Veterans Administration data.

Derivation and Validation of Electronic Child-Turcotte-Pugh Score Algorithms

We developed several potential algorithms to estimate the 5 CTP subscores by using available data. All variables were extracted for each patient for each quarter after cirrhosis diagnosis, including any quarter in which the patient died or underwent transplantation, starting from the first quarter 2008 (January 1, 2008 to January 31, 2008) to the fourth quarter 2010 (October 1, 2010 to December 31, 2010).

For objective laboratory values, we evaluated by using average quarterly values compared with the result most proximate to the final date of a given quarter. After review of published algorithms²⁴ for identifying hepatic decompensation and with expert consensus, we explored various algorithms to quantify the severity of ascites and encephalopathy with data available in the CDW. The investigators ultimately decided that severe ascites would be best characterized by hospitalization for morbid ascites complications such as spontaneous bacterial peritonitis, hepatorenal syndrome, and refractory ascites requiring transjugular intrahepatic portosystemic shunt (TIPSS) placement, and/or frequent large volume paracenteses. Mild to moderate ascites was defined by any patient requiring diuretic therapy for control of ascites without severe manifestations as noted above. The investigators also concurred that severe encephalopathy would be best characterized by more than 1 hospitalization for encephalopathy within 6 months of or 1 month after the quarter being evaluated. Mild to moderate HE was defined as any patient requiring sustained lactulose and/or rifaximin for control of HE. After considering these operational definitions (Table 1), we ultimately designed 4 different algorithms for ascites and 2 for HE that incorporated pharmacy data, CPT codes, and ICD9-CM codes. All algorithms explored are presented in Supplementary Tables 1 and 2.

Table 1.

Operational Definition of Clinical CTP Subscores

	Score 3	Score 2	Score 1
Ascites	Severe complication of ascites, specifically SBP or HRS,a OR more than 1 LVP,a OR prior CPT code for TIPS	One LVPa; OR 1 ICD9-CM for ascites (inpatient or outpatient); OR prescription fillsb for spironolactone or amiloride	Absence of criteria for Score 2 or 3
Encephalopathy	More than 1 hospitalization for HEa	At least one ICD9-CM for encephalopathy (inpatient or outpatient) not meeting criteria for Score 3; OR prescription fillsb for lactulose or rifaximin	Absence of criteria for Score 2 or 3

HRS, hepatorenal syndrome; LVP, large volume paracentesis; SBP, spontaneous bacterial peritonitis.

^aIn the preceding 6 months or 1 month after index date.

^bThree 1-month or one 3-month fill in the preceding 3 months OR 1 fill in the month after the index date.

To validate the algorithms, 6 site investigator teams were each provided 25 randomly selected cirrhotic cases with HCC to independently extract CTP subscores blind to the algorithm results as a first validation cohort. We targeted review of at least 100 charts distributed over the study sites based on previous similar validations.²⁴ Each of the 6 site investigator teams was then provided 25 randomly selected cases with cirrhosis but without HCC as a second validation cohort. Investigators, who were all experienced hepatologists, were asked to review each chart and to report each subscore during first quarter 2008. Subjective variables were assessed by using standard CTP severity language (None, Mild or medically controlled, Severe or medically refractory). Because of random selection, not all charts had a complete data set for the specified quarter; hence, variable total numbers are reported for each subscore in Table 2. Charts were extracted, and administrative data were compared by using Spearman rank correlation, Spearman rho correlation, and the percentage of overall agreement for the CTP subscores and CTP total score as appropriate.

Table 2.

Validation of Algorithm for Extraction of Clinical CTP Subscores

Cohort	Domain	Algorithm	Exact CTP subscore	±1 CTP subscore	Spearman rho	P value	Unweighted kappa	Weighted kappa
HCC	Ascites	2a	97/115	17/115	0.74	<.0001	0.59 (0.44–0.73)	0.62 (0.49–0.75)
HCC	Encephalopathy	2	104/115	11/115	0.54	<.0001	0.51 (0.26–0.76)	0.51 (0.26–0.76)
Cirrhotic	Ascites	2	95/117	21/117	0.35	<.0001	0.35 (0.13–0.56)	0.36 (0.13–0.59)
Cirrhotic	Encephalopathy	2	106/117	9/117	0.60	<.0001	0.52 (0.24–0.79)	0.51 (0.28–0.73)
HCC	INR	Proximate	85/86	1/86	0.84	<.0001	0.85 (0.56–1.00)	0.85 (0.56–1.00)
HCC	INR	Average	84/86	2/86	0.74	<.0001	0.74 (0.39–1.00)	0.74 (0.39–1.00)
HCC	Albumin	Proximate	84/98	14/98	0.84	<.0001	0.75 (0.64–0.87)	0.79 (0.69–0.89)
HCC	Albumin	Average	75/98	22/98	0.68	<.0001	0.60 (0.46–0.74)	0.64 (0.50–0.77)
HCC	Bilirubin	Proximate	101/105	4/105	0.88	<.0001	0.82 (0.66–0.98)	0.87 (0.74–1.00)
HCC	Bilirubin	Average	101/105	4/105	0.85	<.0001	0.84 (0.69–0.99)	0.85 (0.69–1.00)
Cirrhotic	INR	Proximate	58/60	2/60	0.86	<.0001	0.74 (0.47–1.00)	0.74 (0.47–1.00)
Cirrhotic	INR	Average	59/60	1/60	0.86	<.0001	0.85 (0.55–1.00)	0.85 (0.55–1.00)
Cirrhotic	Albumin	Proximate	73/81	8/81	0.87	<.0001	0.80 (0.68–0.93)	0.84 (0.73–0.95)
Cirrhotic	Albumin	Average	73/81	8/81	0.87	<.0001	0.80 (0.68–0.93)	0.84 (0.74–0.95)
Cirrhotic	Bilirubin	Proximate	84/88	4/88	0.82	<.0001	0.74 (0.52–0.97)	0.82 (0.64–0.99)
Cirrhotic	Bilirubin	Average	84/88	4/88	0.86	<.0001	0.55 (0.29–0.81)	0.67 (0.44–0.89)
HCC	Aggregate	2/Proximate	54/79	22/79	0.81	<.0001	0.55 (0.42–0.69)	0.74 (0.63–0.84)
Cirrhotic	Aggregate	2/Proximate	38/55	16/55	0.85	<.0001	0.55 (0.38–0.72)	0.79 (0.70–0.87)

^aAlgorithms are described in detail in Supplementary Tables 1 and 2.

Validation of Performance of the Electronic Child-Turcotte-Pugh in a Cirrhotic Cohort

From the CDW, we selected non-transplanted patients with a diagnosis of cirrhosis January 1, 2008 through December 31, 2010. Missing laboratory data were imputed as described in Supplementary Methods. To evaluate the performance of the eCTP for predicting 5-year overall survival and transplant-free survival (TFS), we initially selected patients alive during first quarter 2008 (January 1, 2008 to March 31, 2008) whose outcome (death or liver transplantation) was followed until censoring on December 31, 2013. Patients who died or underwent transplantation before March 31, 2008 were coded as surviving 1 day. In this group, we computed survival probabilities with the Kaplan-Meier method and examined the association between the patients’ eCTP score, MELD score, CDI, VACS Index,¹⁸ and CirCom score¹⁹ and TFS rate by using Cox proportional hazards regression modeling in R (survival package). These models included adjustment for age and gender with the exception of the VACS Index, which includes age and gender in its calculation. Results were confirmed by using competing risk models in R (mstate package).

Each model’s overall discriminative ability was evaluated by using the concordance system of Harrell et al.²⁵ The incremental value of the Harrell’s C statistic incurred by adding another predictor to the eCTP model was assessed for MELD score, CDI, VACS Index, and CirCom score, respectively. Furthermore, the added improvement in the prediction of the patient’s survival adding each predictive score/index to a Cox regression model with only age and gender information was quantified by the computation of 2 extra statistics, the integrated discrimination improvement index (IDI) and category-less net reclassification index (NRI).^26,27 Sensitivity analyses were performed by using data from second through fourth quarter 2008 for which at least 5 years of follow-up were available and for shorter follow-up durations (1- to 4–year TFS).

Results

Validation of the Electronic Child-Turcotte-Pugh Score

In total, 4 ascites and 2 HE algorithms that use ICD9-CM, CPT, and pharmacy codes to quantify the presence and severity of these complications were evaluated (Supplementary Tables 1 and 2). The final chosen algorithms (ascites algorithm 2 and HE algorithm 2) are presented in Table 1. For estimation of the ascites subscore, algorithms 1 and 2 performed better than 3 and 4 in both the HCC and cirrhosis groups, possibly because of the lack of specificity of furosemide use for the indication of ascites (data not shown). Ascites algorithms 1 and 2 performed nearly identically, with 84% agreement in the HCC group and 80% agreement in the cirrhosis group with less than 1% misclassification by 2 points on the ascites subscore (Table 2). For estimation of encephalopathy, algorithms 1 and 2 performed similarly, but encephalopathy algorithm 2 classified a higher percentage of patients exactly (90% versus 88% in the HCC set, 91% versus 89% in the cirrhosis set; Table 2).

The concordance with chart-extracted objective laboratory data was strongest for an algorithm that selected the laboratory value most proximate to the quarter end date (algorithm 2) (Table 2, Supplementary Figure 1) rather than quarterly average values. By using this algorithm, there was 96.2% agreement in the HCC set and 95.4% agreement in the cirrhosis set, with zero misclassification by 2 points on the CTP subscore for bilirubin. There was 85.7% exact match in the HCC set and 90.1% exact match in the cirrhosis set, with zero misclassification by 2 points on the CTP subscore for albumin. Finally, there was 98.8% exact match in the HCC set and 96.6% exact match in the cirrhosis set, with zero misclassification by 2 points on the CTP subscore for INR. The model that used the proximate laboratory data performed optimally with the best Spearman and kappa correlations in both the HCC cohort (Spearman r for bilirubin 0.84, albumin 0.92, INR 0.84; κ for bilirubin 0.87, albumin 0.79, INR 0.85) and cirrhosis cohort (Spearman r for bilirubin 0.82, albumin 0.87, INR 0.86; κ for bilirubin 0.82, albumin 0.84, INR 0.74).

Overall performance of the aggregate eCTP score (Table 2, Supplementary Figure 1) with algorithm 2 and proximate laboratory values exactly matched 54 of 79 patients with complete data in the HCC cohort, and 22 of 79 were matched within 1 point of the CTP score; therefore, 96% of the eCPT scores were exact or within 1 point of the chart-derived CTP (Spearman correlation 0.81). In the cirrhosis cohort, 38 of 55 with complete data eCTP scores were exactly matched, and 16 of 55 were matched within 1 point in the chart-derived CTP score; therefore, 98% were within 1 point (Spearman 0.85). Thus, the eCTP closely estimated the chart-derived CTP in the vast majority of patients with minimal clinically significant misclassification.

Application of the Electronic Child-Turcotte-Pugh to the Administrative Cohort

By using the CDW, we identified 30,840 patients who were alive without prior liver transplantation and had a diagnosis of cirrhosis within or before first quarter 2008 (Table 3). The majority of patients were well-compensated and without significant comorbid illness. Male patients comprised 97.2%. The majority of patients were white (72.3%). Half of the cohort (48.8%) was infected with hepatitis C virus; 62.6% had a history of alcohol use (these etiologies were not mutually exclusive). Overall, 18.7% had cirrhosis that could not be attributed to alcohol or viral hepatitis. Median laboratory results were notable for mild thrombocytopenia (median platelet count 139) and mild coagulopathy (median INR 1.2). The median MELD score was 10. Median eCTP class/score was A6, and 43% were eCTP B7 or higher. Comorbidities were modest, with 75th percentile CDI of 1, and 78.8% had CirCom scores of 1+0 or 0. Missingness of certain laboratory data, particularly INR, was common. After imputation, very modest differences in the median values for total bilirubin and INR were noted, but no difference in median eCTP score occurred (Table 3).

Table 3.

Baseline Characteristics of the First Quarter 2008 Cirrhotic Cohort (N = 30,840)

	N	Value
Age, y, median (interquartile range)	30,840	58 (53–64)
Gender
Male	30,006	97.2%
Female	834	2.8%
Race
Native American	273	0.9%
Asian or Pacific Islander	407	1.3%
White	22,288	72.3%
Black	4416	14.3%
Not available	3456	11.2%
Etiology
Hepatitis C	15,047	48.8%
Hepatitis B	1802	5.8%
Alcoholism	19,300	62.6%
Other	5758	18.7%
CDI	30,840	0 (0–1)
Jepsen CirCom Index	30,840
0	17,647	57.2%
1+0	6662	21.6%
1+1	2864	9.3%
3+0	935	3.0%
3+1	2716	8.8%
5+1	16	0.1%
Before imputation
Laboratory within first quarter 2008
Total bilirubin (mg/dL)	23,186	0.9 (0.6–1.5)
Albumin (g/dL)	22,079	3.6 (3.1–4.1)
Creatinine (mg/dL)	24,098	1.0 (0.8–1.2)
Platelet (K/mm3)	23,359	139 (92–201)
INR	12,676	1.2 (1.1–1.4)
AST (IU/mL)	23,900	47 (30–80)
ALT (IU/mL)	24,479	38 (24–65)
Hemoglobin (g/dL)	23,513	13.5 (11.9–14.9)
eCTP	10,812	6 (5–8)
VACS Comorbidity Index	18,374	42 (28–56)
MELD	10,420	10.4 (8.2–14.4)
After imputation
Laboratory within first quarter 2008
Total bilirubin (mg/dL)	30,840	1.1 (0.7–1.9)
Albumin (g/dL)	30,840	3.6 (3.1–4.1)
Creatinine (mg/dL)	30,840	1.0 (1.0–1.0)
Platelet (K/mm3)	30,840	144 (94–205)
INR	30,840	1.3 (1.1–1.7)
AST (IU/mL)	30,840	48 (31–79)
ALT (IU/mL)	30,840	38 (23–64)
Hemoglobin (g/dL)	30,840	13.5 (11.9–14.9)
eCTP	30,840	6 (5–8)
VACS Comorbidity Index	30,840	42 (28–56)
MELD	30,840	12.0 (8.9–15.7)

Univariate Performance of Predictive Models for Overall Survival

For prediction of overall survival, each unit change in eCTP was associated with 23% increase in the relative risk of death (Table 4). The Harrell C statistic for eCTP was numerically higher than those of the VACS Index, MELD, CDI, and CirCom for discriminating survival. For external comparison, the hazard ratio for CDI of 1.18 is nearly identical to the hazard ratio identified by Jepsen et al,¹⁹ although by contrast, CirCom performed less well in our cohort. For prediction of TFS, each unit change in eCTP was associated with 39% increased hazard, and eCTP showed numerically higher concordance by Harrell’s C statistic than VACS, MELD, CDI, and CirCom. When applied to second through fourth quarter 2008, predictive performance for TFS remained similar (Table 4). Similarly, when used to predict TFS, eCTP showed high concordance (Harrell’s C 0.756, 0.717, 0.698, and 0.686 for 1-, 2-, 3-, and 4-year TFS, respectively; Supplementary Table 3), again numerically higher than VACS, MELD, and CDI. The incremental NRI values (presented by estimate [95% confidence interval]) of adding eCTP, VACS, MELD, CDI, and CirCom separately to a Cox model of TFS with age and gender were 0.280 (0.269–0.290), 0.244 (0.233–0.255), 0.160 (0.149–0.170), 0.170 (0.160–0.179), and 0.079 (0.070–0.090) (Table 4, Supplementary Table 4). The corresponding IDI incremental values were 0.109 (0.103–0.115), 0.091 (0.085–0.096), 0.042 (0.038–0.046), 0.049 (0.045–0.0530), and 0.021 (0.018–0.024). eCTP resulted in the highest increment values of both NRI and IDI values compared with other 4 predictors of patient survival. In competing risk models, eCTP showed similarly numerically higher concordance in models predicting TFS and overall survival (Supplementary Table 5).

Table 4.

Cox Regression and Discriminative Statistics for Individual Models Predicting 5-year Overall Survival and TFS

Variables	5-year Overall survival			5-year TFS
	Summary statistic Adjusted HR (95% CI)	P value	Harrell’s C statistic C (error)	Summary statistic Adjusted HR (95% CI)	P value	Harrell’s C statistic C (error)	Harrell’s C statistica C (error)	NRI NRI (95% CI)	IDI IDI (95%CI)
eCTP (first quarter 2008)b	1.23 (1.22–1.24)	<.0001	0.649 (0.003)	1.39 (1.37–1.40)	<.0001	0.680 (0.002)	0.665 (0.005)	0.280 (0.269–0.290)	0.109 (0.103–0.115)
VACS (per 10 units)	1.23 (1.23–1.24)	<.0001	0.643 (0.003)	1.27 (1.26–1.28)	<.0001	0.656 (0.002)	0.635 (0.005)	0.244 (0.233–0.255)b	0.091 (0.085–0.096)b
MELDb	1.05 (1.04–1.05)	<.0001	0.618 (0.003)	1.06 (1.06–1.07)	<.0001	0.623 (0.002)	0.649 (0.005)	0.160 (0.149–0.170)	0.042 (0.038–0.046)
CDIb	1.18 (1.17–1.20)	<.0001	0.610 (0.003)	1.33 (1.32–1.35)	<.0001	0.622 (0.002)	0.623 (0.005)	0.170 (0.160–0.179)	0.049 (0.045–0.0530)
CirCom Indexb			0.590 (0.003)			0.593 (0.002)	0.584 (0.005)	0.079 (0.070–0.090)	0.021 (0.018–0.024)
0	Reference			Reference
1+0	0.79 (0.75–0.83)	<.0001		1.29 (1.23–1.34)	<.0001
1+1	0.95 (0.89–1.02)	.156		1.68 (1.59–1.77)	<.0001
3+0	0.69 (0.61–0.77)	<.0001		1.22 (1.11–1.33)	<.0001
3+1	1.05 (0.99–1.12)	.117		1.98 (1.88–2.09)	<.0001
5+1	5.18 (3.01–8.93)	<.0001		7.32 (4.41–12.15)	<.0001
eCTP (second quarter 2008)b				1.39 (1.38–1.41)	<.0001	0.680 (0.002)		0.279 (0.267–0.289)	0.111 (0.105–0.117)
eCTP (third quarter 2008)b				1.39 (1.38–1.41)	<.0001	0.685 (0.002)		0.291 (0.279–0.301)	0.118 (0.112–0.124)
eCTP (fourth quarter 2008) b				1.39 (1.37–1.41)	<.0001	0.684 (0.002)		0.292 (0.282–0.303)	0.117 (0.111–0.123)

CI, confidence interval; HR, hazard ratio.

^aConcordance in subset of 9358 patients without imputation required for eCTP, MELD, and VACS Index.

^bAdjusted for age and gender.

Additive Performance of Predictive Models

Combining each predictive model, except the VACS model, to eCTP resulted in negligible differences in its ability to predict overall survival (Table 5). Addition of VACS increased the concordance of the overall model by 2.77%. For prediction of TFS, addition of both MELD (+2.65%) or VACS (+1.91%) to eCTP modestly increased model concordance.

Table 5.

Effect of Additive Predictive Models on eCTP Performance

Variables	5-year Overall survival		5-year TFS
	Harrell’s C statistic	Difference (%)	Harrell’s C statistic	Difference (%)
eCTPa	0.649 (0.003)	—	0.679 (0.002)	—
eCTPa + MELD	0.649 (0.003)	0.00	0.697 (0.002)	2.65
eCTPa + CDI	0.655 (0.003)	0.92	0.679 (0.002)	0.00
eCTPa + VACS	0.667 (0.003)	2.77	0.692 (0.002)	1.91
eCTPa + CirCom	0.654 (0.003)	0.77	0.685 (0.002)	0.88

^aAge and gender were adjusted.

Actual One-, Two-, Three-, and Five-year Transplant-free Survival in the Cohort by Electronic Child-Turcotte-Pugh Score

As shown in Table 6, one-year TFS for patients classified as eCTP A (5 or 6), eCTP B (7–9), or eCTP C (10–15) was 94.6%, 81.8%, and 51.3%, respectively; 2-year TFS was 86.5%, 66.8%, and 35.5%, respectively. Kaplan-Meier curves for each eCTP score are shown in Supplementary Figure 2. For specific subgroups of patients, specifically those with severe ascites, severe hepatic encephalopathy, and coagulopathy but not patients with chronic kidney disease, eCTP much more closely estimated TFS than MELD or VACS (Supplementary Table 6).

Table 6.

Actual 1-, 2-, 3-, and 5-year Overall TFS in Cohort by eCTP

eCTP	N	TFS (%)
		Year 1	Year 2	Year 3	Year 5
5	9585	96.25	90.04	82.56	69.61
6	7975	92.61	82.28	71.03	54.46
All CTP A	17,560	94.60	86.51	77.32	62.73
7	5511	86.97	73.60	60.04	42.86
8	3446	79.40	63.84	49.97	35.35
9	2114	72.47	54.02	41.86	28.52
All CTP B	11,071	81.84	66.82	53.44	37.78
10	1172	64.16	46.33	34.73	23.04
11	567	45.86	30.86	19.93	13.76
12	268	32.84	18.66	14.18	8.21
13	133	21.80	12.03	9.77	6.77
14	59	6.78	0.00	0.00	0.00
15	10	0.00	0.00	0.00	0.00
All CTP C	2209	51.29	35.49	25.85	17.16
Total	30,840	86.92	75.79	65.06	50.51

Discussion

Although the CTP score was developed to predict portosystemic shunt surgery outcomes in cirrhotic patients,^1,3 historically it has remained the most widely used staging system to predict long-term survival in cirrhosis.² Despite widespread acceptance, several flaws of CTP as a prognostic staging system have been identified. First, it relies on 2 subjective variables (without prior operational definitions), fostering interobserver variation and hampering application to large data sets. Second, the arbitrarily chosen cut points for the objective laboratory variables have never been validated. Third, for the purpose of transplant allocation, the narrow range of classification (ABC) precluded adequate patient stratification.²⁹ In this study, we demonstrate that by creating operational definitions that are based on ICD diagnostic codes, CPT procedure codes, and laboratory and pharmacy refill data, CTP can be accurately estimated (eCTP). This algorithm accurately predicts TFS in a large cohort of patients with cirrhosis and can be used in future epidemiologic studies to assess the impact of various interventions (eg, antiviral therapy) on cirrhosis outcomes.

The subjectivity of the ascites and encephalopathy subscores within the CTP score is often cited as its major limitation. Clinical definitions of ascites are variable. For instance, a small amount of perihepatic ascites on an ultrasound may be considered significant to some clinicians and not to others.^2,4 In addition, patients with insomnia or vague irritability may be started on lactulose and/or rifaximin for presumed early HE, whereas others may not receive treatment until their first presentation with frank asterixis. Not unexpectedly, the kappa coefficient for ascites determined administratively compared with values determined by clinicians was the lowest among the subscores. We suspect that the primary driver for this discordance was the use of furosemide and spironolactone for the management of preascitic edema by hepatologists. Although our algorithm may overestimate the ascites subscore for these individuals, it is also likely that these individuals have early portal hypertension–related circulatory dysfunction and indeed are at increased risk for subsequent infectious and hepatorenal complications. Overall, the variability within this one subscore is minimized when the composite model is applied.

By applying eCTP to the largest cohort of cirrhotic patients studied to date, we demonstrate that eCTP has the strongest capacity to predict overall survival and TFS relative to other predictors of liver-related mortality, MELD score and VACS Index, and relative to 2 predictors of non-hepatic mortality, CDI and CirCom score. Our estimates for 1-year and 2-year cumulative TFS for eCTP A, B, and C (94.6%, 81.8%, and 51.3%, respectively, 1-year TFS; 86.5%, 66.8%, and 35.5%, respectively, 2-year TFS) are nearly identical to those identified in the largest meta-analysis to date (95%, 80%, and 45% 1-year TFS; 90%, 70%, and 38% 2-year TFS),² despite the higher median age of this cohort.

eCTP was markedly more accurate at predicting 5-year overall survival and TFS compared with the solely laboratory-based MELD score. Although MELD has shown superiority over CTP in predicting short-term (<1 year) mortality, most studies similarly have shown its inferiority in predicting long-term mortality.⁴ Although serum creatinine in MELD would be expected to correlate to some degree with hepatorenal physiology and ascites formation, no component in MELD acts as a surrogate for HE, a critical complication of cirrhosis for which MELD has been shown to underestimate mortality.^30,31 Although it is tempting to attribute the inferiority of MELD in predicting 5-year TFS to the lack of inclusion of encephalopathy in MELD, excluding the encephalopathy subscore from eCTP had only a minimal impact on its concordance in our data set (data not shown). However, for patients with severe HE, eCTP was a much better predictor of outcome than MELD or VACS (Supplementary Table 6). We did not evaluate eCTP as a predictor of short-term outcomes.

The limitations of this study include the use of a complex algorithm that requires access to both laboratory and pharmacy data in addition to more commonly accessible ICD9-CM and CPT codes. Therefore, replicating this study requires comprehensive administrative data sets. Internally, when applied to different quarters in our cohort, the performance of eCTP is identical. External validation of the eCTP score in non-–Veterans Administration administrative data sets will be critical for broad application of this approach. In the future, we will also explore potential simplification of the algorithm. The performance of the eCTP score compared with other disease severity models must take into context the purpose of each model. The CirCom score was designed specifically to assess comorbidities in cirrhotic patients; the VACS score was derived from a human immunodeficiency virus–positive population in whom liver disease is a major comorbidity. The CDI examines overall comorbidity and its effect on survival. Because of the negligible additive benefits of combining the eCTP score with any of these models, it appears that including non-hepatic comorbidity has limited impact on predicting survival in patients with cirrhosis. An additional potential limitation of our study design was that clinicians who evaluated the severity of HE and ascites from clinical charts were also initially involved in creating the operational definitions. When chart extraction was performed by these experienced hepatologists more than 6 months later, the case report form did not include the operational definitions but instead requested 1–3 scoring that was based on standard CTP language (eg, None, Mild or medically controlled, Severe or poorly controlled).

Conclusion

We developed and validated an algorithm to calculate an “electronic” CTP score from data in a large administrative database with excellent correlation to actual CTP score on chart review. When applied to an administrative database, this algorithm may be a highly useful predictor of survival for patients with advanced liver disease.

Abbreviations used in this paper

ALT

alanine aminotransferase

AST

aspartate aminotransferase

CDI

Charlson-Deyo Comorbidity Index

CDW

Corporate Data Warehouse

CPT

Common Procedural Terminology

CTP

Child-Turcotte-Pugh

eCTP

electronic Child-Turcotte-Pugh

CirCom

Cirrhosis Comorbidity

HCC

hepatocellular carcinoma

HE

hepatic encephalopathy

ICD9-CM

International Classification of Diseases, version 9

IDI

integrated discrimination improvement index

INR

international normalized ratio

MELD

Model for End-Stage Liver Disease

NRI

net reclassification index

TFS

transplant-free survival

TIPSS

transjugular intrahepatic portosystemic shunt

VACS

Veterans Aging Cohort Study

VOCAL

Veterans Outcomes and Costs Associated with Liver Disease