International Classification of Diseases Codes are Useful in Identifying Cirrhosis in Administrative Databases

Abstract

Background

Health administrative databases are essential to define patient populations, make socioeconomic predictions, and facilitate medical research and healthcare planning. The accuracy of this data is dependent on valid codes/coding algorithms.

Aims

The aim of this study was to systematically identify and summarize the validity of International Classification of Diseases (ICD) codes for identifying patients with cirrhosis in administrative data.

Methods

Electronic databases, MEDLINE (via Ovid), EMBASE (via Ovid), the Web of Science, and CINAHL (via EBSCO-host), were searched for validation studies which compared ICD codes related to cirrhosis to a clinical reference standard, and reported statistical measures of performance.

Results

Fourteen studies were included in the review. There was a large variation in the algorithms used to validate ICD codes to diagnose cirrhosis. Despite the variation, the positive predictive value (PPV) was greater than 84% and the specificity was greater than 75% in the majority of the studies. The negative predictive value (NPV) was lower, but still was associated with values greater than 70% in the majority of studies. Sensitivity data varied significantly with values ranging from 0.27 to 99%.

Conclusions

Evaluated ICD codes for cirrhosis, including codes for chronic liver disease, cirrhosis-specific codes, and cirrhosis-related complications, have demonstrated variable sensitivity and reasonable specificity for the identification of cirrhosis. Additional research is needed to maximize the identification of persons with cirrhosis to avoid underestimating the burden of disease.

Introduction

Liver cirrhosis is the leading cause of death due to digestive diseases globally [1]. In 2017, 1.32 million deaths (representing 2.4% of global mortality) were caused by this condition [2]. Progressing from a compensated stage to a decompensated stage, the more advanced stage of cirrhosis is defined by complications including ascites, esophageal variceal bleeding and hepatic encephalopathy [3, 4]. With decompensation, the median survival drops from an average of 12 years to 2 years, with a median case-fatality rate estimated at 40% in the first year following decompensation [2]. The resource implications of this condition on the healthcare system are tremendous [5]. Hospitalizations are frequent and re-hospitalization rates are estimated at 53% within 90 days at a cost of > $20,000 per admission [6]. With the growing burden of obesity, the prevalence of decompensated cirrhosis related to non-alcoholic steatohepatitis has tripled over the last 30 years [2] and will continue to escalate [7].

Administrative databases include population information that are routinely collected during hospital, clinic, laboratory, or pharmacy visits for use by institutions and healthcare providers across large geographic areas [8]. These databases allow data linkages to occur across a large amount of information—patient demographics, disease diagnoses, heath service utilization (ambulatory billing, medication costs and hospital and procedural costs) and vital statistics data. They are valuable data sources for the purpose of defining patient populations, determining patient outcomes, making socioeconomic predictions from disease epidemiological patterns [9], carrying out healthcare planning, evaluation and medical research [8, 10 11].

There are, however, constraints with the use of administrative data. The information that is collected varies widely by patient population and data elements. Poor rigor and possible inaccuracies in clinical coding transcriptions, incomplete hospital discharge records, or other administrative errors are a potential source of significant bias [8]. Moreover, these databases are only useful if the diagnostic codes that they use are valid (i.e. they identify patients with cirrhosis). This validation can be achieved by comparing diagnostic codes used for disease identification in the database against an accepted clinical reference standard, such as physician charts or medical records [12, 13].

Although several studies in the literature have validated ICD codes for liver cirrhosis in healthcare administrative databases, these studies have been heterogenous with regard to the code(s) or coding algorithms that were used and clinical settings that were evaluated.

In order to provide a more comprehensive overview of what is available in the literature, this systematic review synthesizes the evidence on the validity of International Classification of Diseases (ICD) codes for identifying patients with liver cirrhosis in healthcare administrative databases.

Methods

Search Strategy

This systematic review was guided by the PRISMA checklist [14]. Through consultation with our research librarian, we developed a search strategy by combining MeSH terms and keywords (Supplementary materials—Table S1).

We performed a comprehensive search of MEDLINE (via Ovid), EMBASE (via Ovid), the Web of Science, and CINAHL (via EBSCOhost) to identify relevant articles. All databases were searched from their inception to present day. We examined the reference lists of studies meeting our inclusion criteria to identify further relevant studies. Additionally, we searched for dissertations via ProQuest Dissertations and Theses Global portal, and for other grey literature via Grey Matters (CADTH). There was no restriction on setting, language, or publication date.

Study Selection

Citations identified through the searches were uploaded to EndNote library (Version 9.3). Full text studies were included if they met the following eligibility criteria:

(1) The presence of known cases of compensated liver cirrhosis (characterized by an asymptomatic phase [15]) or decompensated cirrhosis (characterized by the presence of jaundice, ascites, bleeding varices, hepatic encephalopathy, use of spironolactone without alternative indication, or explicit mention of decompensated cirrhosis [16, 17]) in an adult population (≥ 18 years of age); (2) the use of an administrative database as the data source (i.e. databases which routinely and passively collected health information for administrative purposes without a priori research question [12]); (3) the use of a reference clinical standard (such as electronic hospital record, physician chart, outpatient claim data, laboratory test, etc.) for validation of diagnostic codes; and (4) the presence of at least one validity measure for the diagnostic test (such as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV)), or the presence of sufficient data to calculate the validity measures.

We excluded studies that included patients < 18 years of age and studies that did not have full text available in English. Furthermore, we excluded studies in which the databases were not truly administrative, such as epidemiology surveillance systems, or databases that included pre and post data collected for quality improvement programs to assess changes in health service delivery or health status of a targeted patient population [18].

Five reviewers (MD, SR, LB, EE, and AH) independently screened the titles and abstracts of all studies identified through the search (primary screening). This was followed by secondary screening during which three reviewers (MD, SR, and EE) independently reviewed the full text of all studies that met the inclusion criteria in the primary screening, and assessed their eligibility to be included in the final review. Consensus following disagreements was reached through discussion with senior authors (PT, ET). We documented reasons for excluding irrelevant studies in the secondary screening round.

Data Extraction

EE and MD extracted data from all included studies into structured data extraction form in Microsoft Excel, with one row for each study. SR and MD verified the data extracted from the studies for accuracy and completeness. Information extracted from each study includes:

(i) Study characteristics—author(s), year of publication, country, year(s) of data collection, funding source, study design, objective(s); (2) Population characteristics—the target population, sample size or the number of records evaluated, age group(s), gender; (3) Disease characteristics —whether cirrhosis was the primary or secondary diagnosis, definition of the condition, diagnostic code(s) used, the version of ICD used (e.g. ICD-9, ICD-10); (iv) Whether validation was the primary objective (yes/no); (v) The type of administrative database; (vii) The type of the reference standard/ gold standard used to measure the validity of the diagnostic codes; (vii) Characteristics of the test including sensitivity, specificity, as well as PPV, NPV, positive and negative likelihood ratios (if available).

Quality Appraisal

Quality appraisal was conducted on all included studies. Three reviewers (MD, EE, and SR) assessed the risk of bias for each study independently, any major disagreements were resolved by discussion with senior authors (PT, ET). The design and methods of included studies were assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool (available from: http://www.bristol.ac.uk/population-health-sciences/projects/quadas/), which is the modified version of the original QUADAS [19]. QUADAS-2 has been recommended as an appropriate quality appraisal tool for systematic reviews of diagnostic accuracy studies [20, 21], and it has been widely used in previous systematic reviews of diagnostic accuracy studies [22-25]. The tool is comprised of four domains including patient selection, index test, reference standard, and flow and timing. For each study, we documented the risk of bias assessment (high/ low/ unclear) for each domain and concerns regarding applicability (high/ low/ unclear) for the first three domains.

Results

Literature Search and Study Flow

A total of 6551 abstracts were identified with 232 studies reviewed in full text, of which 14 studies met all eligibility criteria (Study flow—Fig. 1).

Fig. 1.

Study Flow: Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) style Flowchart of Study Selection and Review

Study Characteristics

Characteristics of the included studies are presented in Table 1. Together, the studies included data collected from 1975 to 2020. Nine of these studies were conducted in the United States of America [26-34], two in Canada [16, 35], one in Australia [36], one in England [37], and one in Sweden [38]. The mean age was reported in seven of the studies [28, 29, 31, 32, 35, 36, 38] and the median age in three of the studies [33, 34, 37]. The median age ranged from 51 to 69 years. The sample size of the included studies ranged from 100 study participants to 6714 study participants (Fig. 2).

Table 1.

Characteristics of the included studies

Author	Country of Study	Objective of Study1	Patient Population	Did the study include non- cirrhosis patients?	Sample Size	Age Mean (SD)	Male % (n)	Validation Method	Type of ICD Code	Inclusion criteria for ICD codes	Cirrhosis definition based on ICD codes
Blake, 1988	Canada	Primary	Patients with ALD	Yes	108	Biopsy proven cirrhosis: 52.1 (1.4) No cirrhosis: 53.5 (1.8)	52.7 (57)	Biopsy records confirming cirrhosis	ICD-9	Algorithm 1: ≥ 1 ICD-9 codes for CLD or cirrhosis Algorithm 2: ≥ 1ICD-9 codes specifically for cirrhosis	Presence of ≥ 1 of the following codes: 571, 572, and/or 573.9 on death certificates
Chang, 2016	United States	Primary	Primary-care patients	Yes	168	NR	NR	Diagnosis by 1. imaging, 2. 1 code for cirrhosis complications, 3. 1 code for cirrhosis complication + 1 code for CLD and suspicious labs (Bonacini cirrhosis discriminant score > 7)	ICD-9	> 1 ICD-9 codes for CLD	Cirrhosis-candidate primary care patients were patients with ≥ 1 of the following codes: 571.0, 571.2, 571.3, 571.40, 571.41, 571.42, 571.49, 571.5, 571.6, 571.8
Driver, 2019	England	Primary	Hospitalized patients with new diagnosis of HCC	Yes	289	NR (median: 69)	NR	Medical records	ICD-10	Presence of cirrhosis-related codes up to 5 years before HCC diagnosis	≥ 1 inpatient codes including the following: K70.3, K71.7, K72.1, K74.4, K74.5, K74.6, K76.6, K72.1, K72.9, K70.4, K70.9, R18.X, I85.9, I86.4, I98.2, I85.0, I98.3, T46.1, T46.2, J06.1, J06.2, G10.4, G10.8, G10.9, G14.4, G17.4, G43.4, G43.7, J06.1, J06.2, K92.0, K92.1, K92.2
Goldberg, 2012	United States	Primary	In-patients or out-patients cared for at the University of Pennsylvania or Penn Presbyterian Medical Center	Yes	244	NR	NR	Liver biopsy, imaging, physicians documentation on medical records for cirrhosis or decompensation event	ICD-9	≥ 1 ICD-9 in patient or out-patient codes indicative of end-stage liver disease (CLD, cirrhosis, hepatic decompensation)	NR
Kramer, 2008	United States	Primary	Patients with ICD-9 code for HCV or ALD	Yes	331	49.3 (8.6)	98 (324)	Medical records, lab findings, imaging	ICD-9	≥ 1 ICD-9 code for cirrhosis	NR
Rakoski, 2012	United States	Secondary	Patients aged 65 and above that are community dwelling	Yes	100	74.7 (0.4)	NR	Medical records, histological analysis, imaging, and/or evidence of hepatic decompensation events	ICD-9	≥ 1 ICD-9 inpatient or outpatient code for cirrhosis or complications of cirrhosis	NR
Shim, 1991	United States	Secondary	Patients who died between January 1, 1973-December 31 1985, aged 25–54, with liver cirrhosis or related disease reported as underlying cause of death	Yes	1729	NR	68.7 (1187)	Autopsy or biopsy reports, medical records, supportive biochemical tests, imaging	ICD-9, ICD-82	NR	NR
Mapkashi, 2018	United States	Primary	Patients in the Veterans Affairs database with ICD-10 codes for cirrhosis and/or cirrhosis complications	Yes	307	63.8 (7.3)	97 (297)	Medical records, biopsy findings, imaging	ICD-10	≥ 1 ICD-9 code for cirrhosis or complications of cirrhosis	NR
Hayward, 2020	United States	Primary	Inpatients in Australian tertiary care hospital	Yes	540	55.1 (12.8)	68.1 (367)	Medical records	ICD-10 (Australian modification)	≥ 1 ICD-9 code for cirrhosis or complications of cirrhosis	NR
Ramrakhiani, 2020	Australia	Primary	Adult patients from chronic viral hepatitis and nonalcoholic fatty liver disease database	Yes	3396	55.6 (13.9)	52.5 (1784)	Medical records, liver biopsy, fibroscan results, imaging, presence of hepatic decompensation, physician notes	ICD-10	≥ 1 ICD-9 code for cirrhosis or complications of cirrhosis	NR
Nehra, 2013	United States	Primary	Inpatients and outpatients at Parkland Memorial Health and Hospital	Yes	2893	NR (median: 54)	63 (1822)	Medical records, histological analysis, imaging	ICD-9	≥ 1 ICD-9 inpatient or outpatient code for cirrhosis or complications of cirrhosis (excluding code for biliary cirrhosis—571.6)	NR
Lo Re, 2011	United States	Primary	Inpatient and outpatients with ICD-9 codes suggestive of hepatic decompensation (patients are HIV-infected or demographically similar HIV-non uninfected veterans enrolled in the Veterans Aging Cohort study)	Yes	295	NR (median: 51)	98 (288)	Medical records, imaging reports, laboratory data, biopsy, endoscopic reports	ICD-9	≥ 1 ICD-9 code suggestive of hepatic decompensation	NR
Lapointe Shaw, 2018	United States	Primary	HBV, HCV, or viral/non-viral liver disease patients at the Kingston Health Sciences Hepatology Clinic	Yes	6714	NR	NR	Presence or documentation of decompensation events on clinical records, use of spironolactone or nadolol without alternative indication, liver biopsy results, fibroscan results	ICD-9, ICD-10	≥ 1 ICD-9 code for cirrhosis, CLD, or complications of cirrhosis	NR
Bengttsson, 2020	Canada	Primary	Patients registered in the Swedish National Patient Register with diagnoses registered as: cirrhosis without aetiology, alcohol-related cirrhosis, oesophageal varices with or without bleeding, ascites, and hepatocellular carcinoma	Yes	630	64 (22–97)	61 (382)	Medical records, where cirrhosis was defined based off liver biopsy, radiological evidence, presence of ascites or esophageal varices together with a physicians annotation documenting cirrhosis	ICD-10	≥ 1 ICD-10 code for alcohol related cirrhosis, cirrhosis without aetiology, oesophageal varices with or without bleeding, ascites, and hepatocellular carcinoma Subgroup analyses included codes only from hospitalizations	NR

¹Validation of ICD codes as primary or secondary objective of the study

²ICD-8 code validated: 571

Fig. 2.

Scatterplots of Sensitivities, Specificities, PPV, and NPV of Chronic Liver Disease/Cirrhosis and/or Cirrhosis Complication Codes in the Validation of Cirrhosis as Reported by Included Studies. Scatterplot representation of the reported values of common performance metrics (a. sensitivities, b. specificities, c. PPV, and d. NPV) in the validation of ICD codes for cirrhosis. The numerical values, including confidence intervals, are included in Table 3. ¹Ramrakhiani—Model 1: K74.6, ²Ramrakhiani—Model 2: K76.6, ³Ramrakhiani—Model 3: K74.69, ⁴Ramrakhiani—Model 4: K70.30, ⁵Ramrakhiani—Model 5: K70.31, ⁶Ramrakhiani—Model 6: K71.7, ⁷Ramrakhiani—Model 7: Any of the above codes, ⁸Lapointe-Shaw—HBV (cirrhosis)—Model 1: Algorithm 1, ⁹Lapointe-Shaw—HBV (cirrhosis)—Model 2: Algorithm 2, ¹⁰Lapointe-Shaw—HBV (cirrhosis)—Model 3: Algorithm 3, ¹¹Lapointe-Shaw—HCV (cirrhosis)—Model 1: Algorithm 1, ¹²Lapointe-Shaw—HCV (cirrhosis)—Model 2: Algorithm 2, ¹³Lapointe-Shaw—HCV (cirrhosis)—Model 3: Algorithm 3, ¹⁴Lapointe-Shaw—KHSC (cirrhosis)—Model 1: Algorithm 1, ¹⁵Lapointe-Shaw—KHSC (cirrhosis)—Model 2: Algorithm 2, ¹⁶Lapointe-Shaw—KHSC (cirrhosis)—Model 3: Algorithm 3, ¹⁷Lapointe-Shaw—HBV (decompensated)—Model 1: Algorithm 1, ¹⁸Lapointe-Shaw—HBV (decompensated)—Model 2: Algorithm 2, ¹⁹Lapointe-Shaw—HBV (decompensated)—Model 3: Algorithm 3, ²⁰Lapointe-Shaw—HCV (decompensated)—Model 1: Algorithm 1, ²¹Lapointe-Shaw—HCV (decompensated)—Model 2: Algorithm 2, ²²Lapointe-Shaw—HCV (decompensated)—Model 3: Algorithm 3, ²³Lapointe-Shaw—KHSC (decompensated)—Model 1: Algorithm 1, ²⁴Lapointe-Shaw—KHSC (decompensated)—Model 2: Algorithm 2, ²⁵Lapointe-Shaw—KHSC (decompensated)—Model 3: Algorithm 3, ²⁶Kramer—Algorithm 1, ²⁷Kramer—Algorithm 2 (Algorithm 1 + HCV codes), ²⁸Kramer—Algorithm 3 (Algorithm 1 + ALD codes), ²⁹Kramer—Algorithm 4 (Algorithm 1 + HCV + ALD codes), ³⁰Hayward—Algorithm 1, ³²Hayward—Algorithm 2, ³³Bengtsson—Model 1: K74.6, ³⁴Bengtsson—Model 2: K70.30, ³⁵Bengtsson—Model 3: K74.6 (only inpatient codes), ³⁶Bengtsson—Model 4: K70.30 (only inpatient codes), ³⁷Bengtsson—Model 5: K74.6 (internal medicine or transplant clinic), ³⁸Bengtsson—Model 6: K70.30 (internal medicine or transplant clinic), ³⁹Bengtsson—Model 7: K74.6 (university hospital cde), ⁴⁰Bengtsson—Model 8: K70.30 (university hospital code)

In six of the included studies, a specific population was targeted: Blake et al. included only patients with known alcoholic liver disease, Driver et al. included only patients with a new diagnosis of hepatocellular carcinoma, Kramer et al. included patients with codes for Hepatitis C (77.6%) and/or alcoholic liver disease (55.3%), Ramrakhiani et al. included patients from a database including chronic viral hepatitis (33.1% HBV, 15.4% HCV) and non-alcoholic fatty liver disease (51.4%) patients, Lo Re et al. included patients who were HIV infected (50%) or demographically similar to HIV-non infected veterans (37% hazardous alcohol use, 6% HBV, 37% HCV), and Lapointe-Shaw et al. focused on Hepatitis B (HBV) (50.4%), Hepatitis C (HCV) (43.0%), and non-viral liver disease patients followed by a hepatology clinic (6.6%).

The ICD code legend is presented in Table 2. Seven studies used ICD-9 codes [26-29, 33-35], five used ICD-10 codes [31, 32, 36-38], one study used both ICD-8 and ICD-9 codes [30], and one used both ICD-9 and ICD-10 codes [16]. Validation was the primary objective in all but two of the studies [29, 30]. The statistical data included in each study varied: PPV data were included in all but one of the studies [16], NPV data were included in nine of the studies [26, 28, 30, 32-37], nine of the studies included sensitivity data, while eight included specificity data [16, 26, 29, 30, 32-35, 37], and kappa scores were only reported in three of the studies [28, 34, 36].

Table 2.

Administrative data codes used to identify cirrhosis: a. ICD-9 & b. ICD-10

Disease Coded	ICD-9 Codes
(a) ICD-9 codes
Chronic Liver Disease	571
Alcoholic fatty liver	571.0
Acute alcoholic hepatitis	571.1
Alcoholic cirrhosis	571.2
Alcoholic liver damage	571.3
Chronic hepatitis, unspecified	571.40
Chronic persistent hepatitis	571.41
Autoimmune hepatitis	571.42
Other chronic hepatitis	571.49
Cirrhosis, no mention of alcohol	571.5
Biliary cirrhosis	571.6
Other chronic non-alcoholic liver disease	571.8
Unspecified chronic liver disease without mention of alcohol	571.9
Chronic liver disease NOS	571.40, 571.41, 571.8, 571.9, 573.0, 573.3, 575.8, 573.9
Hepatitis B	070.20, 070.21, 020.22, 070.23, 070.3, 070.31, 070.32, 070.33, V02.61
Viral Hepatitis NOS	070.40, 070.49, 070.59, 070.60, 070.70, 070.71, 070.90, 573.1
Hepatitis C	070.41, 070.44, 070.51, 070.54, 070.70, 070.71, V02.62
Hepatitis D with Hepatitis B	070.42, 070.52
Cirrhosis Complications
Esophageal varices with bleeding	456.20, 456.20
Esophageal varices without bleeding	456.1, 456.21
Ascites	789.5, 789.59
Hepatic encephalopathy	572.2
Peritonitis	567.0, 567.2, 567.21, 567.23, 567.29, 567.8, 567.89, 567.9
Hepatorenal syndrome	572.4
Hepatocellular carcinoma	155.x
Paracentesis	54.91
Portal hypertension	572.3
Other sequelae of chronic liver disease	572.8
Hepatopulmonary syndrome	573.5
GI Bleeding	578.0, 578.1, 578.9, 530.82, 531.00, 531.01, 531.20, 531.21, 531.40, 531.41, 531.60, 531.61, 532.00, 532.01, 532.20, 532.21, 532.40, 532.41, 532.60, 532.61, 533.00, 533.01, 533.20, 533.21, 533.40, 533.41, 533.60, 533.61, 534.00, 534.20, 534.21, 534.40, 534.41, 535.01, 535.11, 535.21, 535.31, 535.41, 535.51, 535.61, 537.83
Disease Coded	ICD-10 Codes
(b) ICD-10 Codes
Chronic Liver Disease	K70.0, K70.2, K73.X, K754, K758, K75.9, K76.0, B18.0, B18.1, B18.2, B18.8, B18.9
Alcoholic hepatic failure	K70.4
Alcoholic liver disease	K70.9
Cirrhosis Specific Codes	K70.30, K70.31, K71.7, K72.1, K74.4, K74.5, K74.60, K74.69, K74.3, K72.1, K72.9
Cirrhosis Complications
Ascites	R18.8, K70.31, K70.11
SBP	K65.2, K65.0, K65.9
Varices	I85.9, I85.00, I86.4, I98.2, I85.10
Bleeding varices	I85.01, I98.3, I85.11
Hepatic encephalopathy	K70.41, K72.11, K72.91, B15.0, B16.0, B16.2, 17.11, B19.0, B19.11, B19.21, G31.2, G93.4
Hepatocellular carcinoma	C22.0, C22.9, C22.9
Portal hypertension	K76.6
Hepatorenal syndrome	K76.7
Cirrhosis Treatments
i. Treatment of ascites	T46.1, T46.2, J06.1, J06.2
ii. Treatment of varices	G43.7, J06.1, J06.2, G10.4, G10.8, G10.9, G14.4, G17.4, G43.4
GI hemorrhage	K92.0, K92.1, K92.2

Chart reviews, including biopsy reports, radiological evidence, and/or physician documentation in medical records, were the basis of the gold standard in validation for thirteen of the studies, while one study used death reports as the gold standard [35].

Study Quality

Study quality was evaluated based on the QUADAS-2 tool (Supplementary Materials—Table S2).

Validity of the Cirrhosis Diagnoses

The statistics data for the validity of ICD codes in cirrhosis diagnoses are presented in Table 3. Although many studies had multiple algorithms, only algorithms that were emphasized to be of particular significance by each validation study were used. The PPV was greater than 84% in twelve of the studies and 78% in the remaining one study [33]. In the nine studies in which NPV data were included, NPV was greater than 70% in all included algorithms of seven studies. Of the remaining studies, one study had a NPV of 50.9% [35] and one study had a NPV greater than 70% for only one algorithm, and 60% for the other included algorithm [36] In the nine studies that included sensitivity data, three of the studies had a sensitivity greater than 85% in all included algorithms [30, 33, 37], where the remaining studies had sensitivities ranging from 0.27 to 99%. In the eight studies that included specificity data, six of the studies had a specificity greater than 75% in all included algorithms [26, 30, 32, 34, 35, 37], one had a specificity of 43% [33] and one had a specificity greater than 75% for all but one algorithm [16]. Kappa scores were only reported in three of the studies [28, 34, 36], with scores ranging from 0.48 to 0.79. Due to the heterogeneity of the data, pooled statistical values were not used.

Table 3.

Type of ICD code and validation characteristics

Author	Chronic Liver Disease/Cirrhosis	Cirrhosis Complications	PPV (95% CI)	NPV (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	k
Blake, 1988	X		84.9 (NR)	50.9 (NR)	62.5 (NR)	77.8 (NR)	NR
Chang, 2016	X	X	91.6 (NR)	71.9 (NR)	47.1 (NR)	96.9 (NR)	NR
Driver, 2019	X1	X2	98.8 (95.4–99.7)	78.7 (72.1–84.1)	86 (82–90)	98 (96–100)	NR
Goldberg, 2012 3	X	X	94.3 (NR)	NR	NR	NR	NR
Kramer, 2008
Algorithm 1	X		90 (NR)	87 (NR)	NR	NR	0.7
Algorithm 2 (1 + HCV)	X		92 (NR)	88 (NR)	NR	NR	0.63
Algorithm 3 (1 + ALD)	X		90 (NR)	77 (NR)	NR	NR	0.67
Algorithm 4 (1 + ALD + HCV)	X		93 (NR)	86 (NR)	NR	NR	0.79
Rakoski, 2012	X	X	88 (NR)	NR	67 (NR)	NR	NR
Shim, 1991	X		98.9 (98.3–99.4)	73.7 (69.8–77.3)	93.2 (91.7–94.4)	95.1 (92.0–97.3)	NR
Mapkashi, 2018	X		92.8 (89.4–95.2)	NR	NR	NR	NR
Hayward, 2020
Algorithm 1 4	X		96 (94–98)	60 (53-67)	NR	NR	0.61
Algorithm 2 5	X	X	88 (84–90)	76 (67–84)	NR	NR	0.56
Ramrakhiani, 2020
K74.6	X		89.1 (86.2–91.9)	88.7 (87.6–89.9)	55.7 (52.1–59.3)	98.1 (97.6–98.6)	NR
K76.6		X	94.9 (92.2–97.8)	83.6 (82.3–84.5)	30.5 (27.2–33.8)	99.6 (99.3–99.8)	NR
K74.69	X		93.2 (89.9–96.4)	83.3 (82.0–84.6)	29.3 (25.9–32.5)	99.4 (99.1–99.7)	NR
K70.30	X		100 (93.4–100)	79.3 (77.9–80.7)	7.25 (5.4–9.1)	100 (99.7–100)	NR
K70.31	X		100 (94.0–100)	79.5 (78.1–80.8)	8.05 (6.1–10.0)	100 (99.9–100)	NR
K71.7	X		100 (15.8–100)	78.1 (76.6–79.5)	0.27 (0.03–0.97)	100 (99.9–100)	NR
Any of the above	X	X	88.3 (85.5–91.1)	89.7 (88.6–90.8)	60 (56.5–63.5)	97.8 (97.2–98.3)	NR
Nehra, 2013	X	X6	78.0	91.2	97.9	43.0	NR
Lo Re, 2011 7		X	91	99	33	99	0.48
Lapointe Shaw, 2018 (HBV patients—cirrhosis)
Algorithm 1 8	X		NR	NR	57 (53–60)	96 (96–97)	NR
Algorithm 2 9	X		NR	NR	98 (96–99)	78 (76–80)	NR
Algorithm 3 10	X	X	NR	NR	40 (36–41)	98 (98–99)	NR
Lapointe Shaw, 2018 (HCV patients—cirrhosis)
Algorithm 1 8	X		NR	NR	73 (71–76)	91 (90–93)	NR
Algorithm 2 9	X		NR	NR	99 (98–99)	64 (62–66)	NR
Algorithm 3 10	X	X	NR	NR	55 (52–58)	93 (92–94)	NR
Lapointe Shaw, 2018 (KHSC patients—cirrhosis)
Algorithm 1 8	X		NR	NR	77 (71–82)	92 (89–96)	NR
Algorithm 2 9	X		NR	NR	99 (97–100)	66 (59–72)	NR
Algorithm 3 10	X	X	NR	NR	44 (38–51)	97 (95–99)	NR
Lapointe Shaw, 2018 (HBV patients—decompensated cirrhosis)
Algorithm 1 11	X	X	NR	NR	79 (71–87)	98 (98–99)	NR
Algorithm 2 12	X	X	NR	NR	86 (79–93)	95 (95–96)	NR
Algorithm 3 13	X	X	NR	NR	88 (81–94)	95 (94–96)	NR
Lapointe Shaw, 2018 (HCV patients—decompensated cirrhosis)
Algorithm 1 11	X	X	NR	NR	79 (74–83)	95 (94–96)	NR
Algorithm 2 12	X	X	NR	NR	91 (88–94)	88 (86–89)	NR
Algorithm 3 13	X	X	NR	NR	92 (89–95)	88 (86–89)	NR
Lapointe Shaw, 2018 (KHSC patients—decompensated cirrhosis)
Algorithm 1 11	X	X	NR	NR	89 (83–86)	89 (86–93)	NR
Algorithm 2 12	X	X	NR	NR	99 (97–100)	79 (74–83)	NR
Algorithm 3 13	X	X	NR	NR	99 (97–100)	79 (74–83)	NR
Bengtsson, 2020
K74.6	X		91 (85–95)	NR	NR	NR	NR
K70.3	X		93 (87–96)	NR	NR	NR	NR
K74.6 14	X		95 (86–99)	NR	NR	NR	NR
K70.3 14	X		91 (84–96)	NR	NR	NR	NR
K74.6 15	X		89 (82–95)	NR	NR	NR	NR
K70.3 15	X		94 (88–98)	NR	NR	NR	NR
K74.6 16	X		100 (92–100)	NR	NR	NR	NR
K70.3 16	X		100 (93–100)	NR	NR	NR	NR

¹Algorithm includes codes specifically for alcoholic liver disease and alcoholic hepatic failure

²Ascites code was only included if present pre-HCC diagnosis

³Algorithm includes ≥ 1 code for cirrhosis + ≥ 1 complication code ± ≥ 1 code for chronic liver disease

⁴Algorithm codes defined as Cirrhosis “grouped”: K70.3, K74.4, K74.5, K74.6

⁵Algorithm included Cirrhosis “grouped” codes [4] and codes specifically for alcoholic hepatic failure, HCC, varices, and portal hypertension

⁶Cirrhosis complication codes did not include codes for ascites

⁷Algorithm includes ≥ 1 inpatient or ≥ 2 outpatient codes for ascites, SBP, and/or variceal bleed

⁸Algorithm includes ≥ 1 hospital diagnosis of cirrhosis

⁹Algorithm includes ≥ 1 physician visit of cirrhosis

¹⁰Algorithm includes ≥ 1 hospital diagnosis of cirrhosis or ≥ 2 physician visit of cirrhosis and ≥ 1 complication and ≥ 1 documentation of chronic liver disease

¹¹Algortihm includes ≥ 1 physician visit of cirrhosis and ≥ 1 hospital visit cirrhosis

¹²Algortihm includes ≥ 1 physician visit of cirrhosis and (≥ 1 hospital visit cirrhosis or ≥ 1 procedure)

¹³Algorithm includes ≥ 1 physician visit and (≥ 1 hospital visit cirrhosis or ≥ 1 procedure or ≥ 1 death code)

¹⁴HICD-10 codes from only inpatient encounters

¹⁵ICD-10 codes from internal medicine or transplant clinic

¹⁶ICD-10 codes from university hospital

Eleven studies validated ICD codes by using more than one algorithm, where the remaining three studies grouped one or more ICD codes into a single algorithm [26, 29, 30]. In the three studies that grouped ICD codes into a single algorithm, two used a combination of both chronic liver disease/cirrhosis codes [26, 29], while the remaining study used only chronic liver disease/cirrhosis codes [30]. All three studies had PPV’s greater than 88%, and for the two with specificity data, the specificity was high at values greater than 95%. In the two studies [26, 29] that used a combination of chronic liver disease/cirrhosis codes and complication codes in a single algorithm, the sensitivity was poor with values less than 70%, while the study that used only chronic liver disease/cirrhosis codes had a higher sensitivity at 93.2% [30].

In the studies that used more than one algorithm, the number of algorithms that were used varied from two [35] to twenty-one [33]. The types of algorithms differed among each study, where no study had the exact same algorithms; however, similarities between algorithms were appreciated. Two of the studies used only chronic liver disease codes in their algorithms [28, 35]. Seven of the studies included a combination of chronic liver disease codes and cirrhosis complication codes [16, 27, 31, 33, 34, 36, 37], while two studies had multiple algorithms that focused on only a single ICD code [32, 38]. Of note, one study differentiated the algorithms based on the number of ICD codes provided from the hospital and/or physician visits [16], while another study included laboratory abnormalities in their algorithm [34]. Despite having different algorithmic approaches to validation, in all studies but Nehra et al., the PPV and/or specificity data were strong, while sensitivity was more variable. More specifically, in the two studies [28, 35] that used only chronic liver disease/cirrhosis codes in their algorithms, the PPV ranged from 84.9 to 93%. Of these two studies, only Blake et al. (1988) included specificity and sensitivity data, with a specificity of 77.8% and sensitivity was 62.5%. In the two studies [32, 38] that used only a single ICD code per algorithm, PPV was high and observed to be greater than 88%. Of these two single ICD code per algorithm studies, only one provided specificity and sensitivity data, and it was found that the single ICD codes used had poor sensitivity values ranging from 0.27 to 60%, but high specificity values ranging from 97.8 to 100% [32]. In the remaining seven studies [6, 27, 31, 33, 34, 36, 37] that used a combination of one or more chronic liver disease/cirrhosis codes and complication codes, PPV’s varied from 91 to 98.8%, while specificity values ranged from 64 to 98%, with the majority of specificity readings over 79%. Sensitivity was far more variable with values ranging from 40 to 99%.

Discussion

Administrative data are a rich resource for research and relies on accurate diagnostic coding. Various approaches to the identification of cirrhosis have proliferated the literature. While systematic reviews of the validity of ICD-codes have been performed for many conditions, including heart failure, diabetes, and depression [13, 39, 40], we lack similar syntheses for cirrhosis. In this systematic review, we found that the sensitivity/NPV of the codes utilized varied widely. Conversely, the specificity/PPV of the utilized codes were much better. The consistency of these observations independent of study characteristics suggests that while false-positive identification of cirrhosis is less frequent, additional research is needed to maximize the identification of persons with cirrhosis to avoid underestimating the burden of disease.

When considering the trade-off between sensitivity and specificity, the measure of interest will determine which coding algorithm would be of importance. For example, in the setting where the outcome of interest is identifying cirrhosis in a given patient population, ICD codes with the highest sensitivity should be used to maximize the identification of patients with cirrhosis. Conversely, if medical researchers are interested in cirrhosis as an exposure measure in a given population of patients, one would preferentially choose ICD codes with higher specificity to maximize the likelihood that the individuals included have cirrhosis as the exposure of interest.

Optimizing Coding Algorithms

Although the collective data did not yield a single ideal algorithm, what is suggested by the majority of the papers that validated the use of more than one ICD code is that a combination of codes yields algorithms with high reliability when identifying cirrhosis patients [16, 27, 33, 36, 37]. Combinations may be strengthened by considering their source. For example, Lapointe-Shaw et al. showed that a single outpatient visit code was highly sensitive (> 98%) for cirrhosis while a single inpatient code was specific (> 91%). The denominator is a critical determinant of code performance. When the cohort consists primarily of persons with chronic liver disease, single ICD may be sufficient to accurately identify cirrhosis patients. [32, 38].

Beyond optimizing the use of codes, there are two major ways that have been explored to improve the identification of persons with cirrhosis. First, structured health record data improve coding accuracy. At its most basic implementation, Blake et al. found that the detection of cirrhosis was improved if the word “cirrhosis” was listed in the patient Death Certificate [35]. Chang et al. extended this field with the use of a Natural Language Processing (NLP). In particular, NLP searches of the health record vastly improved the sensitivity/NPV for the presence of cirrhosis [26]. Second, adding laboratory data may be helpful to identify persons with cirrhosis. Lo Re et al. assessed the impact of adding laboratory results to ICD codes in the detection of hepatic decompensation events and found that although using algorithms that used both diagnostic codes and laboratory abnormalities increased the PPV for true events, the algorithm identified less cases than an algorithm that used 1 inpatient and 2 or more outpatient codes for ascites, peritonitis, and variceal hemorrhage. In this context where Lo Re et al. were focusing solely on hepatic decompensation events, severe laboratory derangements may not necessarily equate to hepatic decompensation events in patients with cirrhosis, and therefore not account for all patients with hepatic decompensation. However, the use of laboratory data, in combination with ICD codes, may still be useful in increasing the reliability in identifying patients with cirrhosis, with or without hepatic decompensation.

It is important to note that the patient population can also influence the performance of ICD codes for the identification of patients with cirrhosis. Including patients with known liver disease can maximize the performance of ICD codes. NPV and PPV are dependent on the prevalence of the disease condition in question in a given patient population, and so, in a patient with known liver disease, theoretically, one would expect higher NPV and PPV among this group, versus a generalized population. Interestingly, we found that when comparing studies that included patients with known liver disease [26, 27, 29, 33, 36] versus a generalized patient population [16, 28, 30, 31, 32, 34, 35, 37, 38], the specificity and PPV were similar between the two subgroups; however, the NPV and sensitivity improved when only using a patient population with known liver disease, suggesting that the false negative rate is lower among these patients.

Contextual Factors

These data must be interpreted in the context of the study design. Firstly, we only included papers available with full-text available in English, excluding insights from non-English language journals. Second, it was not our purpose to validate the diagnostic codes for particular complications of cirrhosis (e.g. ascites, variceal bleeding) and therefore these studies were not considered.

Conclusions

These data support the validity of using ICD codes for chronic liver disease, cirrhosis-specific codes, and/or cirrhosis-related complications to identify patients with cirrhosis. The data are useful for researchers who use administrative data sources to identify cirrhosis in patients where validation studies may not have been previously performed. In addition, it provides a concise summary of studies to date that have validated the use of cirrhosis-related administrative codes in specific patient populations and suggests algorithms that may be of interest for future investigators. Due to the heterogeneity of the available data, we were unable to generate a specific ICD code or algorithm that can be used by researchers and healthcare providers in identifying cirrhosis patients by administrative databases. Although the sensitivity/NPV of the codes utilized varied widely, the specificity/PPV of the utilized codes were much better. Supplementing ICD codes with other validation tools, like medical records or laboratory data, can further improve the reliability in correctly identifying cirrhosis patients in a given population. Additional research is needed to maximize the identification of persons with cirrhosis to avoid underestimating the burden of disease.

Abbreviations

ALD

Alcoholic liver disease

GI

Gastrointestinal

HBV

Hepatitis B

HCC

Hepatocellular carcinoma

HCV

Hepatitis C

HIV

Human immunodeficiency virus

ICD

International Classification of Diseases

KHSC

Kingston Health Sciences Clinic

NLP

Natural language processing

NPV

Negative predictive value

PPV

Positive predictive value

QUADAS-2

Quality Assessment of Diagnostic Accuracy Studies