Validation of coding algorithms for the identification of patients with primary biliary cirrhosis using administrative data

Abstract

BACKGROUND:

Large-scale epidemiological studies of primary biliary cirrhosis (PBC) have been hindered by difficulties in case ascertainment.

OBJECTIVE:

To develop coding algorithms for identifying PBC patients using administrative data – a widely available data source.

METHODS:

Population-based administrative databases were used to identify patients with a diagnosis code for PBC from 1994 to 2002. Coding algorithms for confirmed PBC (two or more of antimitochondrial antibody positivity, cholestatic liver biochemistry and/or compatible liver histology) were derived using chart abstraction data as the reference. Patients with a recorded PBC diagnosis but insufficient confirmatory data were classified as ‘suspected PBC’.

RESULTS:

Of 189 potential PBC cases, 119 (60%) had confirmed PBC and 28 (14%) had suspected PBC. The optimal algorithm including two or more uses of a PBC code had a sensitivity of 94% (95% CI 71% to 100%) and positive predictive values of 73% (95% CI 61% to 75%) for confirmed PBC, and 89% (95% CI 82% to 94%) for confirmed or suspected PBC. Sensitivity analyses revealed greater accuracy among women, and with the use of multiple data sources and one or more years of data. Inclusion of diagnosis codes for conditions frequently misclassified as PBC did not improve algorithm performance.

CONCLUSIONS:

Administrative databases can reliably identify patients with PBC and may facilitate epidemiological investigations of this condition.

Primary biliary cirrhosis (PBC) is a chronic cholestatic disorder characterized by nonsuppurative destruction of the interlobular and septal bile ducts, which may progress to cirrhosis (1). The hallmark serological feature is the presence of antimitochondrial antibodies (AMAs) (2). In general, PBC is considered to be a rare disease predominantly affecting women. However, incidence and prevalence figures have varied from two to 49 cases per million, and 19 to 402 cases per million, respectively (3,4). Contemporary data describing the epidemiology of PBC in Canada are limited; only two population-based studies have been reported (5,6). In the first, Witt-Sullivan et al (5) surveyed 502 Ontario physicians regarding their patients with PBC. In 1987, the incidence and prevalence of AMA-positive, biopsy-proven PBC were 3.3 and 22.4 per million, respectively. In a Quebec study from the early to mid-1980s, Villeneuve et al (6) reported incidence and prevalence rates of 3.9 and 25.4 per million, respectively. Population-based studies describing the natural history of PBC are also limited (7,8). This paucity of data is partly explained by the rarity of PBC and the complexity of its diagnosis, which requires clinical, biochemical, serological and, in some cases, histological data. These problems are compounded by the difficulty of collecting data from multiple sources – which can be time-consuming, expensive and difficult over prolonged periods – and the requirement for collaboration among providers spanning large geographical areas.

Administrative databases, which are used in all areas of health care financing and delivery, represent an alternative data source that may overcome these limitations. Health care providers, policy-makers and payers use administrative data for reimbursement, budgetary planning, monitoring clinical activities, measuring the quality of care and health services research (9,10). The critical variable in these applications is the patient diagnosis, typically recorded using the International Classification of Diseases (ICD) – Ninth Revision, Clinical Modification (ICD-9-CM) (11) or 10th Revision (ICD-10) (12) coding systems. These data can be used to identify specific patient cohorts and assess disease epidemiology, risk factors and outcomes. Clearly, the accuracy and completeness of diagnoses within these databases is vital to reaching valid conclusions (13). As such, the validation of administrative data has been the focus of several investigations, typically via medical record audits (14–26). Although administrative databases have been used in several studies to help identify patients with PBC (6,7,27–35), their accuracy has not been rigorously evaluated. In the majority of these reports, multiple additional case-finding approaches have been used, including surveys, transplant registries, death certificates, histology databases and laboratory reports for positive AMA serology. Although such multifaceted approaches to case ascertainment may maximize sensitivity, administrative databases have the advantage of broad geographical coverage, relatively complete capture of health care encounters and limited expense (9). In addition, because administrative databases are ubiquitous, they may facilitate comparisons of PBC across regions with variable access to other data sources. To embark on such studies, the accuracy of a PBC diagnosis based on administrative data must be confirmed. Therefore, the objective of the present study was to validate diagnostic coding algorithms for PBC using three population-based administrative databases for use in future epidemiological studies.

METHODS

Data sources

The present study used administrative data to identify potential cases of PBC in the Calgary Health Region (CHR) between fiscal years 1994 and 2002 (April 1, 1994, to March 31, 2003). The CHR is one of the largest fully integrated, publicly funded health care systems in Canada, and provides all medical and surgical care to residents of Calgary and surrounding communities in southern Alberta (population approximately 1.1 million in 2002). Contained in the region are 12 academic and community hospitals, including three adult hospitals within the city of Calgary. Three databases were used to identify potential PBC cases (36). These databases have been used to examine the epidemiology (37–39), outcomes (40–43) and coding accuracy (14,17–20,37) of a variety of medical conditions.

Physician claims database

The physician claims database records claims submitted for payment by Alberta physicians for services provided to registrants of the Alberta Health Care Insurance Plan. Approximately 4500 providers submit more than 36 million claims annually (36). Each record in the database includes up to three diagnosis fields, the date of service and the specialty of the care provider.

Inpatient discharge abstract database

The inpatient discharge abstract database contains patient demographic, diagnosis, procedure and mortality information on all discharges from hospitals within the CHR. These data are routinely transmitted to the Canadian Institute for Health Information for aggregation with nationwide hospitalization data (36). Chart validation studies have shown rates of agreement exceeding 95% for demographic data and 75% to 96% for most responsible diagnosis codes (44).

Ambulatory care classification system database

The ambulatory care classification system (ACCS) database contains information on facility-based ambulatory care, including clinic and emergency department visits, same-day surgery, day procedures and rehabilitation services. Data are available from fiscal year 1996 onward (36).

Study population

The administrative database population included adults 20 years of age and older with at least one health care encounter in which an ICD-9-CM (571.6) or ICD-10 diagnosis code for PBC (K74.3) was recorded during the study interval (11,12). Whereas the ICD-10 code is specific to PBC, the ICD-9-CM code also codes for ‘biliary cirrhosis’. Therefore, this code may misclassify cases of secondary biliary cirrhosis as PBC. Date of birth and sex were extracted from the Alberta Health Care Insurance Plan Registry, which contains demographic details on more than 99% of Alberta residents who participate in this government-administered universal health care plan (36). To calculate the sensitivity of the administrative data, a cohort of 17 well-characterized PBC patients who participated in two clinical trials for PBC at the University of Calgary (Calgary, Alberta) were included (45,46). All patients were women and were diagnosed before or during the study interval. Sixteen of the 17 patients (94%) had definite or probable PBC (see case definitions for PBC below). The remaining patient, who relocated to the CHR after her PBC diagnosis by a hepatologist, was classified as having suspected PBC because the diagnostic details could not be confirmed.

Validation study

The validation study was designed to develop coding algorithms for diagnosing PBC using administrative data. A unique patient identifier in the administrative databases enabled linkage with medical records that included the outpatient charts of all hepatologists and gastroenterologists practicing at the University of Calgary Medical Clinic. Due to the rarity of PBC and the potential requirement for liver transplantation, most patients are referred to a hepatologist at some point during the course of their disease. All CHR hepatologists practice at this clinic. Inpatient medical records from the three adult acute care hospitals in Calgary were also reviewed. Charts were reviewed by a trained physician using a structured data collection instrument.

Case definitions for PBC in medical records

Using chart review data, the strength of each PBC diagnosis from the administrative data was graded as definite, probable, suspected, not PBC or unconfirmed. A diagnosis of PBC was considered definite when all three of the following criteria were met: cholestatic liver biochemistry (ie, raised serum alkaline phosphatase and/or gamma glutamyl-transpeptidase concentration), positivity for AMA (titre 1:40 or higher) and/or antibodies against the pyruvate dehydrogenase complex (2,47,48), and compatible liver histology (49). Probable PBC was defined when any two of these criteria were met. Because it is widely accepted that fulfillment of at least two of these criteria is confirmatory of PBC (1,34), the primary outcome measure was definite or probable PBC. The date of diagnosis was defined as the earliest date at which the patient was found to have fulfilled any two of these criteria (50). A PBC diagnosis was considered suspect if any physician note (eg, admission history, progress note or discharge summary) stated that a patient had PBC. Although not a rigorous definition, it was hypothesized that misclassification would be minimal due to the rarity of this disease and that patients would be unlikely to state that they had PBC unless they were truly afflicted with the condition. Similarly, a physician would be unlikely to record this condition if uncertain of the diagnosis. Therefore, as a secondary outcome measure, the presence of definite, probable or suspected PBC was considered. A diagnosis was considered to be not PBC if there was clear evidence of an alternative hepatic condition. Finally, a diagnosis was considered unconfirmed if insufficient data were available to assign a particular diagnosis.

Administrative data coding definitions

A variety of coding definitions as predictors of a diagnosis of PBC were examined. Data from all three databases, combined and individually, were used. For the inpatient discharge abstract and ACCS databases, the presence of at least one and at least two encounters, respectively, with a code for PBC were considered. Because professional health records coders input these data, it was assumed that misclassification was minimal. For the physician claims database, the following case definitions were examined: at least one claim by any physician, at least one claim by a general practitioner (GP), at least one claim by a specialist and at least two claims by any physician. Because PBC is an uncommon disorder typically managed by specialists, it was hypothesized that specialists would be more accurate than GPs in coding. Moreover, because PBC is a chronic disease, it was hypothesized that multiple uses of the codes over a prolonged period of time would be associated with greater accuracy. Therefore, sensitivity analyses were conducted to determine the effect of the interval between the first and second health care contact (within one, two and three years). Because PBC predominantly affects women, sex-specific sensitivity analyses were also conducted. Finally, the databases for diagnosis codes of conditions commonly misclassified as PBC were queried to determine if they could be used to improve the diagnostic accuracy of the algorithms. Specifically, the codes for primary sclerosing cholangitis (PSC, ICD-9-CM 576.1; ICD-10 K83.0), secondary biliary cirrhosis (ICD-10 K74.4, K74.5) and autoimmune hepatitis (AIH, ICD-9-CM 571.4; ICD-10 K73.x, K75.4) were searched.

Statistical analyses

Using data obtained from medical records as the gold standard, the positive predictive values (PPVs) (with exact binomial CIs) of the administrative data coding definitions for the diagnosis of PBC were calculated. Due to the absence of an unselected control group, specificities and negative predictive values could not be determined. However, the sensitivities of these definitions were calculated using the aforementioned cohort of 17 PBC clinical trial patients (see Study population in the Methods section) (45,46). The Appendix includes a glossary of the statistical terminology.

Descriptive statistical methods were used to describe the characteristics of the study cohort. Comparisons between groups were made using Fisher’s exact and χ² tests for categorical variables, and Mann-Whitney and Kruskal-Wallis rank tests for continuous variables. Statistical analyses were performed using Intercooled Stata 10.0 (StataCorp, USA) and SAS 9.1.3 (SAS Institute, USA) software. The study protocol was approved by the Conjoint Health Research Ethics Board of the University of Calgary.

RESULTS

Study population

Between 1994 and 2002, there were 1387 ‘hits’ or ‘contacts’ in the administrative data including a diagnosis code for PBC among 325 individuals. A flow diagram of the derivation of the study population is presented in Figure 1. The majority of contacts (84%) were identified from the physician claims database. Of the 325 patients, the medical records of 198 (61%) were available for review. According to the PBC case definitions, 21% had definite PBC, 39% had probable PBC, 14% had suspected PBC and, in one case (0.5%), a hepatic diagnosis could not be established (ie, ‘unconfirmed PBC’). Fifty patients (25%) had a liver condition other than PBC. The most frequently misclassified diseases were PSC (n=14 [28%]), AIH (n=10 [20%]), secondary biliary cirrhosis (n=5 [10%]), hepatitis C (n=5 [10%], four of whom had cirrhosis), alcoholic cirrhosis (n=4 [8%]) and cryptogenic cirrhosis (n=4 [8%]). One patient had PSC/AIH overlap syndrome.

Figure 1).

Flow diagram illustrating the derivation of the study population. ACCS Ambulatory care classification system; PBC Primary biliary cirrhosis

Characteristics of the study population

Demographics and details of the administrative data according to disease classification are outlined in Table 1. The data for the single patient with ‘unconfirmed PBC’ were excluded from the table. Compared with patients with definite or probable PBC (n=119), those with other liver conditions (n=50) were more likely to be men (9% versus 50%; P<0.00001). For example, 87% of PSC cases were male versus only 9% with definite or probable PBC (P<0.00001). The proportion younger than 40 years of age was also higher among PSC patients (33% versus 14%; P=0.07). The sex and age distributions of AIH and PBC cases did not differ significantly. Among seven men younger than 40 years of age – for which PBC is uncommon – six (86%) had conditions other than PBC (PSC [n=4]; PSC/AIH [n=1], AIH [n=1]). In contrast, among 21 women in this age group, 17 patients (81%) had definite, probable or suspected PBC. Compared with patients with definite or probable PBC, those with other diagnoses had fewer total physician claims and inpatient contacts for PBC. However, the number of contacts for alternative diagnoses (PSC, AIH and secondary biliary cirrhosis) did not differ between groups.

TABLE 1.

Characteristics of the study population

Characteristics	Definite or probable PBC (n=119)	Suspected PBC (n=28)	Not PBC (n=50)
Demographic characteristics
Female sex	91 (108)*	82 (23)*	50 (25)
Age, years
At first contact	52 (44–63)	57 (49–72)*	50 (42–63)
20–39	14 (17)	4 (1)	20 (10)
40–59	50 (60)	50 (14)	52 (26)
60–79	34 (40)	36 (10)	24 (12)
≥80	2 (2)	11 (3)	4 (2)
Administrative data coding
Total PBC contacts	4 (2–8)*	4 (2–8)*	2 (1–4)
≥2 contacts	82 (98)*	79 (22)*	30 (15)
Total PBC claims	3 (2–6)*	3 (1–5)*	1 (1–1)
Total inpatient PBC contacts	0 (0–0)	1 (0–2)*	0 (0–0)
Total ACCS PBC contacts	0 (0–1)*	0 (0–0)	0 (0-0)
≥2 non-PBC contacts	14 (17)	18 (5)	22 (11)
PSC	8 (10)	18 (5)	12 (6)
Autoimmune hepatitis	2 (2)	4 (1)	4 (2)
Secondary biliary cirrhosis	7 (8)	11 (3)	2 (1)

Data presented as median (interquartile range) or proportions % (n).

^*P<0.05 for comparison with patients with a liver condition other than primary biliary cirrhosis (PBC) (ie, ‘Not PBC’). ACCS Ambulatory Care Classification System; PSC Primary sclerosing cholangitis

In patients with definite or probable PBC, the median age at diagnosis was 52 years (interquartile range [IQR] 44 years to 63 years) and 91% were women. The majority (79%) were AMA positive (median titre 1:640 [IQR 1:160 to 1:640]). An additional nine patients (8% of those with definite or probable PBC) were antipyruvate dehydrogenase complex positive (E2 positive, n=9; X positive, n=4). The median (IQR) serum alkaline phosphatase, alanine aminotransferase, and bilirubin concentrations at diagnosis were 268 U/L (176 U/L to 373 U/L), 67 U/L (45 U/L to 100 U/L), and 11 μmol/L (7 μmol/L to 15 μmol/L), respectively. The diagnosis of PBC was histologically confirmed in 60 patients (50%).

Validity of administrative data for definite or probable PBC

Of the 198 patients with at least one contact for PBC, 119 had definite or probable PBC (PPV 60%; 95% CI 53% to 67%). This definition was 94% sensitive (95% CI 71% to 100%) for the 17 PBC clinical trial patients. The median delay between the diagnosis of PBC and the first administrative data contact was 54 days (IQR zero to 309 days). The PPV of the administrative data increased and the sensitivity decreased as the number of contacts necessary to confirm PBC increased (Table 2). The optimal definition combining all three databases required at least two contacts for PBC (PPV 73%, 95% CI 61% to 75%; sensitivity 94%, 95% CI 71% to 100%). The PPV of this definition (and the remainder) was much higher in women than in men (78% versus 40%, respectively; P=0.0009) and during the later years of the study (1994 to 1996: 61% versus 1997 to 1999: 66% versus 2000 to 2002: 90%; P=0.004). Inclusion of diagnosis codes for other conditions did not improve the predictive utility of the algorithm (data not shown). For example, an algorithm requiring at least two contacts for PBC but less than two contacts for other liver conditions had a PPV of 74% (85 of 115, 95% CI 65% to 82%) and a sensitivity of 88% (15 of 17, 95% CI 64% to 99%).

TABLE 2.

Operating characteristics of coding algorithms for definite or probable primary biliary cirrhosis (PBC)^*

Data source and diagnostic criterion	Positive predictive value, % (95% CI), (n/n)			Sensitivity for PBC trial patients, % (95% CI), (n/n)
	Overall	Women	Men
All databases
≥1 contact	60 (53–67), (119/198)	69 (61–76), (108/157)	27 (14–43), (11/41)	94 (71–100), (16/17)
≥2 contacts	73 (61–75), (98/135)	78 (70–85), (90/115)	40 (19–64), (8/20)	94 (71–100), (16/17)
≥3 contacts	73 (64–81), (79/108)	79 (70–87), (73/92)	38 (15–65), (6/16)	71 (44–90), (12/17)
≥4 contacts	72 (61–81), (61/85)	79 (68–88), (57/72)	31 (9.1–61), (4/13)	65 (33–82), (10/17)
≥5 contacts	75 (63–85), (48/64)	82 (69–91), (44/54)	40 (12–74), (4/10)	47 (23–72), (8/17)
Physician claims database
≥1 claim	64 (56–71), (109/171)	74 (65–81), (100/136)	26 (13–43), (9/35)	88 (64–99), (15/17)
≥1 GP claim	73 (52–88), (19/26)	90 (68–99), (18/20)	7 (0.4–64), (1/6)	18 (3.8–43), (3/17)
≥1 specialist claim	66 (58–73), (107/163)	74 (65–81), (98/133)	30 (15–49), (9/30)	82 (57–96), (14/17)
≥2 claims	75 (66–82), (91/122)	79 (71–87), (85/107)	40 (16–68), (6/15)	88 (64–99), (15/17)
≥3 claims	75 (65–84), (70/93)	82 (72–90), (65/79)	36 (13–65), (5/14)	71 (44–90), (12/17)
≥4 claims	73 (61–83), (51/70)	81 (69–90), (47/58)	33 (10–65), (4/12)	53 (28–77), (9/17)
≥5 claims	78 (64–88), (42/54)	85 (71–94), (39/46)	38 (8.5–76), (3/8)	41 (18–67), (7/17)
Inpatient database
≥1 inpatient contact	51 (37–64), (29/57)	55 (39–70), (24/44)	39 (14–68), (5/13)	5.9 (0.1–29), (1/17)
≥2 inpatient contacts	48 (26–70), (10/21)	60 (32–84), (9/15)	7 (0.4–64), (1/6)	5.9 (0.1–29), (1/17)
ACCS database
≥1 ACCS contact	74 (58–86), (31/42)	77 (60–90), (27/35)	57 (18–90), (4/7)	24 (6.8–50), (4/17)
≥2 ACCS contacts	78 (40–97), (7/9)	83 (36–100), (5/6)	67 (9.4–99), (2/3)	5.9 (0.1–29), (1/17)

^*The coding algorithms with the optimal combination of positive predictive value and sensitivity are shown in bold. Use of the alternative algorithms is not recommended. ACCS Ambulatory Care Classification System; GP General practitioner

Because the majority of contacts were identified using the physician claims database, the PPV of the optimal definition in this database was similar to that of all three databases combined (75%; 95% CI 66% to 82%). However, the sensitivity was slightly lower (88%; 95% CI 64% to 99%). Coding by GPs was less sensitive than specialists (18% versus 82%; P=0.0004), but the PPVs were similar (73% versus 66%; P=0.51). Although the PPVs in the ACCS database were similar (74% to 78%) to those of the optimal definition from all three databases, the sensitivities were much poorer (6% to 24%). Similarly, the inpatient database was not sensitive, with a maximum PPV of only 51%.

Validity of administrative data for definite, probable or suspected PBC

Table 3 includes the operating characteristics of the same coding definitions for identifying patients with definite, probable or suspected PBC (n=147). As described above, the definition requiring at least two contacts from any of the databases had the optimal balance between PPV (89%, 95% CI 82 to 94%) and sensitivity (94%, 95% CI 71% to 100%). For this case definition, the PPVs among women and men were 94% (95% CI 88% to 98%) and 60% (95% CI 36% to 81%), respectively (P=0.0002). The remainder of the analyses paralleled those described above, although all PPVs were higher for this less stringent case definition.

TABLE 3.

Operating characteristics of coding algorithms for definite, probable or suspected primary biliary cirrhosis^*

Data source and diagnostic criterion	Positive predictive value, % (95% CI), (n/n)
	Overall	Women	Men
All databases
≥1 contact	74 (68–80), (147/198)	83 (77–89), (131/157)	39 (24–56), (16/41)
≥2 contacts	89 (82–94), (120/135)	94 (88–98), (108/115)	60 (36–81), (12/20)
≥3 contacts	90 (83–95), (97/108)	95 (88–98), (87/92)	63 (35–85), (10/16)
≥4 contacts	91 (82–96), (77/85)	96 (88–99), (69/72)	62 (32–86), (8/13)
≥5 contacts	95 (87–99), (61/64)	100 (93–100), (54/54)	70 (35–93), (7/10)
Physician claims database
≥1 claim	77 (70–83), (132/171)	88 (81–93), (119/136)	37 (22–55), (13/35)
≥1 GP claim	77 (56–91), (20/26)	95 (75–100), (19/20)	17 (0.4–64), (1/6)
≥1 specialist claim	80 (73–86), (130/163)	88 (81–93), (117/133)	43 (26–63), (13/30)
≥2 claims	90 (83–95), (110/122)	94 (88–98), (101/107)	60 (32–84), (9/15)
≥3 claims	90 (82–96), (84/93)	96 (89–99), (76/79)	57 (29–82), (8/14)
≥4 claims	90 (81–96), (63/70)	97 (71–91), (56/68)	58 (28–85), (7/12)
≥5 claims	94 (85–99), (51/54)	100 (92–100), (46/46)	63 (24.5–92), (5/8)
Inpatient database
≥1 inpatient contact	81 (68–90), (46/57)	84 (70–93), (37/44)	69 (39–91), (9/13)
≥2 inpatient contacts	86 (64–97), (18/21)	93 (68–100), (14/15)	67 (22–96), (4/6)
ACCS database
≥1 ACCS contact	88 (74–96), (37/42)	91 (77–98), (32/35)	71 (29–96), (5/7)
≥2 ACCS contacts	89 (52–100), (8/9)	100 (54–100), (6/6)	67 (9.4–99), (2/3)

^*The coding algorithms with the optimal combination of positive predictive value and sensitivity are shown in bold. Use of the alternative algorithms is not recommended. ACCS Ambulatory Care Classification System; GP General practitioner

Sensitivity analysis of the diagnostic definitions for PBC according to the time interval between contacts

As illustrated in Table 4, the PPVs of the diagnostic definitions requiring at least two contacts for PBC did not change significantly (72% to 74%) according to the interval between the first and second contact. However, restricting the analyses to patients with the first and second contact within the same year led to a substantial reduction in sensitivity (from 94% to 71% with all three databases combined, and from 88% to 71% with the physician claims database). These data suggest that more than one year of administrative data are necessary to maximize the identification of PBC patients.

TABLE 4.

Sensitivity analysis of diagnostic definitions for primary biliary cirrhosis (PBC) according to the time interval between the first and second contacts in the administrative data

Data source and time interval between contacts	n	Positive predictive value, % (95% CI), (n/n)		Sensitivity for PBC trial patients, % (95% CI), (n/n)
		Definite or probable PBC	Definite, probable or suspected PBC
All databases
Within 1 year	112	72 (63–80), (81/112)	91 (84–96), (102/112)	71 (44–90), (12/17)
Within 2 years	129	73 (64–80), (94/129)	90 (83–95), (116/129)	94 (71–100), (16/17)
Within 3 years	133	72 (64–80), (96/133)	89 (82–94), (118/133)	94 (71–100), (16/17)
Physician claims database
Within 1 year	103	74 (64–82), (76/103)	90 (83–95), (93/103)	71 (44–90), (12/17)
Within 2 years	117	74 (66–82), (87/117)	91 (84–95), (106/117)	88 (64–99), (15/17)
Within 3 years	121	74 (66–81), (90/121)	90 (83–95), (109/121)	88 (64–99), (15/17)

DISCUSSION

Our study demonstrates the utility of administrative data for the identification of patients with PBC. Using three administrative databases containing nine years of data, the optimal case definition required at least two contacts for PBC. This definition had a PPV of 73% for definite or probable PBC, and 89% for definite, probable or suspected PBC; its sensitivity was 94%. In our opinion, this degree of accuracy is sufficient to justify the use of administrative data in future studies. To our knowledge, only one other study has examined the utility of administrative data for this purpose. Villeneuve et al (6) reviewed the charts of 648 patients with an ICD-9 code for PBC in a hospitalization database. Only 257 of these patients had definite or probable PBC; the 40% PPV is similar to the 51% that we observed using the inpatient database. However, the poor sensitivity of this approach (6% in our study) reinforces the importance of using multiple data sources including outpatient databases (see below).

We identified various diseases misclassified as PBC when a single contact in the administrative data suggested this diagnosis. Because false-positive cases had a fewer number of PBC contacts, increasing the number required to establish a diagnosis reduced misclassification, but was less sensitive. However, attempted exclusion of these competing conditions using their own diagnosis codes did not improve algorithm performance. In terms of specific conditions, misclassification of secondary biliary cirrhosis was inevitable because it shares the same ICD-9-CM code as PBC. Because this disease is uncommon, we expect this issue to have minimal impact on future studies that use this methodology. In contrast, patients with PSC represented a sizable proportion of false-positive cases (28% versus 20% in the study by Villeneuve et al [6]). This finding likely reflects a transcription error in some cases because the ICD-9-CM codes are similar (571.6 for PBC versus 576.1 for PSC). In addition, both disorders are characterized by chronic cholestasis, symptoms including fatigue and pruritus, and autoantibodies, and may have overlapping histological features (51). Finally, patients with coexisting PBC and PSC (ie, ‘PBC/PSC overlap syndrome’), including one from the CHR (52), have been described. However, as confirmed by our results, the usual patient demographics differ – whereas PBC predominantly affects middle-aged women, PSC is more common in young men. Twenty per cent of false-positive cases were due to AIH, likely because both conditions are more common in women and often associated with autoantibodies (53).

We conducted several sensitivity analyses aimed at refining the use of administrative data for identifying PBC patients. Our results demonstrate the benefits of using multiple data sources. As expected, the majority of our patients (85%) were identified using the physician claims database because PBC is predominantly a disease of outpatients. Although the PPVs of the claims database were similar to that of all three databases combined, its sensitivity was lower (88% versus 94%). Nevertheless, based on this diagnostic performance, it would be reasonable to use this data source when the others are unavailable. Although reasonable for studies of incidence and prevalence, this approach would be inappropriate for outcome studies (eg, analyses of rates of hepatic failure) because these events require hospitalization data for identification. On the other hand, the low PPVs and sensitivities of the inpatient and ACCS databases preclude their use in isolation. This finding is not unexpected because the inpatient database is most useful for detecting patients with PBC complications (eg, decompensation), or those hospitalized for nonhepatic conditions in which PBC may not be recorded. Similarly, the major role of the ACCS database is to identify emergency department visits, expected to be uncommon in PBC, or day procedures such as liver biopsy and endoscopy, which play only a secondary role in the management of these patients.

Because PBC is more common in women, it is not surprising that the PPVs of the coding algorithms – which are prevalence-dependent – were higher in women. For definite or probable PBC, the definition requiring at least two PBC contacts had a PPV of 78% in women versus only 40% in men. In contrast, many conditions confused with PBC (eg, PSC, hepatitis C and alcoholic cirrhosis) are more common in men. Thus, the probability of erroneously recording a diagnosis code for PBC should be higher in men – an effect that would contribute to lower PPVs in this subgroup. An alternative explanation is that clinicians have greater difficulty diagnosing PBC in men, although evidence to support this suggestion is lacking.

We also confirmed our hypothesis that specialists more accurately code for PBC than GPs. Although the PPVs of at least one claim by a specialist or GP were similar (approximately 80%), the sensitivity of this criterion among specialists was much higher (82% versus 18%). This finding likely reflects a greater awareness of PBC among specialists, the methods of its diagnosis and its diagnosis codes. In an inflammatory bowel disease (IBD)-related study (54) that addressed the latter issue among Canadian physicians, gastroenterologists were more likely to know the codes for IBD than GPs, and used them more frequently for both IBD- and non-IBD-related services. We also assessed the impact of the duration over which the diagnosis codes were recorded on the performance of the administrative data (Table 4). In this analysis, the PPV of the definition requiring at least two contacts was similar when the first and second contact occurred within one, two or three years of each other. However, the sensitivity was significantly lower when restricted to patients with contacts occurring within the same year (71% versus 94% for less than two and less than three years). This finding was likely due to the infrequent follow-up of most PBC patients, who are often seen annually (or less frequently) if stable (1). It suggests that future analyses using administrative data should include multiple years of data to avoid missing nearly 30% of cases that would otherwise have an insufficient observation period to accrue two or more contacts.

Our findings support the use of administrative data in future epidemiological studies of PBC. Because we demonstrated a short interval between diagnosis dates established using clinical data and the first contact in the administrative data (median of less than two months), accurately timing the date of diagnosis using administrative data is feasible. This point is essential for defining incident cases and establishing ‘time zero’ for natural history studies. Interestingly, the PPVs of the coding algorithms were higher in recent years, suggesting improved accuracy over time. This finding likely relates to greater difficulty in confirming a diagnosis of PBC during the earlier years of the validation study (eg, due to missing laboratory reports and clinical data [see below]), or perhaps increased awareness of the diagnosis codes for PBC more recently. This finding must be considered when interpreting temporal changes in PBC burden. The major advantage of the administrative databases that we used in the current study is their population-based nature, which limits the selection bias inherent in many single-centre studies. If our findings are validated in different settings, inter-regional comparisons of PBC epidemiology will be facilitated.

Our study has several limitations. First, we were unable to locate the medical records of approximately 40% of patients. In many cases, charts could not be retrieved because the study period dated back to 1994. In addition, we could not access the records of GPs or specialists practicing outside of the University of Calgary Medical Clinic. Because coding accuracy was associated with physician specialty, this limitation may have overestimated algorithm performance. On a related note, a significant proportion of patients (n=28 [14%]) were labelled as ‘suspected PBC’ and excluded from our primary outcome due to a lack of diagnostic information. It is likely that many, if not all, of these patients actually had PBC. For example, three patients were AMA-positive with cholestasis, but could not be given a diagnosis of ‘probable PBC’ because their AMA titre was unavailable. Many additional patients – including one of the 17 clinical trial patients – were diagnosed in other health regions by experienced physicians who prescribed ursodeoxycholic acid. Thus, we would argue that the correct PPV of the optimal algorithm is closer to the 89% observed in our analysis of definite, probable or suspected PBC.

CONCLUSION

The present study demonstrated the feasibility of identifying patients with PBC using administrative data. In future studies, we plan to apply these coding algorithms to additional data sources to more accurately define the current epidemiology and natural history of PBC in Canada. If validated in other settings, these algorithms will also enable comparisons of PBC burden and outcomes across regions. These studies will prove useful for resource planning, patient counselling regarding prognosis and treatment decisions. Moreover, administrative data will facilitate the identification of PBC patient cohorts, which can be studied in greater detail to fill existing gaps in the literature (eg, the impact of early diagnosis on outcome, disease associations, modes of presentation, etc). Finally, comprehensive evaluation of such cohorts (eg, via surveys examining potential risk factors or biofluid collection for high-throughput studies) may further our understanding of disease pathogenesis including the influence of environmental and genetic factors.

APPENDIX. Glossary of statistical terminology used in the assessment of algorithm performance