The reliability of diagnostic coding and laboratory data to identify tuberculosis and nontuberculous mycobacterial disease among rheumatoid arthritis patients using antitumor necrosis factor therapy

Abstract

Purpose

Anti-tumor necrosis factor alpha (anti-TNF) therapies are associated with severe mycobacterial infections in rheumatoid arthritis patients. We developed and validated electronic record search algorithms for these serious infections.

Methods

The study used electronic clinical, microbiologic, and pharmacy records from Kaiser Permanente Northern California (KPNC) and the Portland Veterans Affairs Medical Center (PVAMC). We identified suspect tuberculosis and nontuberculous mycobacteria (NTM) cases using inpatient and outpatient diagnostic codes, culture results, and anti-tuberculous medication dispensings. We manually reviewed records to validate our case-finding algorithms.

Results

We identified 64 tuberculosis and 367 NTM potential cases respectively. For tuberculosis, diagnostic code positive predictive value (PPV) was 54% at KPNC and 9% at PVAMC. Adding medication dispensings improved these to 87% and 46% respectively. Positive tuberculosis cultures had a PPV of 100% with sensitivities of 79% (KPNC) and 55% (PVAMC). For NTM, the PPV of diagnostic codes was 91% (KPNC) and 76% (PVAMC). At KPNC, ≥1 positive NTM culture was sensitive (100%) and specific (PPV, 74%) if non-pathogenic species were excluded; at PVAMC, ≥1 positive NTM culture identified 76% of cases with PPV of 41%. Application of the American Thoracic Society NTM microbiology criteria yielded the highest PPV (100% KPNC, 78% PVAMC).

Conclusions

The sensitivity and predictive value of electronic microbiologic data for tuberculosis and NTM infections is generally high, but varies with different facilities or models of care. Unlike NTM, tuberculosis diagnostic codes have poor PPV, and in the absence of laboratory data, should be combined with anti-tuberculous therapy dispensings for pharmacoepidemiologic research.

Background

Over two million persons in the United States suffer from rheumatoid arthritis (RA),¹ a systemic disease characterized by inflammation and destruction of various joints, lung, skin, and other organ systems. In the last decade, the biologic immunosuppressive therapies infliximab (Remicade®, Centocor, PA, USA), etanercept (Enbrel®, Immunex, WA, USA), and adalimumab (Humira®, Abbott Laboratories, IL, USA) have become widely used to treat patients with RA and other chronic inflammatory diseases.^2,3 These therapies inhibit tumor necrosis factor-alpha (TNF), a cytokine important in the pathogenesis of these diseases, but also integral to host immune defense against a variety of opportunistic pathogens.^4–6

To date, mycobacterial complications including tuberculosis and nontuberculous mycobacterial (NTM) disease represent the most commonly reported opportunistic infections in patients using these therapies.^7–9 The public health ramifications of these complications are potentially large, and to date in the US, there is little population-level data available from which to estimate the risk of these infectious adverse events. We undertook the present study to develop and validate algorithms to identify tuberculosis and NTM in an effort to facilitate future drug safety studies assessing adverse events associated with these and other biologic, immunosuppressant therapies.

Methods

Study Population

The study included RA patients treated with anti-TNF therapy from Kaiser Permanente Northern California (KPNC) and the Portland Veteran’s Affairs Medical Center (PVAMC). KPNC is a large health maintenance organization serving 3.2 million members in Northern California, while PVAMC is the data center for Veteran’s Integrated Service Network 20 (VISN 20), the Northwest network of the US Veteran’s Affairs Administration medical centers serving 400,000 members within Oregon, Washington, Alaska, and Idaho. The study was approved by the Human Subjects Review Board at participating institutions.

Anti-TNF treated cohort identification

By searching electronic medical records, we identified all patients within these medical systems with at least one clinical visit and ≥ 1 outpatient prescription for etanercept or adalimumab, or at least one infusion of infliximab during a nine-year study period from January 1, 2000 through Dec. 31, 2008. For all identified anti-TNF users, we searched their electronic medical records for outpatient or inpatient international classification of disease (ICD-9) codes for the following inflammatory diseases for which anti-TNF therapy is generally used: rheumatoid arthritis (714), psoriasis (696.0, 696.1), Crohn’s disease (555), ulcerative colitis (556), and ankylosing spondylitis (720.0). Patients with diagnostic codes for human immunodeficiency virus (HIV) were excluded.

Case-finding and validation

The same algorithms were applied to databases at KPNC and PVAMC. However, unlike KPNC, there were very few (n=3) suspect tuberculosis and NTM cases identified in the anti-TNF cohort at PVAMC. Therefore, we validated our case-finding algorithms using the general population (i.e. all patients) at PVAMC.

We searched electronic clinical, microbiologic, and pharmacy records to identify suspect tuberculosis and NTM cases. For tuberculosis, we defined a suspect case as (1) any patient with one or more acid fast bacilli culture isolate or polymerase chain reaction (PCR) result identified as “M. tuberculosis” or “M. tuberculosis complex,” (2) a patient with at least one inpatient or outpatient diagnostic code for active tuberculosis (codes 010-018), or (3) a patient with at least one inpatient or outpatient prescription for “rifampin” and “isoniazid” given on the same day, or for “pyrazinamide” given anytime during the study period.

NTM suspect cases were identified using the following algorithm: (1) one or more positive acid fast bacilli cultures for nontuberculous mycobacteria, with elimination of patients with only isolates of M. gordonae, because this species is generally a contaminant and does not cause disease, or (2) one inpatient or outpatient diagnostic code for NTM disease (code 031).

Using the lists of tuberculosis and NTM suspects generated with these search criteria, we reviewed the full electronic medical record of all suspects and used US Centers for Disease Control and Prevention (CDC) tuberculosis disease criteria to define confirmed tuberculosis cases.¹⁰ In addition, at PVAMC we cross-matched our tuberculosis suspect case list with the State of Oregon tuberculosis registry to verify that all cases met the CDC tuberculosis case definition. We used the American Thoracic Society/ Infectious Diseases Society of America (ATS/IDSA) criteria to confirm cases of pulmonary NTM. This case definition requires patients to fulfill microbiologic (one positive culture from a bronchocscopic specimen, or at least two positive isolates from sputum), clinical (respiratory symptoms), and radiologic criteria (characteristic radiographic findings).¹¹ Patients with extrapulmonary disease were confirmed if they had an NTM isolated from a normally sterile site (blood, cerebrospinal fluid, tissue, pleural space, peritoneum).

Following case validation of the full electronic medical record, we calculated the sensitivity and positive predictive value (PPV) of our search algorithms alone and in combination separately for tuberculosis and NTM using the list of confirmed cases as our gold standard with the assumption that this list comprised of all suspect cases occurring during the study period. For sensitivity and PPV calculations, we developed 95% exact binomial confidence interval using SAS.¹¹

Results

KPNC

We identified 4,524 anti-TNF users during the study period; 4,450 (98%) carried RA diagnostic codes. Among this group we identified 57 suspect cases; of these, we found 14 and 18 confirmed tuberculosis and NTM cases respectively. For tuberculosis, searching mycobacterial culture results identified 10 (79%) cases. Diagnostic codes for active tuberculosis identified all 14 cases, however, they were of low PPV as 12 (46%) patients with codes did not have active tuberculosis (Table 1a). Of these 12, six (50%) had latent tuberculosis infection, one had community acquired pneumonia, one had repeated M. gordonae isolation, and in the other four cases there was no identifiable reason for their diagnostic code. Tuberculosis diagnostic code PPV substantially improved by adding anti-tuberculous pharmacy dispensing to the search algorithm with little diminishment in sensitivity (Table 1a).

Table 1.

a: Sensitivity and positive predictive value of various case-finding algorithms for active tuberculosis within Kaiser Permanente Northern California anti-TNF cohort (n=14 tuberculosis cases identified).+
Case-finding Algorithm (number of patients identified)	Sensitivity* [# (%) of total cases identified]	Sensitivity 95% CI	PPV+ (95% CI)
011-018 Diagnostic codes¥
≥ 1 (n=26)	14 (100%)	0.77-1.00	54% (0.33-0.73)
≥ 2 (n=14)	10 (71%)	0.42-0.92	71% (0.42-0.92)
Isoniazid/Rifampin± (n=17)	11 (79%)	0.49-0.95	64% (0.38-0.86)
Pyrazinamide€ (n=12)	12 (85%)	0.57-0.98	100% (0.74-1.00)
Isoniazid/Rifampin± AND any 011-018 Code (n=13)	11 (79%)	0.49-0.95	85% (0.65-0.98)
Pyrazinamide€ AND any 011-018 Code (n=12)	12 (85%)	0.57-0.98	100% (0.74-1.00)
Pyrazinamide€ or Isoniazid/Rifampin± AND any 011-018 Diagnostic codes¥ (n=15)	13 (93%)	0.66-1.00	87% (0.60-0.98)
M. tuberculosis isolated in Culture (n=10)	10 (79%)	0.42-0.92	100% (0.69-1.00)

b: Sensitivity and positive predictive value of various case-finding algorithms for active tuberculosis within Portland Veteran Affairs Medical Center’s general population (n=22 tuberculosis cases identified)+
Case-finding Algorithm (number of patients identified)	Sensitivity* [# (%) of total cases identified]	Sensitivity 95% CI	PPV+ (95% CI)
011-018 Diagnostic codes¥
≥ 1 (n=197)	17 (77%)	0.55-0.92	9% (0.05-0.14)
≥ 2 (n=68)	14 (64%)	0.41-0.83	21% (0.12-0.32)
Isoniazid/Rifampin± (n=43)	12 (55%)	0.32-0.76	28% (0.15-0.44)
Pyrazinamide€ (n=38)	11 (50%)	0.28-0.72	29% (0.15-0.46)
Isoniazid/Rifampin± AND any 011-018 Code (n=26)	12 (55%)	0.32-0.76	46% (0.27-0.67)
Pyrazinamide€ AND any 011-018 Code (n=24)	11 (50%)	0.28-0.72	46% (0.26-0.67)
Pyrazinamide€ or Isoniazid/Rifampin ± AND any 011-018 Diagnostic codes¥ (n=27)	12 (55%)	0.32-0.76	44% (0.26-0.65)
Pyrazinamide€ or Isoniazid/Rifampin ± AND ≥ 2 011-018 Diagnostic codes¥ (n=17)	12 (55%)	0.32-0.76	71% (0.44-0.90)
M. tuberculosis isolated in Culture OR (Pyrazinamide€ or Isoniazid/Rifampin ± AND ≥ 2 011-018 Diagnostic codes ¥) (n=22)	17 (77%)	0.55-0.92	77% (0.55-0.92)
M. tuberculosis isolated in culture (n=12)	12 (55%)	0.32-0.76	100% (0.74-1.00)

Preferred algorithms that maximize sensitivity and PPV are in bold

⁺CDC tuberculosis case definition used to confirm cases by chart review

^*Calculation of sensitivity using assumption that all potential cases were identified by searching Diagnostic codes, tuberculosis drug prescriptions, and mycobacterial culture results (n=14).

^¥Active tuberculosis Diagnostic codes received ≥ 1 or ≥ 2 times during the study period

^±Isoniazid and rifampin prescribed on same day during the study period

^€Pyrazinamide prescribed at least once during the study period

Preferred algorithms that maximize sensitivity and PPV are in bold

⁺Oregon Public Health Division (OPHD) Tuberculosis Control Program case registry used to confirm cases

^*Calculation of sensitivity using assumption that all potential cases were identified by searching Diagnostic codes, tuberculosis drug prescriptions, and mycobacterial culture results (n=22).

^¥Active tuberculosis Diagnostic codes received ≥ 1 or ≥ 2 times during the study period

^±Isoniazid and rifampin prescribed on same day during the study period

^€Pyrazinamide prescribed at least once during the study period

For NTM, diagnostic codes detected 50% of cases, but with high PPV for true disease. After excluding M. gordonae from culture results, ≥1 NTM culture was 74% predictive for NTM disease (Tables 2a) Using the microbiologic criteria of the ATS/IDSA pulmonary NTM disease definition to define pulmonary disease (i.e. one NTM isolate from bronchoscopy specimen, or two sputum samples with NTM of the same species) and a sterile site isolate to define extrapulmonary disease, resulted in identifying all (100%) 18 cases with PPV of 100%.

Tables 2.

a. Sensitivity and positive predictive value of various case-finding algorithms for nontuberculous mycobacteria (NTM)within Kaiser Permanente Northern California anti-TNF cohort(n=18 cases identified) +
Case-finding Algorithm (number of patients identified)	Sensitivity* [# (%) of total cases identified]	Sensitivity 95% CI	PPV+ (95% CI)
031 Diagnostic codes¥
≥ 1 (n=11)	9 (50%)	0.26-0.74	82% (0.48-0.98)
Nontuberculous Mycobacteria isolated in Culture± (n=23)	18 (100%)	0.81-1.00	78% (0.56-0.93)
ATS/IDSA microbiologic disease criteria € (n=18)	18 (100%)	0.81-1.00	100% (0.81-1.00)
Nontuberculous Mycobacteria isolated in Culture± AND any 031 Diagnostic codes¥ (n=10)	9 (50%)	0.26-0.74	90% (0.56-1.00)

b: Sensitivity and positive predictive value of case-finding algorithms for nontuberculous mycobacteria (NTM) within Portland Veteran’s Affairs Medical Center’s general population, (n=71 cases identified) +
Case-finding Algorithm (number of patients identified)	Sensitivity* [# (%) of total cases identified]	Sensitivity 95% CI	PPV+ (95% CI)
031 Diagnostic codes¥
≥ 1 (n=62)	46 (65%)	(0.53-0.76)	74% (0.62-0.85)
Nontuberculous Mycobacteria isolated in Culture± (n=132)	54 (76%)	(0.65-0.85)	41% (0.32-0.50)
ATS/IDSA microbiologic disease criteria€ (n=58)	45 (63%)	(0.51-0.75)	77.6% (0.65-0.88)
Nontuberculous Mycobacteria isolated in Culture± AND any 031 Diagnostic codes¥ (n=39)	30 (42%)	(0.31-0.55)	77% (0.61-0.89)
Nontuberculous Mycobacteria isolated in Culture± OR any 031 Diagnostic codes¥ (n=171)	70 (99%)	(0.92-1.00)	41% (0.34-0.49)
031 Diagnostic codes¥ AND ATS/IDSA microbiologic disease criteria€ (n=24)	24 (34%)	(0.23-0.46)	100% (0.86-1.00)
031 Diagnostic codes¥ OR ATS/IDSA microbiologic disease criteria € (n=96)*	67 (94%)	(0.86-0.98)	70% (0.60-0.79)
031 Diagnostic codes¥ AND Azithromycin or clarithromycin ≥30 Days∞ (n=24)	24 (34%)	(0.23-0.46)	100% (0.86-1.00)
ATS/IDSA microbiologic disease criteria€ AND Azithromycin/Clarithromycin ≥30 Days ∞ (n=23)	21 (30%)	(0.19-0.42)	91% (0.72-1.00)

Preferred algorithms that maximize sensitivity and PPV are in bold

⁺American Thoracic Society/Infectious Diseases Society of America (ATS/IDSA) criteria for nontuberculous mycobacteria pulmonary disease used to define pulmonary disease; extrapulmonary disease defined by nontuberculous mycobacteria isolation from a sterile site. Cases confirmed by chart review

^*Calculation of sensitivity using assumption that all potential cases were identified by searching Diagnostic codes and mycobacterial culture results (n=20)

^¥NTM Diagnostic codes received ≥ 1 time during the study period

^±M. gordonae (non-pathogenic species) excluded from consideration

^€ATS/IDSA disease criteria for NTM pulmonary disease include microbiologic, radiologic, and symptomatic criteria. Microbiologic criteria include NTM isolated from one bronchoscopy specimen or at least 2 sputum specimens; extrapulmonary disease defined by nontuberculous mycobacteria isolation from a sterile site.

Preferred algorithms that maximize sensitivity and PPV are in bold

⁺American Thoracic Society/Infectious Diseases Society of America (ATS/IDSA) criteria for nontuberculous mycobacteria pulmonary disease used to define pulmonary disease; extrapulmonary disease defined by nontuberculous mycobacteria isolation from a sterile site. Cases confirmed by chart review

^*Calculation of sensitivity using assumption that all potential cases were identified by searching Diagnostic codes and mycobacterial culture results (n=71)

^¥Nontuberculous mycobacteria Diagnostic codes received ≥ 1 time during the study period

^±M. gordonae (non-pathogenic species) excluded from consideration

^€ATS/IDSA disease criteria for NTM pulmonary disease include microbiologic, radiologic, and symptomatic criteria. Microbiologic criteria include NTM isolated from one bronchoscopy specimen or at least 2 sputum specimens.

^∞Long term macrolide use is cornerstone of NTM disease therapy

PVAMC

We identified 1,133 patients using anti-TNF therapy at PVAMC during the study period with only three suspect cases identified. Accordingly, we chose to validate our algorithms using the general VA population receiving care at PVAMC. From this population, we identified 147 potential NTM cases (identified by positive culture or diagnostic code) and 220 suspect tuberculosis cases. Similar to KPNC, diagnostic codes had high sensitivity but low PPV for active tuberculosis (Table 1b). Diagnostic code PPV was improved by adding evidence of antimycobacterial drug dispensings. For NTM (Table 2b), the microbiologic component of the ATS/IDSA NTM case definition had high PPV (77%). An algorithm combining these criteria with an NTM diagnostic code yielded a search strategy that maximized sensitivity and PPV. Algorithms combining long-term (>30 days) macrolide antibiotic use in combination with NTM diagnostic codes yielded a PPV of 100%, although the sensitivity of this approach was half that of codes alone (Table 2b).

Discussion

We constructed case-finding algorithms to find cases of tuberculosis and NTM disease at KPNC, a large health-maintenance organization, and at PVAMC, a large regional VA within the Pacific Northwest. Our findings suggest that in healthcare systems where microbiologic information is electronically recorded, search strategies identifying positive mycobacterial culture results provide highly sensitive and accurate ways to identify these infections. Our results also suggest that studies with access only to diagnosis and procedure codes will poorly ascertain these important opportunistic infections. NTM diagnostic codes have reasonably high PPV, but lack sensitivity relative to culture-based case-finding algorithms, while tuberculosis diagnostic codes alone are not reliable and are useful only in combination with pharmacy dispensing data for antituberculous medications.

Our results suggest that previous studies relying solely on diagnostic codes have likely overestimated tuberculosis prevalence.¹³ It appears that many patients receive diagnostic codes for active tuberculosis when they are actually diagnosed with latent tuberculosis infection by a tuberculin skin test. In other cases, patients hospitalized with pneumonia as tuberculosis suspects might be given the code before later ruling out for tuberculosis. In our study, one-time receipt of drugs relatively specific for the treatment of active tuberculosis disease markedly improved the PPV of these diagnostic codes. This improvement was more marked at KPNC, although improvement was noted at PVAMC particularly if patients had multiple diagnostic codes for active tuberculosis. In the United States, most patients within private healthcare systems diagnosed with tuberculosis are referred to and treated by public health departments. Accordingly, relying on case-finding algorithms that incorporate pharmacology data for anti-tuberculous medicines would miss many patients with active tuberculosis.¹⁴ The practice of initiating anti-tuberculous therapy could vary between health care systems and regions, making this case-finding algorithm less or more reliable than found within our study. Interestingly, and for unclear reasons, the PPV of anti-tuberculous therapy alone and in combination with diagnostic codes was lower in the VA than it was in the KPNC system. This might reflect a lower threshold for initiating empiric anti-tuberculous medications in hospitalized veterans who might be sicker or have greater numbers of co-morbidities.

In general, NTM diagnostic codes appear more useful than tuberculosis codes. Although recent studies suggest NTM is more prevalent than tuberculosis in the United States, it is likely that clinicians are less familiar with this disease entity. It is underappreciated and more difficult to diagnose.^10,11,15 Further, unlike tuberculosis, there is no latent form of infection that could be confused with active disease from a coding standpoint. For these reasons, NTM diagnostic codes are more likely to represent true disease. The strategy of adding evidence of antibiotic therapy to NTM diagnostic codes increases PPV. Macrolide antibiotics (clarithromycin, azithromycin) are the cornerstone of multi-drug anti-NTM therapy, and generally employed for 18 months or longer,¹¹ and such long-term macrolide therapy is unlikely in other types of infections. However, as many as 2/3 of NTM patients at any one time might not be receiving antibiotic therapy for NTM for a variety of clinical reasons^16,17 (unpublished observation) such that adding long-term (e.g.>30 days) macrolide therapy to the search algorithm would markedly decrease an algorithm’s sensitivity. In the VA system where we evaluated this strategy, the combination of ICD-9 code plus macrolide therapy yielded a PPV of 100% but identified 50% fewer cases than codes alone.

In both systems, we most efficiently and accurately identified tuberculosis and NTM cases with electronic microbiologic record search. While the PPV of a positive tuberculosis culture is 100%, that for a positive NTM culture is lower. This finding is a result of the ATS/IDSA pulmonary NTM case definition that mandates patients fulfill microbiologic criteria, clinical symptoms, and radiographic findings in order to be classified as having pulmonary NTM disease. We recently conducted a state-wide public health surveillance project in Oregon that found between 1/3 to 1/2 of persons with one positive respiratory NTM culture have true disease, and that 86% of those meeting the microbiologic criteria also meet the full ATS/IDSA case criteria.¹⁶ We found the PPV of the microbiologic criteria to be similar across our study sites and have validated its use as a highly accurate search algorithm for NTM disease. In regards to the sensitivity of microbiologic record search algorithms, it is interesting that the ATS/IDSA microbiologic criteria were not 100% sensitive at PVAMC due to the fact that many VA patients receive parts of their care outside of the VA. We found that a substantial number of patients who met criteria for NTM had parts or all of their microbiologic work-up outside of the VA system and therefore were not identified by searching microbiologic data in the VA system. Finally, it should be noted that the sensitivity of culture-based record searching for tuberculosis will generally not exceed 80%, as approximately 20% of tuberculosis cases in the United States are culture-negative, and this proportion is likely higher in immunosuppressed settings in which extrapulmonary forms of tuberculosis are more common and positive diagnostic cultures are frequently more difficult to obtain.¹⁸

It is unclear if the findings from this study are generalizable to healthcare systems other than Kaiser or PVAMC. In the VA system, a study within the New England Veteran’s Integrated Service Network one (VISN 1) found higher PPV for tuberculosis diagnostic codes (73%), but they evaluated only inpatient codes. ¹⁹ Our study included both outpatient and inpatient codes, and it is likely that outpatient codes are less specific for active disease because the diagnosis of latent tuberculosis infection would me more likely to be made on an outpatient basis. Unfortunately, we did not distinguish between in and outpatient codes within our study. We also recognize that the use and accuracy of diagnostic codes vary between physicians and healthcare systems. Although there are differences in point estimates for code sensitivity and PPV between the Kaiser and PVAMC within our study, our findings are fairly congruent between these two very different healthcare systems. For this reason, we suspect evaluation of these search algorithms elsewhere would likely yield similar results. Further, our study highlighted the usefulness of electronic microbiologic records for accurate NTM and tuberculosis case-finding. Although the data quality of electronic microbiologic records might vary between healthcare systems, the PPV of positive tuberculous cultures or NTM microbiologic criteria should remain high no matter what healthcare system is evaluated as misclassification of such results is unlikely.

In summary, we have identified and validated diagnostic code and culture-based case-finding algorithms for tuberculosis and NTM disease within two population-based healthcare systems. These algorithms could be employed in future pharmacovigilance studies assessing the safety of biologic or other drug therapies. In addition, they could be used for surveillance purposes to determine disease prevalence or to monitor trends in disease in certain high risk populations. Our findings suggest that studies using administrative data should not utilize diagnostic codes alone to identify tuberculosis, as these codes have poor predictive value and are not reliable, and appear to have some utility only when used in combination with pharmacy data showing evidence of anti-tuberculous drug therapy. Conversely, NTM diagnostic codes have fairly high PPVs and are likely to predict true disease, although they lack sensitivity relative to culture-based case-finding algorithms such that studies using code-only algorithms will underestimate the prevalence of true NTM disease. In contrast to code-based algorithms, microbiologic-based search algorithms for NTM and tuberculosis are highly sensitive and accurate, and would be most useful for future pharmacovigilance studies.

Key points.

(1) In healthcare systems where microbiologic information is electronically recorded, search strategies identifying positive mycobacterial culture results provide highly sensitive and accurate ways to identify cases of tuberculosis and nontuberculous mycobacterial (NTM) disease. (2) Diagnostic codes for tuberculosis are not reliable and are useful only in combination with pharmacy dispensing data for anti-tuberculous medications. (3) NTM diagnostic codes have fairly high positive predictive value but lack sensitivity in identifying NTM cases.