Validity of Clinical Prediction Rules for Isolating Inpatients with Suspected Tuberculosis

Abstract

Objective

Declining rates of tuberculosis (TB) in the United States has resulted in a low prevalence of the disease among patients placed on respiratory isolation. The purpose of this study is to systematically review decision rules to predict the patient's risk for active pulmonary TB at the time of admission to the hospital.

Data Sources

We searched MEDLINE (1975 to 2003) supplemented by reference tracking. We included studies that reported the sensitivity and specificity of clinical variables for predicting pulmonary TB, used Mycobacterium TB culture as the reference standard, and included at least 50 patients.

Review Method

Two reviewers independently assessed study quality and abstracted data regarding the sensitivity and specificity of the prediction rules.

Results

Nine studies met inclusion criteria. These studies included 2,194 participants. Most studies found that the presence of TB risk factors, chronic symptoms, positive tuberculin skin test (TST), fever, and upper lobe abnormalities on chest radiograph were associated with TB. Positive TST and a chest radiograph consistent with TB were the predictors showing the strongest association with TB (odds ratio: 5.7 to 13.2 and 2.9 to 31.7, respectively). The sensitivity of the prediction rules for identifying patients with active pulmonary TB varied from 81% to 100%; specificity ranged from 19% to 84%.

Conclusions

Our analysis suggests that clinicians can use prediction rules to identify patients with very low risk of infection among those suspected for TB on admission to the hospital, and thus reduce isolation of patients without TB.

Isolation of inpatients with active pulmonary tuberculosis (TB) is essential to avoid nosocomial transmission of TB. The current guidelines for controlling Mycobacterium TB transmission within institutions recommend isolating all potentially contagious patients in single bed, negative-pressure rooms. Patients should remain in isolation until 3 consecutive sputum smears are negative for acid-fast bacilli (AFB) or if the patient is found to have TB, effective treatment is initiated, there is a clinical response, and 3 AFB smears are negative.¹

Implementation of these guidelines has resulted in a decreased transmission of TB in some institutions.² Declining rates of TB in the United States³ and lower rates of hospitalization of persons with TB however, have lead to a low prevalence of TB among patients isolated because of suspicion of the disease. This is a major problem in areas with low prevalence of TB where as many as 92 patients without TB are placed on respiratory isolation for every patient with the disease.⁴ Despite these high rates of isolation of patients at low risk of TB, delayed identification of patients with TB is a well-documented problem.^5–9 Typically, patients with TB are identified using AFB smears, however the sensitivity of this test is not high (50% to 60%),¹⁰ and transmission of TB from patients with negative smears has been reported.^11–15

Decreasing resource utilization by identifying patients who have a very low risk of TB could help reduce hospital costs. Under the current isolation policy, assessing risk of TB is based on the presence of risk factors, physical findings, and chest radiograph. Physicians usually use their own experience and intuition to interpret these findings and decide whether a patient needs isolation. Prior experience and intuition however, may be misleading. Clinical prediction models are useful tools that quantify the individual contributions that various components of the history, physical examination, and basic laboratory results make toward the diagnosis. Several studies have assessed the value of clinical predictors and clinical decision models as a guide for identifying patients requiring respiratory isolation. Tuberculosis respiratory guidelines that incorporate these prediction models may achieve the important goal of reducing costs without compromising patient care.

We conducted a systematic review to summarize the available evidence about the sensitivity and specificity of clinical parameters to identify patients at low risk of active pulmonary TB.

METHODS

Data Sources and Study Eligibility

We conducted a literature search of MEDLINE (1975 to 2003) using combinations of the key words diagnosis, prediction rule, clinical predictors, sensitivity, specificity, isolation, AFB, and TB. We augmented our literature search by manually reviewing the reference lists of identified studies. We limited our search to studies published in English.

Two investigators (D.S. and J.P.W.) independently evaluated potential articles to decide if they were eligible for inclusion in the systematic review. For the purpose of the analysis we defined clinical prediction rules as the combination of 2 or more clinical predictors (elements of the patient's history, physical examination, and chest radiograph) designed to quantify the probability of active pulmonary TB. Thus, studies were eligible for inclusion if they (1) addressed the usefulness of at least 2 clinical predictors to quantify the probability of pulmonary TB among patients suspected for TB upon admission to the hospital; (2) included at least 50 adult patients; (3) used sputum cultures for Mycobacterium TB as the referent standard; and if (4) the absolute numbers of true-positive, false-negative, true-negative, and false-positive observations were available or derivable for the data presented. If a single study conducted a derivation and validation of a prediction model using independent populations, both estimates of the accuracy of the decision model were included in the systematic review.

Assessment of Study Quality

The methodologic quality of the selected studies was assessed, and data were abstracted independently by 2 reviewers (C.M. and D.S.). A checklist adapted from Lapaucis et al.,¹⁶ Irwig et al.,¹⁷ and the Evidence-Based Medicine Working Group¹⁸ was used for quality assessment. Reviewers were blinded to details of authorship and journal of publication.

Variables extracted included the study design, characteristics of the study population (age, gender, and ethnicity distribution, and percentage of HIV-positive patients), clinical variables evaluated and whether they were clearly defined, and the diagnostic performance of the prediction rule.

The optimal design for assessing the accuracy of a diagnostic test is considered to be a prospective blind comparison of the test and the reference standard in a consecutive series of patients from a relevant population.^19,20 To assess the quality of the studies included in the systematic review, data were collected regarding the study design, patient spectrum, the presence of verification or selection bias, and the statistical techniques. We considered that predictor variables were clearly defined if the authors provided a precise enough definition so that it would have a similar meaning to everyone who may use them.²¹ We also assessed whether the authors reported details about the reference standard, including the method to obtain samples, the type of medium used to culture Mycobacterium TB, and the duration of the incubation period. Studies were classified as blinded if the presence of predictors was ascertained without knowledge of the results of the TB culture results.

An adequate description of the spectrum of patients included in a study can help clinicians know whether to generalize the results to their patients. We considered that a study met this criterion if the following information was provided about the study population: (1) demographics, (2) proportion of HIV-positive patients, and (3) the distribution of symptoms in patients with and without TB.

Verification bias occurs if the decision to perform the reference standard is based on the result of the prediction rule under study. For instance, if patients with positive, as opposed to negative, results for the prediction model preferentially receive the reference standard evaluation, the sensitivity of the test can be falsely elevated because of the incorrect exclusion of false negatives from the analysis. In cases in which not all patients were subjected to the reference standard, the study was scored as having verification bias.²²

Selection bias can be present when not all patients presenting with the relevant condition are included in order of entry (consecutive) into the study, or when this selection is not random. If it was not clear from the text that a consecutive series of patients was included or a random subset, the corresponding study was scored as nonconsecutive.

Finally, we assessed the possibility of publication bias; the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings.^23,24

Statistical Analysis

To evaluate agreement between raters for the assessments of study quality, we calculated the κ coefficient for interrater reliability.^25,26

For each study, we calculated sensitivity and specificity according to standard methods.²⁷ The 95% confidence intervals for sensitivity and specificity were estimated according to the binomial distribution.

To assess publication bias, we created an inverted funnel plot of individual study log odds ratio (OR) (log odds of true-positive rate-log odds of false-positive rate) plotted against sample size.²⁴ An asymmetrical funnel plot would suggest that additional small studies may have been conducted but not published because of negative results.

RESULTS

Selection of Studies

A total of 439 citations were screened, and 11 met the selection criteria.^4,28–37 Three studies reported results for overlapping patient populations.^30,32,36 To avoid duplication, we retained the second report³⁰ as it included a larger number of patients and used statistical methods comparable with the other studies in the systematic review. Of the 9 studies included in the analysis, 5 were formal prediction rules and 4 reported the accuracy of combination of clinical predictors of TB. Three studies derived and validated a prediction model in independent groups of subjects.^30,34,37 The characteristics of the studies are outlined in Table 1. Additional details the predictor variables and scoring methods for the prediction rules are provided in an online appendix.

Table 1.

Characteristics of the Studies Included in the Systematic Review

Study (Year)	Total Number of Participants	TB Patients (%)	Study Design	Setting	TB Patients with Positive AFB Smear (%)	Sensitivity, (%, 95% CI)	Specificity, (%, 95% CI)
Scott et al. (1994)	86	50	Retrospective evaluation of predictors	Inpatient and outpatient	67	86 (72 to 95)	84 (69 to 93)
Bock et al. (1996)	295	17	Prospective evaluation of predictors	Inpatient	76	81 (66 to 90)	62 (56 to 68)
Cohen et al. (1996)	101	44	Prospective evaluation of predictors	Inpatient	75	98 (88 to 100)	19 (11 to 31)
El-Solh et al. (1997)	563	9	Retrospective derivation and validation	Inpatient	45	100 (93 to 100)*	48 (44 to 52)*
						100 (78 to 100)†	50 (44 to 57)†
Gaeta et al. (1997)	103	27	Prospective validation	Inpatient	—	96 (81 to 100)	14 (8 to 25)
Redd et al. (1997)	141	25	Retrospective derivation	Emergency Department	43	96 (85 to 100)	54 (44 to 64)
Tattevin et al. (1999)	277	30	Prospective derivation and retrospective validation	Inpatient	55	100 (94 to 100)*	49 (41 to 57)*
						91 (78 to 98)†	39 (23 to 59)†
Wisnivesky et al. (2000)	112	50	Retrospective derivation	Inpatient	54	98 (91 to 100)	46 (33 to 60)
Wisnivesky et al. (2005)	516	4	Prospective validation	Inpatient	73	95 (74 to 100)	36 (31 to 40)

^*Results obtained from derivation study.

^†Results obtained from validation study.

TB, tuberculosis; CI, confidence interval; AFB, acid-fast bacilli.

The 9 studies in the systematic review included 2,194 participants. The mean age of participants ranged from 37 to 46 years and 55% to 89% of the participants were men. Fifty-eight percent of the patients with pulmonary TB in these studies had positive sputum AFB smears. Overall, approximately 18 patients without TB for every patient who had culture-proven TB were placed on respiratory isolation.

Table 2 shows the clinical predictors of TB identified on the studies. Overall, there was considerable overlap among studies. Most studies included TB risk factors or chronic symptoms among the predictors in the model.^{29–31,33–35} The definition of these variables however, varied among studies. Three studies ^30,31,34 reported a significant association between the patient's HIV status as the presence of a positive culture for TB. Approximately half of the models included self-reported positive tuberculin skin test (TST) and the presence of fever on admission as predictors of TB. All studies included specific findings on the admission chest X-ray (CXR) in the decision model (upper lobe infiltrates and CXR consistent with TB). Based on ORs reported on individual studies, self-reported positive TST and CXR consistent with TB were the predictors showing the strongest association with TB (OR: 5.7 to 13.2 and 2.9 to 31.7, respectively). The sensitivity of the decision models for identifying patients with active pulmonary TB who require respiratory isolation varied from 81% to 100%; specificity ranged from 19% to 84% (Table 1).

Table 2.

Clinical Predictors of TB in the Studies Included in the Systematic Review

Study (Year)	Tuberculosis Risk Factors or Chronic Symptoms*	HIV Status	Self-reported Positive Tuberculin Skin Test	Fever	Upper Lobe Infiltrates on CXR†	Other CXR Findings	Other Variables
Scott et al. (1994)	—	—	—	—	—	CXR consistent with TB‡	—
Bock et al. (1996)	—	—	Yes	—	Yes	Cavities	Prior use of INH§
Cohen et al. (1996)	Yes	—	—	—	—	CXR consistent with TB∥	—
El-Solh et al. (1997)	Yes	Yes	—	Yes	Yes	—	—
Gaeta et al. (1997)	Yes	Yes	Yes	—	Yes	CXR consistent with TB¶	—
Redd et al. (1997)	Yes	—	Yes	Yes	—	CXR abnormalities	—
Tattevin et al. (1999)	Yes	Yes	—	—	—	CXR consistent with TB#	—
Wisnivesky et al. (2000)	Yes	—	Yes	Yes	Yes	—	Shortness of breath and crackles on lung exam**
Wisnivesky et al. (2005)	Yes	—	Yes	Yes	Yes	—

^*The definition of TB risk factors and chronic TB symptoms varies in different studies. TB risk factors included: exposure to an individual with TB, prior history of TB, injection drug use, homelessness, and institutionalization [prison, shelter, or nursing home]. Chronic symptoms included: chronic cough, nonpurulent sputum production, weight loss, fever, night sweats, anorexia, and weakness or malaise.

^†Chest radiograph.

^‡Defined as the presence of one or more of the following findings: cavitary pneumonia, apical infiltrates or masses, or diffuse nodular infiltrates.

^§Prior use of INH was found to be associated with the absence of TB (odds ratio 0.18).

^∥Defined as the presence of nodular, alveolar, or interstitial infiltrates predominantly affecting the zones above the clavicles or upper zones.

^¶CXR showing apical densities/upper lobe infiltrates, mediastinal adenopathy, or cavitary lesions.

^#Defined as the presence of nodular, alveolar, or interstitial infiltrates predominantly affecting the zones above the clavicles or upper zones (typical for TB) or enlarged hilar nodes, pneumonic lesion, atelectasis, mass lesion, miliary, or pleural effusion (compatible with TB).

^**Both variables were associated with the absence of TB.

TB, tuberculosis; CXR, chest X-ray; INH, isoniazid.

Methodologic Quality and Study Characteristics

The observed interrater agreement for assessment of study quality was substantial (κ=0.70).²⁶ The results of the quality assessment of the included studies are listed in Table 3. Four studies used a case-control and 5 a cohort design. Details regarding the reference standard (i.e., TB cultures of the sputum) were adequately reported, except in 2 studies.^31,33 Most studies clearly defined clinical predictors of TB and the characteristics of the study population. Six studies (67%) reported that the presence of predictors was ascertained without knowledge of the results of the TB cultures.^{28,30,33–35,37} No study used selective inclusion of patients (selection bias). Verification bias seemed to be present in only 2 studies,^4,33 potentially inflating the sensitivity and decreasing the specificity of the prediction model. The statistical methods used to identify clinical predictors of TB were described in all but 2 studies.^29,31

Table 3.

Quality of Studies Included in the Systematic Review

Study (Year)	Were Predictor Variables Clearly Defined?	Were Details of the Reference Standard Reported?	Was the Presence of Predictors Ascertained without Knowledge of the Results of TB Culture?	Were Important Patient Characteristics Reported?	Was Selection Bias Present?	Was Verification Bias Present?	Were the Statistical Methods Reported?
Scott et al. (1994)	Yes	Yes	Not stated	Yes	No	Yes	Yes
Bock et al. (1996)	Yes	Yes	Yes	Yes	No	No	Yes
Cohen et al. (1996)	Yes	Yes	Not stated	Yes	No	No	No
El-Solh et al. (1997)	Yes	Yes	Yes	Yes	No	No	Yes
Gaeta et al. (1997)	Yes	No	Not stated	No	No	Yes	No
Redd et al. (1997)	No	No	Yes	Yes	No	No	Yes
Tattevin et al. (1999)	Yes	Yes	Yes	Yes	No	No	Yes
Wisnivesky et al. (2000)	Yes	Yes	Yes	Yes	No	No	Yes
Wisnivesky et al. (2005)	Yes	Yes	Yes	Yes	No	No	Yes

DISCUSSION

The current study presents a systematic review of the literature on the accuracy of clinical variables for the prediction of TB among patients suspected for the disease at admission to the hospital. The sensitivity of the clinical prediction rules to identify cases of active pulmonary TB was high. Thus, our analysis suggests that among adult patients, who are suspected for pulmonary TB, those with very low risk of the disease could be discriminated from those with a higher risk of TB based on relatively simple clinical variables immediately available at the time of admission. Infection control policy incorporating one of these prediction models could decrease the frequency of isolation of patients without TB thereby reducing the cost of hospitalization.

Presence of TB risk factors or chronic symptoms, self-reported positive TST, fever, and upper lobe abnormalities on CXR were found to be predictors of TB in most studies. Among these, self-reported positive TST and upper lobe infiltrates on CXR showed the strongest association with the presence of TB. These clinical predictors should be considered carefully in deciding whether a patient needs to be isolated. Additionally, physicians should give less importance to other factors (such as hemoptysis or admission white blood cell count) that failed to show predictive power in all the studies reviewed.

Based on their methodological quality, 3 prediction models,^30,34,35 appear to provide the highest level of evidence. These models were assessed in a relatively large population that included a wide spectrum of patients, have been validated in an independent group of patients, and fulfilled all quality standard criteria evaluated as part of the systematic review. The high sensitivity of these models found in our analysis suggests that clinicians or institutions interested in changing their respiratory isolation policy can consider using them. There is considerable concordance among the predictors in these models and their use would lead to similar recommendations in clinical practice. Using the estimates of accuracy obtained in the validation studies and assuming a prevalence of TB of 5% among isolated cases (1:18 ratio), use of these models would avoid unnecessary isolation of 6 to 9 patients without TB for every case of TB admitted to the hospital. None of these models however, has been subjected to an impact analysis to evaluate if they actually change physician behavior while maintaining quality of care.¹⁸

The clinical prediction rules derived by Redd and Susser³³ and Bock et al.²⁸ appears to have the best quality review among the other studies included in the systematic review. These models however, have been not prospectively validated and Redd and Susser failed to clearly specify some of the clinical predictors and did not provide details about the reference standard. The remaining studies did not clarify whether an assessment of predictors was conducted without knowledge of the results of the TB culture. This is a potential major flaw that may considerably bias the estimates of accuracy.

Rapid isolation of patients with active TB is essential to avoid transmission of Mycobacterium TB. The emergence of multidrug-resistant strains^38,39 has made this issue even more important. In response to this threat, the Centers for Disease Control and Prevention has issued guidelines for controlling the transmission of TB within heath care institutions.¹ These guidelines dictate that patients at risk of TB should be placed in single-bed negative pressure rooms until the results of 3 consecutive AFB smears are found to be negative or effective anti-TB treatment is initiated and the patient is clinically responding.

Although these policies have been shown to decrease the rate of TB transmission in certain institutions,^2,40 the combination of these expanded isolation guidelines and declining rates of TB in the United States have lead to a large number of patients without TB being placed in respiratory isolation. For example, in the studies included in the systematic review, approximately 18 patients without the disease were placed on respiratory isolation for every patient who had culture-proven TB. Thus, the present strategy of isolation may be improved by better discriminating patients at low risk of TB who do not require isolation.

Ideally, respiratory isolation should be initiated for all patients with TB who are admitted to the hospital. Despite the high sensitivity of the prediction models included in the systematic review, a more restrictive respiratory isolation policy may fail to identify a small percentage of TB patients. However, all these models are more sensitive than AFB smears, one of the most commonly used standards to discontinue isolation. These models could be used to identify patients at very high risk of TB that may require further diagnostic tests before isolation is discontinued based on the results of negative AFB smears. Further research is necessary to determine the usefulness of these prediction models to detect AFB smear negative-culture positive patients limiting exposure of health care workers and other patients to TB.

One of the advantages of the models in the systematic review is that the variables used to predict the risk of active TB are readily available in a routine history, physical examination, and radiographic findings at the time of admission, so that they can be applied with no additional cost. Another advantage is that most of the models include a limited number of predictors, are relatively easy to use, and can be applied to all patients with suspicion for TB whether they are HIV-infected or not.

Our study has some methodological limitations. To our knowledge, this is one of the first systematic reviews of clinical prediction rules.⁴¹ Given the lack of specific guidelines, the analysis was conducted following published guidelines for systematic review of diagnostic tests.^42,43 However, there are important differences between classical diagnostic tests and prediction models. The prediction models in the systematic review included different variables; thus, we could not obtain a pooled estimate of the accuracy of the models nor were able to evaluate the adjusted weights of individual predictor variables. This problem however, is not unique to clinical prediction rules. More typical diagnostic tests may be performed using different techniques or be assessed using different methods of interpretation. For example, a recent meta-analysis of positron emission tomography for identifying malignant lesions pooled the result of studies that reported different technical methods to obtain and interpret the images.⁴⁴

In summary, our analysis suggests that clinical prediction rules applied at the time of admission could be used to identify patients who are at very low risk for TB and do not require respiratory precautions. Further research should assess the impact of clinical use of these models by comparing outcomes of patients triaged according with the models with outcomes of patients isolated according to the current guidelines.