Screening for pulmonary tuberculosis in a Tanzanian prison and computer-aided interpretation of chest X-rays

Abstract

Setting: Tanzania is a high-burden country for tuberculosis (TB), and prisoners are a high-risk group that should be screened actively, as recommended by the World Health Organization. Screening algorithms, starting with chest X-rays (CXRs), can detect asymptomatic cases, but depend on experienced readers, who are scarce in the penitentiary setting. Recent studies with patients seeking health care for TB-related symptoms showed good diagnostic performance of the computer software CAD4TB.

Objective: To assess the potential of computer-assisted screening using CAD4TB in a predominantly asymptomatic prison population.

Design: Cross-sectional study.

Results: CAD4TB and seven health care professionals reading CXRs in local tuberculosis wards evaluated a set of 511 CXRs from the Ukonga prison in Dar es Salaam. Performance was compared using a radiological reference. Two readers performed significantly better than CAD4TB, three were comparable, and two performed significantly worse (area under the curve 0.75 in receiver operating characteristics analysis). On a superset of 1321 CXRs, CAD4TB successfully interpreted >99%, with a predictably short time to detection, while 160 (12.2%) reports were delayed by over 24 h with conventional CXR reading.

Conclusion: CAD4TB reliably evaluates CXRs from a mostly asymptomatic prison population, with a diagnostic performance inferior to that of expert readers but comparable to local readers.

Tuberculosis (TB) is responsible for at least 1.5 million deaths annually despite the existence of effective treatment. Data from national TB programmes report 6 million new cases per year, but this figure is probably still 3 million short of the actual number, as many cases are not properly diagnosed or reported.¹ Tanzania is one of 22 high burden countries, with a TB prevalence of 295 per 100 000 population, as shown in the latest national TB prevalence survey.² The problem is accentuated in prisons, where the incidence and prevalence of TB are many times higher than in the general population.^3,4 In particular, prisons in sub-Saharan Africa have a high turnover of detainees, while factors such as overcrowding, poor nutrition and illegal drug use are likely causes for the high TB prevalence of up to 5%.^5,6 Previous studies in prisons in Tanzania have shown a high proportion of new cases and smear-positive TB.⁷ Preliminary data from a study in Mbeya, Tanzania, estimate TB prevalence at 0.88–4.38% in Tanzanian prisons.⁸

Early detection is particularly important for the effective prevention and control of TB in prisons.⁹ The World Health Organization recommends systematic screening of prisoners in countries with high TB prevalence, proposing chest X-ray (CXR) as an initial examination.¹⁰ Radiography would detect more active TB cases than the clinical questionnaires that are currently used to discover TB cases in African prisons.³ Unfortunately, the diagnostic accuracy of CXRs for the detection of active pulmonary TB (PTB) is highly dependent on the reader,^11,12 and health personnel with TB-specific radiology training are scarce in sub-Saharan Africa, especially in the health facilities of penitentiary institutions.

Digital X-ray equipment and the automated reading of CXRs with computer-aided detection (CAD) have the potential to expand the use of radiology in the diagnosis of PTB and to increase its objectivity. Previous studies have established good diagnostic performance of the software Computer Aided Detection for Tuberculosis (CAD4TB, Delft Imaging Systems, Veenendaal, The Netherlands) in a patient population seeking care for TB-related symptoms.^13–15

In this study, we examined the potential benefits of CAD4TB when screening a prison population for active PTB and assessed the diagnostic accuracy of CAD4TB in a predominantly asymptomatic prison population. In addition, we recorded parameters to assess the operational feasibility of such screening. All data were collected during ongoing TB screening of the entire prison population.

METHODS

Design

This was a cross-sectional study in prisoners of the Ukonga prison in Dar es Salaam, Tanzania, conducted from September 2013 to October 2014.

Screening algorithm

This study was linked to a TB REACH project assessing the application of Xpert^® MTB/RIF (Cepheid, Sunny-vale, CA, USA) as a routine screening tool in Tanzanian prisons. All prisoners and new arrivals to Ukonga prison first underwent a CXR as screening test, regardless of whether or not clinical TB symptoms were present, followed by an Xpert assay of sputum in case of ‘any abnormality’ on the CXR. In the event of a positive TB diagnosis, participants received prompt, effective TB treatment on-site according to the Tanzania National Tuberculosis and Leprosy Programme guidelines.

X-ray data sets

All X-ray images were generated on a digital Odelca DR system (Delft Imaging Systems).

A total of 1321 CXRs were acquired during prisoner screening from September 2013 to October 2014. This operational data set was extracted from the picture archive and communication system (PACS), together with the date/time of the image acquisition and the first electronic report filed to the system. The data were used to confirm that CAD4TB works autonomously and to determine the time to detection due to a reporting delay; no additional readings were performed on these images.

A subset of 511 consecutive images (Figure 1) was manually linked to epidemiological information from the database of the TB REACH study and read by multiple X-ray readers. This accuracy data set was used to assess diagnostic performance and examine the utility of the CAD4TB-generated overlay.

FIGURE 1.

Accuracy data set and radiological reference. The numbers reflect the classification in the radiological expert consensus. For the purposes of this study, categories 1+2 were considered TB-negative and categories 3+4 TB-positive. PTB = pulmonary tuberculosis.

X-ray reading and radiological reference

Images were classified into the following four categories: 1) ‘normal CXR’, 2) ‘abnormal CXR, not suggestive of active pulmonary TB’, 3) ‘abnormal CXR, consistent with active pulmonary TB’, and 4) ‘abnormal CXR, highly suggestive of active pulmonary TB’.

All CXRs from the accuracy data set were evaluated multiple times: two experienced TB radiologists viewed the images at their working station and reached a consensus that served as the radiological reference for further analysis (using categories 1+2 vs. 3+4; see Figure 1). The non-expert local readers all routinely interpret CXRs in TB wards in the Dar es Salaam region and were chosen to represent different levels of professional education, training and experience in CXR reading (Table 1). They were introduced to the viewing software (Rogan View Pro-X, Rogan-Delft, Veenendaal, The Netherlands) and viewed all images once, followed by a second reading after a few weeks with an additional overlay (Figure 2, right side). All readers were blinded to clinical information, and the images were shown in random order. The perceived difference due to the overlay was evaluated using a structured questionnaire after the second reading.

TABLE 1.

Characteristics of local X-ray readers

FIGURE 2.

CAD4TB image analysis. An image classified as 4 (highly suggestive of active PTB) by the expert consensus. Raw image on the left side and computer-generated overlay (i.e., the output of lung parenchyma texture analysis subsystem)¹⁹ of the same image on the right side. Note the highlighting (yellow/red) of the parenchymal lesions in the apical zones. CAD4TB abnormality score 87 (out of 100). CAD4TB = Computer Aided Detection for Tuberculosis; PTB = pulmonary tuberculosis.

All images from the accuracy data set as well as the operational data set were also analysed using CAD4TB (version 3.07, developed by the Diagnostic Image Analysis Group at Radboud University Medical Center, Nijmegen, The Netherlands), explained in detail elsewhere.^14,15 The software ran on the same computer that was used as the viewing station and analysed all images in the background autonomously after initial installation.

Data analysis

The data were analysed using R version 3.1.0 (R Foundation for Statistical Computing, Vienna, Austria). Receiver operating characteristics (ROC)¹⁷ were generated using the pROC package,¹⁸ and bootstrapping was used to generate confidence intervals in the ROC plot. McNemar's test was used to compute P values when comparing the sensitivity of human readers and CAD4TB at assumed levels of specificity. The statistical significance level of the change of the number of false positives and false negatives across all readers when using the overlay was calculated using conditional logistic regression, stratifying by CXR and using a fixed effect for reader.

Ethics approval

Ethical clearance was obtained from the National Institute for Medical Research in Tanzania and local ethics committees. Permission to conduct the study was obtained from the prison authorities. All participants signed an informed consent form that was translated into the local language (Swahili). In case of illiteracy, the consent form was explained orally, and the participant confirmed his/her agreement with a fingerprint in the presence of a witness. Information was linked to a unique study ID number, and the data set contains no personal identifying information.

RESULTS

Characteristics of the study population

All 511 prisoners in the accuracy data set (Table 2) were male, with a median age of 37 years (IQR 31–45); 1 (0.2%) was on anti-tuberculosis treatment when screened, 82 (16.1%) had a medical history that included previous anti-tuberculosis treatment, and 58 (11.4%) were human immunodeficiency virus (HIV) positive. The radiological reference classified 64 prisoners (12.5%) as positive for active PTB (Figure 1), of whom 13 (20.3%) tested positive for HIV.

TABLE 2.

Population characteristics (accuracy data set, n = 511)

Diagnostic performance

The radiological reference (Figure 1) served as the gold standard, and ‘any abnormality’ (category 1 vs. 2+3+4) was used as a threshold for the other readers (Table 1) to classify images as TB positive or negative. CAD4TB achieved an area under the curve (AUC) of 0.75 (95% confidence interval [CI] 0.68–0.82; Figure 3). The different readers achieved a sensitivity ranging from 42.2% to 71.9%, and a specificity from 45.9% to 90.6% (Figure 3). Average values were 63.4% for sensitivity and 75.4% for specificity. Two readers performed significantly better than CAD4TB (assistant medical officer [AMO], P = 0.001; medical doctor [MD] 1, P = 0.026), two readers performed significantly worse than CAD4TB (clinical officer [CO] 1, P = 0.021; CO2, P = 0.012), and there was no evidence of a difference between CAD4TB and the other three readers (MD2, MD3, CO3).

FIGURE 3.

Receiver operated curve¹⁷ showing diagnostic performance. The bold line shows the sensitivity/specificity of CAD4TB when using different thresholds for the abnormality score. The cross represents an exemplary cut-off of 70, with resulting specificity/sensitivity in parentheses. AUC = area under the curve; shaded area = confidence interval; CAD4TB = Computer Aided Detection for Tuberculosis.

CAD4TB-generated overlay

The re-reading of the accuracy data set with the additional CAD4TB-generated overlay (Figure 2) had varying effects on the assessment by the different readers (Figure 4). Using the same criteria as for the diagnostic performance (category 1 vs. 2+3+4 and the radiological reference as a gold standard), the average number of false negatives decreased by 17%, from 23.4 to 19.4 (P > 0.5), while the average number of false positives increased by 35%, from 110.0 to 148.6 (P < 0.01). Using the threshold of ‘TB specific abnormalities’ (categories 1+2 vs. 3+4) for the readers resulted in a decrease in the average number of false negatives by 9.6%, from 40.1 to 36.3 (P > 0.5), and a decrease in the average number of false positives by 21.2%, from 29.7 to 23.4 (P > 0.5).

FIGURE 4.

Effect of overlay on false-negatives and false-positives. Stacked barplot shows number of false-negatives and false-positives of individual readers compared to the radiological reference. The left subplot uses the classification ‘any abnormality’ (category 1 vs. 2+3+4); the right subplot uses the classification ‘TB specific abnormalities’ (categories 1+2 vs. 3+4). The height of the barplot represents the number of misclassified images. TB = tuberculosis; AMO = assistant medical officer; CO = clinical officer; MD = medical doctor.

All CXR readers perceived the overlay as helpful, 6 of 7 wanted to use such an overlay in daily practice, 5 of 7 felt more confident in their findings when using the overlay, and 4 of 7 had the impression of analysing the images faster.

Reporting delay

Within the operational data set (n = 1321), CAD4TB autonomously calculated the abnormality score for 1310 CXRs (99.2%). This analysis was performed as soon as the image was taken. In contrast, an analysis of the delay between the image acquisition and the filing of the radiology report in the PACS system showed that 160 (12.2%) reports were delayed by more than 24 h by the X-ray reader of the TB REACH study (Table 3). Of the 160 delayed reports, 149 (93.1%) were clustered in 6 screening days only.

TABLE 3.

Delay of X-ray reports during prisoner screening

DISCUSSION

This study evaluated for the first time the automated reading of CXRs in a penitentiary setting for active case finding of PTB. We used CAD4TB for the first time without any calibration for the X-ray equipment, and the analysis was performed without any additional intervention by the operators. The population screened, 91.5% of whom had none of the typical TB symptoms (see Table 2), was markedly different from previous studies, which examined the performance of CAD4TB with symptomatic patients.^14,15,20

CAD4TB performance was inferior to that of two experienced CXR readers, but better than, or comparable to, that of the other five readers (Figure 3; Table 1). This relative result is in line with previous studies,^15,21 and further validates their findings by comparing the performance of CAD4TB with that of a more varied sample of readers. A limitation of this study is that the CAD4TB results could not be measured against a microbiologically confirmed Mycobacterium tuberculosis sample, only a radiological reference (Figure 1). We expect this to influence the absolute values of the measured sensitivity and specificity, which should be interpreted with care. However, experienced TB radiologists are believed to detect typical changes in CXRs reliably,^22,23 and other readers (and computer programmes) interpret the same signs on these same images. Consequently, the radiological reference is adequate for determining relative performance.

In previous work, the diagnostic accuracy of CAD4TB has been established against a biological reference: Maduskar et al. compared it against culture and found an AUC of 0.73,²⁰ while Moyoyeta et al. compared it against Xpert, with an AUC of 0.71.¹⁴ Breuninger et al. compared it against culture (excluding non-tuberculous mycobacteria, clinically presumed culture-negative cases and extra-pulmonary TB), and reported an AUC of 0.84.¹⁵ The performance of X-ray readers in identifying TB in CXRs was found to be highly variable, and although many studies report high sensitivity (sometimes up to 100%)²⁴ and low specificity, some authors found the specificity to be higher than the sensitivity.^25,26

All readers responded favourably to using CAD4TB as a reading aide, but the effects on reading performance were inconclusive. We preferred to show a coloured overlay instead of the final abnormality score (Figure 2) because the colour code conveys information about the texture analysis and could reduce the number of missed subtle parenchymal changes. Our results (Figure 4) suggest a trend toward reduced false negatives, as expected, but this effect is not very marked, and considerable inter-reader differences persist.

CAD4TB analysed over 99% of all X-ray images and provided the final result in under 2 min. Although human readers can interpret most CXRs very quickly, physical absence of the reader can lead to delayed diagnosis. In our case, the physical absence of the assistant medical officer responsible for reading the images for a few days (Table 3) resulted in a >24 h delay in reading 12.2% of all images. These delays were the cause of further logistic problems along the diagnostic algorithm. Our experience suggests that prompt analysis of all images is a very desirable property of the automated analysis. This is particularly true in a context where many patients are screened sequentially and other tests are performed depending on the result of the CXR examination. If this result is delayed by more than a few hours, patients have to be sent home and scheduled for follow-up visits, considerably increasing the logistic complexity of the screening procedure.

Computer-assisted reading of CXRs is a promising screening tool in a prison setting. Radiography is well established as a first step for active case finding in prisons,^9,10 but skilled radiologists are scarce in low-income countries, which have the highest TB burden. A recent review found that CXR is not commonly used in African prisons and cited a lack of skilled and motivated manpower as a limiting factor in TB control.³ Our study illustrates that diagnostic performance varies strongly between X-ray readers, depending on their experience and personal ability. Automated analysis offers the advantages of objectivity and a highly predictable delay.

The abnormality score is the final output of CAD4TB. Using a fixed low threshold value results in a higher sensitivity but a lower specificity (upper right part of the graph in Figure 3). The follow-up examination in our diagnostic algorithm (Xpert) has a very high specificity and readily filters out the false positives from the preceding step. However, these false positives increase costs and limit the maximum achievable throughput. Alternatively, the threshold can be adapted dynamically to use the follow-up examination to its full capacity.

In conclusion, this article shows that CAD4TB performs with a diagnostic accuracy comparable to that of readers typically available in a low-income setting. Findings from previous studies could be extended to the screening of a prison population with a high percentage of asymptomatic screening subjects. In addition, automated analysis causes a minor and very predictable delay and is therefore ideally suited in multistep screening procedures in a logistically challenging environment.