Diagnosing pneumonia in rural Thailand: Digital cameras versus film digitizers for chest radiograph teleradiology

Abstract

Background

Accurate surveillance for pneumonia requires standardized classification of chest radiographs. Digital imaging permits rapid electronic transfer of data to radiologists, and recent improvements in digital camera technology present high quality, yet cheaper, options.

Methods

We evaluated the comparative utility of digital camera versus film digitizer in capturing chest radiographs in a pneumonia surveillance system in rural Thailand using a panel of radiologists; the gold standard was the hard-copy radiograph. We calculated sensitivity and specificity and conducted a receiver operator characteristics (ROC) analysis.

Results

Of the 192 radiographs from patients with clinical pneumonia, 166 (86%) were classified as pneumonia on the hard copies. Sensitivity and specificity for identifying pneumonia were 89% and 73% for the camera and 90% and 65% for the digitizer. In the ROC analysis, there was no statistically significant difference in the area under the curve (camera, 0.86; film digitizer, 0.91, p = 0.29). The digital camera set cost $965 compared to $3000 for the film digitizer.

Conclusion

Detection of pneumonia was not measurably compromised by using digital cameras compared with film digitizers. The 3-fold lower cost of the digital camera makes this technology an affordable and widely accessible alternative for surveillance systems, vaccine trials, and perhaps clinical use.

Introduction

Pneumonia is a leading cause of morbidity and mortality worldwide.1, 2 However, there are few inexpensive and uniformly effective tools to aid in its clinical identification as well as in the investigation of its etiology.3, 4, 5, 6, 7, 8, 9 Chest radiographs are recommended for the routine clinical evaluation of individual patients hospitalized with suspected pneumonia.4, 10 In public health, they may also be used to quantify and refine estimates of the vaccine-preventable fraction of pneumonia; this approach is particularly useful when blood cultures are not routinely performed.

Vaccine trials are designed to measure disease reduction in a vaccinated group compared to a control group. However, the amount of disease reduction can also be used as an estimate of the burden of that disease in the population. For example, several vaccine trials for Haemophilus influenzae type b (Hib) demonstrated declines in radiographically confirmed pneumonia of 21% to 22% in the vaccinated group, suggesting that up to 22% of pneumonia may be attributable to Hib infection.11, 12 More recently, vaccine trials of pneumococcal conjugate vaccines demonstrated declines in pneumonia with alveolar consolidation of 17–18% in the vaccinated group, suggesting that up to 18% of pneumonia with alveolar consolidation may be attributable to Streptococcus pneumoniae infection.13, 14 If the incidence of radiographically confirmed pneumonia, or pneumonia with alveolar consolidation, is known, then the vaccine preventable fraction of Hib or S. pneumoniae can be estimated. Such estimates are important for countries that are considering adopting new vaccines into their national immunization programs.

In addition to the importance of chest radiography for individual patient care and estimating the burden of disease at the population level, recent events such as the worldwide outbreak of severe acute respiratory syndrome (SARS) highlight the need for rapid, effective, inexpensive tools for recording, archiving and transmitting radiographic images from patients with pneumonia.15, 16 While there are a number of studies that have evaluated the utility of film digitizers, flatbed scanners, or digital cameras, there have been no recent studies evaluating the newer generations of high-resolution digital cameras with respect to interpretive accuracy and cost-effectiveness.17, 18, 19, 20, 21, 22, 23, 24, 25 In this study, we evaluated the comparative utility of digital still cameras versus film digitizers in capturing chest radiograph images in the setting of a pneumonia surveillance project in rural Thailand.

Methods

Study population

In August 2002, the International Emerging Infections Program, a collaboration between the Thailand Ministry of Public Health and the US Centers for Disease Control and Prevention, launched active, population-based surveillance for radiologically confirmed pneumonia in Sa Kaeo Province (population 438 557, Provincial Health Census 2001). Residents admitted with evidence of acute infection and signs or symptoms of respiratory illness have a chest radiograph taken within 48 hours of admission. For the purpose of the study, chest radiographs were taken from the three largest hospitals in this surveillance system to maximize the number of chest radiographs while limiting workload.

Study design

The objective was to compare two digital image modalities for capturing chest radiographs, digital camera and film digitizer, to a hard-copy gold standard. Chest radiographs from 192 patients were selected consecutively from three hospitals in Sa Kaeo; the number selected from each hospital was calculated to be proportional to the number of suspected pneumonia cases (those presenting with evidence of acute infection and signs or symptoms of respiratory infection) with chest radiographs taken. A trained surveillance officer at each hospital scanned and photographed the hard copy images according to standard written guidelines.

Briefly, radiographs were scanned on Vidar SIERRA Plus film digitizers (Vidar Systems Corp., Herndon, VA, USA) according to the manufacturer's instructions. Films were scanned at settings of 12 bits and 150 dots per inch and the images were saved as Tagged Image Format (TIF) files using Microsoft Photo Editor 3.0 (Microsoft Corp., Redmond, WA, USA) and the digitizer's proprietary TWAIN interface standard. Using Photo Editor, the images were contrast-adjusted with the AutoBalance function and all personal identifiers were removed. To ensure that image quality was not degraded, the files were converted to true color (24 bit) format, an option that increases file size by fifty percent.

Digital camera images were taken using a Sony Mavica MVC-CD300 camera (Sony, Tokyo, Japan) with 3.3 mega pixels and 6× optical zoom. Chest radiographs were placed on standard two-switch lightboxes, available in each of the hospitals, with only one panel illuminated; each panel contained two fluorescent light tubes of 15 or 20 watts each. An opaque cardboard screen was used to cover all parts of the lightbox that were not covered by the radiograph. Ambient light was limited, and room lights were dimmed with only one fluorescent bulb at some distance behind the camera kept on. Images were recorded as follows: TIFF image, black and white, resolution of 2048 × 1536 pixels (3.3 mega pixels), no flash, manual focus set to 1.0 meters, exposure value was set to +1.0, white balance and ISO set to auto, picture quality set to fine, and sharpness set to zero. The camera was mounted on a tripod with the height raised so that the camera was level with the center of the radiograph on the lightbox, and a string was used to ensure the length was one meter away from the lightbox surface. A minimal adjustment was made with the zoom in order to center and fill the camera viewfinder screen with the radiograph image.

The radiologists used the WHO standard criteria for the interpretation of chest radiographs for diagnosing pneumonia in children.²⁶ In addition, a pilot study of 100 images was conducted so that readers could compare and standardize interpretations. The digital images, camera and digitizer combined, were randomized and read; after a delay of two weeks, the 192 hard copies were read. A panel of board-certified radiologists in Bangkok reviewed the radiographs. Two primary radiologists reviewed all and a third reviewed images only when there were discrepancies between the results of the first two with respect to the key indicators of possible pneumonia, alveolar infiltrate, or interstitial infiltrate. The final interpretation was based on the first two readers, if in agreement, or on agreement between a primary reader and the third reader, if there were discrepancies between the first two readers. Each film was read independently and the reviewers were discouraged from re-reviewing images; reviewers were blinded to each other's readings. The only information available was age and sex of the patient, patient positioning if known, and a centimeter marker for film size. In addition, readers were aware that the images were from patients suspected of having pneumonia. Digital images were loaded onto a Vepro DICOM Archive and Viewing Station, Model ADAS-DVD, using MEDIMAGE software (Vepro GmbH, Pfungstadt Germany) and viewed on 19-inch cathode ray tube dual display monitors with dot pitch of 0.26 mm and maximum resolution of 1920 × 1440 pixels. Image manipulation by the readers, including zoom, window- and level-adjustment, and image inversion, was encouraged.

The radiologists completed a two-page data collection form that recorded information on image quality, findings relevant to the radiological appearance of pneumonia (e.g., infiltrates), and other pathologic findings (e.g., pleural effusion, atelectasis, hilar lymphadenopathy, cavitation). Overall image quality was categorized as adequate, suboptimal, and unreadable, modified from WHO guidelines. In addition, a modified version of a published checklist of quality criteria was used to look at specific image quality indicators.²⁶ Images not allowing the minimum assessment of the central diagnosis of pneumonia were classified as unreadable. Images that allowed assessment of whether pneumonia was present or not, but not confident assessment of secondary endpoints (e.g., pneumothorax, atelectasis, etc.), were judged suboptimal. Images that permitted confident assessment of the absence or presence of pneumonia as well as other pathological findings were marked as adequate. Unreadable images were excluded from the analysis. Pneumonia was categorized into those with any alveolar consolidation, interstitial infiltrates only, and other evidence of pneumonia. To generate a receiver operator characteristics (ROC) curve, the radiologists were asked how clear the finding for pneumonia was on a 6-point scale: 1 = definitely pneumonia, 2 = probably pneumonia, 3 = possibly pneumonia, 4 = possibly not pneumonia, 5 = probably not pneumonia, 6 = definitely not pneumonia.

Data analysis

Data were entered using EpiInfo 2002 (CDC, Atlanta, Georgia, USA) and analyzed using SPSS 11.0 (SPSS Inc., Chicago, Illinois, USA). Inter-observer variability between the two primary radiology readers was measured using Kappa and the results interpreted as previously reported²⁷; we did not calculate Kappa on outcomes that occurred on five or fewer images. To calculate sensitivity, specificity, and positive and negative predictive value, we used the panel's reading on the hard copy images as the gold standard. A ROC analysis was conducted to evaluate how well the camera and film digitizer readings performed at diagnosing pneumonia compared to the hard copy images; the area under the curve was compared using a univariate z-score test (Rocket 0.9B, University of Chicago, USA).

Results

One hundred and ninety-eight patient radiographs were selected and each had digital camera and film digitizer images taken, for a total of 594 images. The panel's perception of image quality differed according to modality. Of the original 198 radiographs, 163 (82%) were judged to be of adequate quality, 31 (16%) were suboptimal, and 4 (2%) were unreadable compared to 150 (76%), 44 (22%), and 4 (2%) of film digitizer images and 123 (62%), 70 (35%), and 5 (3%) of digital camera images, respectively. The differences in perceived image quality between the original radiographs and the film digitizer was not statistically significant; however, the differences between the original radiographs and the camera, and the film digitizer and the camera, were statistically significant (p  < 0.05). The 13 unreadable images from 6 persons were removed from further analysis, leaving 192 patient radiographs for comparison across all three modalities. Among the 192, 103 (54%) were from men and 62 (32%) were from children of less than 5 years old.

Of 192 hard copy chest radiographs, 185 (96%) were classified as showing evidence of possible, probable, or definite pneumonia, 166 (86%) of probable or definite pneumonia, and 120 (63%) of definite pneumonia. For the remainder of the results, pneumonia is defined as the 166 cases with either probable or definite pneumonia.

There was moderate agreement between the two primary radiology readers on evidence of pneumonia (0.44), and almost perfect agreement on alveolar consolidation (0.89), infiltrates (0.90), and pleural effusion (0.85). The agreement between the readers on other findings is shown in Table 1 .

Table 1.

Inter-rater reliability between the two primary radiologists on 192 hard copy images.

Characteristica	Kappa (standard error)	Interpretation27
Pneumonia	0.44 (0.13)	Moderate agreement
Alveolar consolidation	0.89 (0.03)	Almost perfect agreement
Interstitial pattern only	0.90 (0.03)	Almost perfect agreement
Pleural effusion	0.85 (0.05)	Almost perfect agreement
Hyperaeration	0.55 (0.09)	Moderate agreement
Atelectasis	0.37 (0.12)	Fair agreement
Cavity/abscess	0.70 (0.11)	Substantial agreement
Emphysema	0.44 (0.13)	Moderate agreement

^aCategories are not mutually exclusive.

The radiographic findings of the 166 pneumonias are shown in Table 2 . Seventy-eight (47%) of the pneumonias had alveolar consolidation. A comparison of the test characteristics of the digital camera and the film digitizer to the hard copy gold standard on various radiographic findings for pneumonia is shown in Table 3 . For pneumonia, the sensitivity of the digital camera was 89% and of the film digitizer 90%; specificities were 73% and 65%, respectively.

Table 2.

Findings on 192 chest radiographs from patients with suspected pneumonia as determined by radiology panel.

Characteristica	Number (%)
Pneumonia	166 (86)
Alveolar consolidation	78
Interstitial pattern only	80
Other evidence of pneumonia	8
Pleural effusion	27 (16)
Hyperaeration	25 (15)
Atelectasis	16 (10)
Cavity/abscess	13 (8)
Emphysema	10 (6)
Pneumothorax	0
Lung nodule/mass	4 (2)
Hilar enlargement/lymphadenopathy	3 (2)
Bullae/bleb/cyst	1 (1)

^aCategories are not mutually exclusive.

Table 3.

Sensitivity, specificity, positive and negative predictive value of digital camera and film digitizer images compared to the hard copy gold standard.

Characteristic	Number on hard copy	Percent (95% confidence interval)
		Sensitivity	Specificity	Positive predictive value	Negative predictive value
Digital camera
Pneumonia	166	89% (83–93)	73% (52–88)	96% (91–98)	51% (35–68)
Alveolar consolidation	78	69% (58–79)	94% (87–97)	89% (77–95)	82% (74–88)
Interstitial pattern only	98a	69% (59–78)	83% (74–90)	81% (71–88)	72% (63–80)
Film Digitizer
Pneumonia	166	90% (85–94)	65% (44–82)	94% (89–97)	52% (34–69)
Alveolar consolidation	78	62% (50–72)	96% (90–98)	91% (79–97)	78% (71–85)
Interstitial pattern only	98a	68% (58–77)	78% (68–85)	76% (66–84)	70% (60–79)

^aEighteen persons had interstitial pattern without pneumonia.

An ROC curve was generated comparing the radiologists reading of pneumonia on the original films (i.e., gold standard), digital camera, and film digitizer. The value for the area under the curve was 0.86 (95% CI = 0.78–0.95) for the digital camera and 0.91 (95% CI = 0.87–0.95) for the film digitizer (Figure 1 ). There was no statistically significant difference between the two curves (p  = 0.29). The cost of the film digitizer was $3000; the camera and tripod cost $965.

Figure 1.

Receiver operator characteristic curve analysis of digital camera (solid line) and film digitizer (dotted line) compared to the hard copy ratings for evidence of definite pneumonia. The area under the curve for the two lines was not significantly different (0.86 versus 0.91, p = 0.29).

Discussion

Similar to findings in previous studies, digital images were judged slightly poorer in image quality than the original film copies, with the camera image being rated worse than the film digitizer.17, 18, 20, 22, 25, 28 As expected, the hard copy images were demonstrably better than digital images to detect pneumonia; however, in the setting of a surveillance system in rural Thailand hard copy readings by radiologists are not practical. Our study suggests that in the ability to detect pneumonia, the digital camera and film digitizer performed well compared to the hard copy, and were similar to each other. The ROC curve demonstrated a marginally superior ability to detect pneumonia with the film digitizer versus the digital camera, although the difference was not statistically significant. This finding likely reflects major improvements in digital camera technology. Furthermore, the cost of the digital camera set was one third that of the film digitizer. Overall, our data suggest that for the purposes of pneumonia surveillance in Thailand, use of a digital camera is a cost-effective and practical alternative to a film digitizer.

Chest radiographs can play an important role in surveillance by providing population-based information that can be used to refine estimates of the burden of specific pathogens, such as Hib, S. pneumoniae, or the influenza virus.11, 12, 13, 14 This is particularly important in developing countries that lack the laboratory capability to accurately diagnose pathogens. In rural Thailand, blood cultures are performed on approximately 5% of patients with suspected pneumonia and the capability for viral culture does not exist. Disease burden estimates of vaccine-preventable diseases in developing countries are urgently needed to guide important health policy decisions.

Outbreaks of pneumonia, such as SARS and avian influenza, emphasize the importance of a functioning teleradiology system. For example, during the SARS outbreak, many countries convened a national panel of experts who reviewed clinical, radiologic, and epidemiologic data from suspected patients to determine case status.²⁹ Chest radiographs from patients were digitized using a camera and transmitted via email from local hospitals. Unfortunately, nonstandard approaches to capturing the images and subsequent viewing on low resolution, non-medical monitors likely resulted in some misclassification of patients. The standard approach for the digital camera as outlined in our study could easily be applied in an outbreak setting to improve diagnosis.

Digital cameras are now being used in a variety of medical fields, including surgery and pathology,30, 31, 32, 33 and they are increasingly used in teleradiology for diagnosis of pneumonia on chest radiography.9, 20, 34, 35 Our digital camera protocol was designed to allow reproducibility in different settings, including varying field conditions (e.g., lightbox and film size) and equipment (e.g., camera type).

Although digital imaging has been available since the 1960s, the first non-professional, digital cameras were introduced in the early 1990s.36, 37 Over the last decade there has been rapid advancement in technology with concomitant reductions in price. Resolution, as defined by the pixel, in non-professional cameras has increased from around 0.1 to over 6.0 mega pixels. The camera used in this study was 3.3 mega pixels, a common resolution in digital cameras on the market at the time of the study. The 2004 cost of a similar resolution digital camera and tripod used in this study would be less than $500, half what we paid, a 6-fold lower cost compared to the scanner.

As have others, we found some variability in chest radiology interpretation between radiologists.38, 39, 40 The inter-observer agreement for atelectasis was only fair, suggesting that some outcomes are open to a broader interpretation between radiologists. Although there was only moderate agreement between our readers on pneumonia, our findings are consistent with a recent study that compared chest radiograph interpretations of pneumonia by radiologists from patients with lower respiratory tract infections (Kappa = 0.53, 95% confidence interval = 0.37–0.69).⁴⁰ In another study, agreement between two readers examining chest radiographs from persons with community-acquired pneumonia was low for the finding of infiltrates (Kappa = 0.37, 95% confidence interval = 0.22–0.52).³⁸

In our study, the same radiology panel read the original images and the digital films, introducing the possibility of reader bias through memory recall. Readers might be more likely to record the same result the second time they saw an image and thus bias the results to the null (i.e., no difference). In order to minimize the memory recall effect we randomized the digital images and had them read the original films last since they are perceived as being the highest quality. In addition, there were several weeks between the digital and hard copy readings.

Although the computer hardware and software requirements for both a film digitizer and digital camera are similar, the 3- to 6-fold lower cost of the digital camera and wide availability makes it more appealing for use in less wealthy countries. With advancing technology, cameras are becoming less expensive and able to take and store higher quality images for a fraction of the cost of film digitizers. This remains an area in which future developments could improve the way pneumonia surveillance is conducted.

Conflict of interest: No conflict of interest to declare.

Corresponding Editor: Richard Oberhelman, New Orleans, USA