Observer Study for Evaluating Potential Utility of a Super-High-Resolution LCD in the Detection of Clustered Microcalcifications on Digital Mammograms

Abstract

We evaluated the potential utility of a newly developed liquid-crystal display (LCD), which used an independent sub-pixel drive (ISD) technique for increasing the spatial resolution of a standard LCD three times in one direction, by use of receiver operating characteristic (ROC) analysis and a two-alternative-forced-choice (2AFC) method to determine improvement in radiologists’ accuracy in the detection of clustered microcalcifications (MCLs) on digital mammograms. We used a standard LCD without and with the ISD technique, which can increase the spatial resolution of the LCD three times in one direction from three mega- to nine megapixels without changes in the size of the display. We used 60 single views of digital mammograms (30 with and 30 without clustered MCLs) for ROC studies and 60 regions of interest (ROIs) with clustered MCLs for 2AFC studies. In the ROC study, seven radiologists attempted to detect clustered MCLs without and with the ISD on the same LCD. In the 2AFC study, the same observer group compared the visibility of MCLs by use of the LCD without and with the ISD. Our institutional review board approved the use of this database and the participation of radiologists in this study. The accuracy in detecting clustered MCLs in the ROC study was improved by use of the LCD with the ISD, but the improvement was not statistically significant (p = 0.08). However, the superiority of the LCD with the ISD was demonstrated as significant (p < 0.001) in the 2AFC study. An LCD with ISD can improve the visibility of clustered MCLs when high-resolution digital mammograms are available.

Introduction

As advances in digital imaging technology have been made in radiology and the cost of digital devices reduced, soft-copy reading of digital radiologic images by use of a liquid-crystal display (LCD) has become common in many hospitals¹. Even breast imaging, which requires an extremely high resolution for diagnosing mammographic lesions, has moved to the use of a high-resolution LCD with five megapixels for the interpretation of digital or digitized mammograms^2–5. The clinical utility of soft-copy reading has been evaluated and demonstrated by use of receiver operating characteristic (ROC) analysis for the detection of masses and/or clustered microcalcifications on mammograms^3,6,7. Recently, a novel method, called an independent sub-pixel drive (ISD) technique⁸, has been developed for increasing the spatial resolution of LCDs without modification of hardware. This novel technique may have the potential to improve the spatial resolution of standard LCDs as well as those of high-end LCDs. The cost for installing this new function in the standard LCDs is estimated to be approximately 20% of the original price. In this study, we evaluated the potential utility of the ISD technique applied to a conventional LCD with three megapixels, by use of ROC analysis and the two-alternative-forced-choice (2AFC) method⁹ to determine if there was improvement in radiologists’ accuracy in the detection of clustered microcalcifications (MCLs) on digital mammograms.

Materials and Methods

Independent Sub-Pixel Drive Technique

In general, a monochrome LCD uses a panel of thin-film-transistor (TFT) matrix, which is the same as that employed for a color LCD. Each individual pixel in a monochrome LCD includes three sub-pixels which correspond to three color components (i.e., red, green, and blue) in a color LCD. In the ISD technique, the three sub-pixels can be driven independently, whereas all sub-pixels are generally driven simultaneously to produce a single pixel in a conventional monochrome LCD⁸.

When the pixel size of the original digital image is smaller than the inherent pixel size of the display, the pixel size of the image actually displayed is increased, if the image size is not enlarged; in other words, the spatial resolution of the displayed image is decreased from the original image. For example, as shown in Figure 1, if the pixel size of a digital mammogram is 70 × 70 μm and the pixel size of the standard LCD with three megapixels is 210 × 210 μm, the pixel size of the actually displayed image on the LCD would be increased to 210 × 210 μm and the pixel value for a single pixel displayed on the LCD would be obtained by averaging or interpolation of the pixel values at nine contiguous pixels in the original image. However, if the ISD technique is applied for driving three sub-pixels independently, the pixel size of the displayed images could be reduced from 210 × 210 μm to 70 × 210 μm, and thus the spatial resolution of the display could be improved three times in one direction. Figure 1 illustrates simulated and actual images of a digital mammogram with clustered MCLs displayed on a conventional three megapixel monochrome LCD without and with the ISD technique, where the resolution in the horizontal direction of the conventional LCD is increased from three to nine megapixels. Figure 2 shows modulation transfer function (MTF) obtained with (a) horizontal and (b) vertical direction of the three megapixel monochrome LCD without and with the ISD. The MTFs were measured by use of a bar-pattern image displayed on the LCD, which was then photographed for subsequent analysis by use of a single-lens reflex digital camera⁸. We have confirmed the linearity of the relationship between output signal level of the CCD sensor attached in the digital camera and the absolute luminance, based on the relationship between the averaged CCD pixel values within the target ROIs for respective test patterns (TG18-LN8-01 to TG18-LN8-18) and the luminaces measured by a telescopic luminance meter (LS-110, Konica Minolta). It is apparent in Figure 2 that the resolution only in the horizontal direction is considerably improved.

Fig 1.

Illustration of the relationship between the original image pixel size and the actually displayed image pixel size on the LCD without and with ISD. Three ROIs with clustered microcalcifications without and with magnification (×12) in three different pixel sizes were simulated by use of averaging of pixel values obtained from the actual digital mammogram. Magnification (×16) of an ROI of an actual digital mammogram with clustered microcalcifications and two ROIs of the same image displayed on the LCD without and with ISD.

Fig 2.

Modulation transfer function (MTF) obtained with a horizontal and b vertical direction of the three mega-pixel monochrome LCD without and with the ISD.

For implementation of the ISD technique, the three sub-pixels in the panel of the TFT matrix of the LCD need to be driven individually. Based on our knowledge, some monochrome LCD monitors cannot drive the sub-pixels individually because of a limitation of the LCD hardware, but some can drive them by incorporating some specific software. In the software for implementing the ISD technique on the potential LCD, the original image pixel value for a square pixel will be converted to the modified pixels for rectangular pixels (in a ratio of 3:1) by use of a linear interpolation technique which provides a means of estimating the function at intermediate points for adjacent pixels.

Observer Study

We employed ROC analysis for evaluating the potential improvement in radiologists’ accuracy in the detection of clustered MCLs and the 2AFC method for comparison of the visibility of clustered MCLs. We used cases acquired from one digital mammographic system for the ROC study. Additional cases were acquired from two different digital mammographic systems for the 2AFC study.

Our institutional review board (IRB) approved the use of this database and the participation of radiologists in this observer performance study.

ROC Study

Digital mammograms used in this ROC study were selected from consecutive cases obtained at our institution between September, 2005 and September, 2006. All images were obtained with a Senographe 2000D (GE, Milwaukee, WI, USA) and had a pixel size of 100 μm and a 1,914 × 2,294 matrix size. For improvement of the quality of the original image, the “nominal” pixel size of all images was converted to a pixel size of 50 μm (thus, a matrix size of 3,828 × 4,588) by use of the interpolation technique which was described above. A total of 60 single view digital mammograms, including 30 without and 30 with clustered MCLs, were selected by a breast radiologist (HA) with several inclusion criteria, namely: (1) there was only one MCL cluster in each single view, (2) the subtlety of clustered MCLs for visual detection was considered very subtle or extremely subtle, and (3) the image quality was adequate in terms of patient positioning and gray scale. The gold standard was determined based on pathologic confirmation by core needle biopsy and/or surgery for the images with MCLs. The gold standard for the images without MCLs was determined by consensus of one radiologist (HA) and the commercial computer-aided detection software output (ImageChecker ver. 8.3, Hologic, Bedford, MA, USA). Although a benign lesion or no lesion in the breast is commonly confirmed on 2-year follow-up, we believe that this was not necessary in this study as patients’ outcome did not affect the study.

Seven breast radiologists and fellows (with a mean of 10.9 years of experience in mammography) participated in two reading sessions for the observer study of the detection of clustered MCLs on the LCD once without and once with the ISD technique. The order of the two reading sessions for each observer was determined randomly, but was balanced, and the interval between the two sessions was at least 2 weeks. We used a PC-Windows-based observer study interface in order to display digital mammograms on the portrait LCD without and with the ISD technique. The high-resolution monochrome 21″ LCD monitor we used has an active area of 424 × 318 mm, 2,048 × 1,536 pixels with 207 × 207 μm pixel pitch without the ISD or 6,144 × 1,536 pixels with 69 × 207 μm pixel pitch with the ISD, a 1,000:1 contrast ratio, a 170-degree view angle, and a maximum intensity of 700 cd/m² (Totoku Electric Co., Ltd., Japan). Although the original LCD has a quasi-10-bit gray scale by use of a built-in look-up table technique for simultaneous display of multiple shades for three sub-pixels, the LCD without and with the ISD technique used in this study employed 8-bit gray scale instead. The LCD was calibrated to the Digital Imaging and Communications in Medicine Grayscale Standard Display Function (DICOM GSDF) and there was no difference in the calibration between values without and with the ISD. Therefore, the digital mammograms with a matrix size of 3,828 × 4,588 could be displayed with an adequately small pixel size in the vertical direction by use of the ISD, but with a larger pixel size in the horizontal direction compared with the original image pixel size. In order to simplify the comparison between two readings without and with the ISD, we did not allow observers to use the functions for magnification or adjustment in the gray levels of images. During the observer study, the radiologists were asked whether clustered MCLs were present or absent, and they then marked their confidence level regarding the likelihood of the presence of clustered MCLs by using a continuous rating scale displayed on the LCD¹⁰.

The instructions given to radiologists about readings and cases in this observer study were: (1) indicate the probability (likelihood) of presence of clustered MCLs by clicking on a bar displayed in the lower part of the LCD. This bar represents “definitely present” at the right end and “definitely absent” at the left end; (2) this interface does not provide functions for windowing and magnification; (3) there will be five training cases and 60 test cases; (4) all positive cases have clustered MCLs (a cluster = at least three microcalcifications); (5) MCLs are not necessarily malignant; (6) the number of cases with clustered MCLs is blinded; (7) ignore obvious benign calcifications (i.e., vascular calcifications, coarse “macro” calcifications, occasional solitary calcifications); and (8) try to use the rating scale consistently and uniformly.

A binormal ROC curve was fitted to each radiologist’s confidence rating data obtained without and with the ISD. A computer program (DBM-MRMC with PROPPROC^11,12) was used for obtaining “proper” binormal ROC curves and for evaluating statistically significant differences between the areas under ROC curves (AUCs) obtained without and with the ISD for all radiologists. In this computer program, the statistical significance of the difference was tested based on the analysis of variance for the matrix arrays of pseudo values of AUCs for all observers ¹¹.

2AFC Study

In the 2AFC study for evaluating the superiority of visibility in the interpretation of MCLs on digital mammograms, we used three sets of image database which were acquired from three digital mammographic systems with different pixel sizes. Each set included 20 ROIs with clustered MCLs at the center of the ROI. The first set of 20 ROIs (hereinafter called the GE set) was selected from the GE cases used for the ROC study. The second set of 20 ROIs (hereinafter called the Siemens set), which were obtained with a MAMMOMAT Novation DR (Siemens, Erlangen, Germany), was provided by a cooperating institution under IRB approval for the use of those images. The third set of 20 ROIs (hereinafter called the KonicaMinolta set), were obtained with a Regius Pure View Phase Contrast Mammography System (KonicaMinolta, Hino, Japan), and selected from cases acquired at our institution between September, 2006 and September, 2007 under a separate IRB protocol. Table 1 shows the image sizes, pixel sizes, and matrix sizes for cases used in the 2AFC study. Note that for the 2AFC study, images displayed on the LCD without and with the ISD were of the same size. In the PC-Windows-based interface used in the 2AFC study, a pair of ROIs without and with the ISD technique were displayed side by side, right and left, in the same window. In this computer interface, all sub-pixels were driven individually, although the average pixel value for a set of three pixels was calculated for providing the corresponding pixel value for display without the ISD. In order to evaluate the subjective visibility of MCLs, radiologists were allowed to use functions for magnification (×2 only) and windowing, each of which operated for the pair of ROIs simultaneously. There were three cases for training and 120 images (2 × 60 cases) for testing in one reading session of the 2AFC study for each observer. In the observer study, each pair of ROIs was displayed without and with the ISD and the observer was asked to select the ROI with better visibility.

Table 1.

Pixel Size and Matrix Size in the Original Image and Matrix Size for Sample Image for Three Types of High-Resolution Digital Mammograms Used in the 2AFC Study

	GE	Siemens	KonicaMinolta
Pixel size	100 μm	70 μm	25 μm
Matrix size (image size)	1,914 × 2,294 (19.2 × 23.0 cm)	2,560 × 3,328 (18.0 × 24.0 cm)	7,080 × 9,480 (17.7 × 23.7 cm)
Matrix size for sample image used in 2AFC	1,500 × 1,500a	1,000 × 1,000	3,000 × 3,000

^aOriginal pixel size (100 μm) was converted to 50 μm by use of linear interpolation

For eliminating the effects of the right/left location and of the reading order in the observer study, the locations of the two ROIs without and with the ISD for each case were changed randomly for the first 60 images. Additionally, the right/left locations without and with the ISD were switched for each case in the second repeated 60 images. Hence, the total of 60 cases from the three different sets was compared twice by changing the right/left location without and with the ISD. The superiority of the LCD with the ISD in the visibility of clustered MCLs was evaluated by the average number of images displayed on the LCD with the ISD which were selected as having better visibility. The 95% confidence interval of the average fractional superiority, which was the ratio of the number of images selected as having “superior visibility” to the total number of comparisons, for seven observers was used for testing of the superiority of the LCD with the ISD statistically.

Results

Figure 3 and Table 2 show the average ROC curves and the AUC values, respectively, for all observers, obtained from the LCD without and with the ISD. Although the average AUCs for the detection of clustered MCLs were slightly increased by use of the LCD with ISD (AUC = 0.937) from that without the ISD (AUC = 0.949) and there was only one observer who slightly decreased his/her performance, the difference between the average AUC values was not statistically significant (p = 0.080).

Fig 3.

Average ROC curves for the seven breast radiologists’ performance in the detection of clustered microcalcifications on digital mammograms (30 with and 30 without clustered microcalcifications) displayed on the LCD without and with the ISD.

Table 2.

Area Under the ROC Curve (AUC) Values Obtained From ROC Analysis for the Detection of Clustered MCLs on the LCD Without and With the ISD for the Seven Radiologists

Observer	AUC
	Without ISD	With ISD	ΔAUC
A	0.944	0.960	0.016
B	0.912	0.932	0.020
C	0.944	0.974	0.030
D	0.945	0.950	0.005
E	0.877	0.879	0.002
F	0.964	0.961	−0.003
G	0.969	0.991	0.022
Average	0.937	0.949	0.013
95% confidence interval of average	0.888, 0.985	0.907, 0.992	−0.002, 0.028a

^aThere was no statistically significant difference (p = 0.08) as tested by DBM-MRMC program (11, 12)

Table 3 shows the numbers of cases and the corresponding fractions [%] selected in the 2AFC study as having better visibility by use of the LCD with the ISD for three sets of digital mammograms. As shown in Table 3, all observers indicated that the fraction of cases considered as having better visibility of clustered MCLs by use of the LCD with the ISD was greater than that without the ISD. The fraction of the cases selected by use of the LCD with the ISD in all three sets of digital mammograms was statistically significant (p < 0.01) for the null hypothesis, “LCD with the ISD has better visibility than that without the ISD (i.e., the fraction >>50%).” In addition, the fraction of cases considered as having better visibility by use of the LCD with the ISD was increased as the pixel size of the digital mammograms was decreased as shown in Figure 4.

Table 3.

Number of cases [%] selected as having better visibility in the 2AFC study, by use of the LCD with ISD

Data set	GE (n = 40) [%]	Siemens (n = 40) [%]	KonicaMinolta (n = 40) [%]	All cases (n = 120) [%]
Original pixel size	100 μm	70 μm	25 μm	–
Observer A	24 [60.0]	28 [70.0]	30 [75.0]	82 [68.3]
B	25 [62.5]	26 [65.0]	37 [92.5]	88 [73.3]
C	31 [77.5]	33 [82.5]	27 [67.5]	91 [75.8]
D	31 [77.5]	37 [92.5]	31 [77.5]	99 [82.5]
E	33 [82.5]	34 [85.0]	40 [100.0]	107 [89.2]
F	27 [68.5]	28 [70.0]	38 [95.0]	93 [77.5]
G	30 [75.0]	29 [72.5]	35 [87.5]	94 [78.3]
Mean	28.7 [71.8]	30.7 [76.8]	34.0 [85.0]	93.4 [77.9]
95% Cl	65.5, 78.1a	69.4, 84.2a	76.2, 93.8a	72.9, 82.8a

^aStatistically significant (p < 0.01) for the alternative hypothesis “LCD with the ISD has better visibility than that without the ISD”

Fig 4.

Relationship between the original image pixel size and fraction [%] of cases selected as having better visibility in the 2AFC study, by use of the LCD with the ISD technique.

Discussion

During the last several years, a number of researchers have tried to demonstrate the usefulness of new display devices in soft-copy reading of digital radiologic images compared to the conventional display devices such as screen-film systems and high-resolution cathode ray tube monitors^2–7. However, as far as we know, there was no paper which provided the statistically significant differences between display devices in terms of the diagnostic accuracy. Therefore, we believe that it would be a challenging task to demonstrate a difference between observers’ performance obtained by use of different display devices. This is probably because it is very difficult to collect a number of extremely subtle cases which could be identified by some of display devices. In general, such difficult cases were rarely observed even in a large hospital. In addition, it is very difficult to include a number of expert radiologists who could detect such subtle cases in the observer study. Under these difficult circumstances, we used digital mammograms with the original pixel size of 100 μm (GE system) in our ROC study, because a number of cases adequate for the ROC study could be obtained only by this system. As a result, the radiologists’ accuracy in the detection of a clustered MCLs was slightly improved by use of the ISD technique, but the improvement was not significant statistically when the original pixel size of the displayed digital mammograms was 100 μm.

In order to further investigate a lack of statistically significant difference in improvement of diagnostic accuracy by use of the LCD with the ISD technique, we performed a 2AFC study in addition to the ROC study, because the 2AFC method is very sensitive in distinguishing between two similar images¹³. In addition, we used two other image sets with smaller pixel sizes in the 2AFC study, because we thought the pixel size of 100 μm (this is the largest pixel size in commercially available clinical digital mammogram units) might not be small enough for comparing the LCDs without and with the ISD. However, the numbers of extremely subtle cases obtained with these two image sets with smaller pixel sizes were not enough to be used as sample cases in the ROC study. The potential usefulness of the LCD with the ISD for increasing the subjective visibility of clustered MCLs was demonstrated by use of the 2AFC study. Although the observers were blinded to the right/left location in the paired ROIs displayed on the LCD with the ISD, the majority of ROIs (77.9%) displayed with the ISD were selected as having better visibility for the interpretation of clustered MCLs. Moreover, our results suggest that the potential utility of the LCD with the ISD is greater as the spatial resolution of the digital mammograms increases. This is probably because the difference in the actually displayed pixel size on the LCD without and with the ISD became greater when the pixel size of the digital mammogram was adequately smaller. For example, the pixel size of images actually displayed on the LCD without and with the ISD for KonicaMinolta cases (a pixel size of 25 × 25 μm) was approximately 150 × 150 μm and 50 × 150 μm, respectively. Thus, the spatial resolution of the LCD was improved three times in one direction by use of the ISD.

Although the spatial resolution of the LCD was improved by use of the ISD, there might be a detrimental effect by use of an asymmetric pixel shape with aspect ratio of one to three. Before and after the observer studies, we did not provide observers any information how the resolution of the LCD was improved by the ISD technique and what was the shape of pixel. However, no observer complained about or noticed the pixel shape during and after observer study.

From the results of two observer studies, one question could be raised as to why an improvement in diagnostic accuracy was not demonstrated in ROC study if the visibility became superior statistically by use of the ISD. In the ROC analysis, the observers’ performances were evaluated in both sensitivity for positive (MCLs) cases and specificity for normal cases whereas the 2AFC study was performed only by use of positive cases without normal cases. Therefore, it should be noted that the superior visibility of normal structures of mammograms on the LCD with the ISD was not demonstrated in this 2AFC study.

In terms of gray scale detectability, the ISD technique had a limitation that can display images only in 8-bit gray scale whereas the original LCD could display quasi-10-bit gray scales. However, from the previous study by Krupinski et al.¹⁴, there was no statistically significant difference between 8-bit and 11-bit gray scales. Although Krupinski's ROC study was performed for the detection of subtle lung nodules on digital chest radiographs, we believe that the reduction in gray scale would not be significant compared to the increase in the spatial resolution.

It should be noted that this ROC study involved several limitations which are well understood to be limitations of such conventional ROC analysis^15,16. The scoring of true-positive responses on each image by radiologists did not take the perceived location of clustered MCLs into account when the ROC curves were estimated, so that some false-positive responses in actual cases with clustered MCLs end up counted as true-positive responses^17,18. Another limitation of this study was the small number of cases used in the ROC study. For example, the standard errors of the two AUCs estimated as 0.937 and 0.949, which were obtained by use of 30 positive and 30 negative cases in the ROC study, are approximately 0.031 and 0.028¹⁹, respectively. Therefore, the small observed difference in AUCs (0.013) would not be considered statistically significant.

Conclusion

The super-high-resolution LCD with the ISD technique, which can be implemented on the standard LCD with a 20% increase in cost, has the potential to increase the sensitivity in the detection of clustered MCLs, and it can improve the visibility of MCLs, especially when high-resolution digital mammograms are displayed. However, further investigation would be needed to study the clinical utility in microcalcification detection of the ISD technique for LCD display of high-resolution digital mammograms.