Dear Dr. Honeyman-Buck,
I was fascinated to read the article by Saleh et al. entitled “An Automated Blood Vessel Segmentation Algorithm Using Histogram Equalization and Automatic Threshold Selection” in the June 2010 issue of the Journal of Digital Imaging. It rouses my interest by presenting such a simple and seemingly very efficient method in segmenting retinal vessels in fundus camera images. However, I have not been able to replicate the results and I strongly suspect that the conclusions are flawed or the methodology is not fully presented in the article. I have tried to communicate my concerns with the authors, but without reply. I find this unfortunate since a direct communication would have made it possible to state my concerns here more concisely.
It is unfortunate that the parameters used in the Contrast Limited Adaptive Histogram Equalization (CLAHE) method (in Matlab this corresponds to the command adapthisteq) are not clearly defined. The tile size stated (8 × 8 pixel) is not consistent with the appearance of image 4b and the clip limit is not given in the article. If one assumes that image 4b is produced with the appropriate parameters one can compare it with the same retinal image (02_test.tif in the DRIVE database) processed by the CLAHE method with different sets of parameters. Then it looks like the parameters used are close to the default values for the Matlab adapthisteq function, with the image divided in 8 × 8 tiles (corresponding to about 71 × 73 pixels per tile) and clip limit of 0.01.
After this, it was straight forward reproducing the background exclusion so, by visual inspection, it looks similar with the article's Fig. 6b. But the next step, thresholding with the isodata method as described in the article, gives much less pixels selected than in the article's Fig. 7b. In addition to this, it is hard to understand how image 7b could have been produced by thresholding of Fig. 6b for any single threshold (see Fig. 1 of this letter for an example).
Fig. 1.
To the left is a copy of image 4b with a marked area. To the right is the corresponding area enlarged in image 4b (top), 6b (middle), and 7b (bottom). It is clear that there is a vessel in image 7b not present in the original image. Although these images were taken from the article and the original images have better resolution it is nevertheless hard to see how image 7b could be generated by thresholding image 6b by a simple value
My last concern with the article is that I do suspect that the large accuracy of 0.9630 in the proposed method, exceeding even trained human observers, might be due to an error. As stated in the DRIVE database homepage (http://www.isi.uu.nl/Research/Databases/DRIVE/results.php) “There are two classes (vessel and background), and only pixels inside the field of view are taken into account (the mask images supplied in the DRIVE database provide the field of view for each image).” The authors of the article discussed here make no mentioning of this in their method. When trying to replicate the method (with above-stated limitations) it was only possible to get accuracies in the 0.95 range (and due to much lower values for a few images the mean accuracy was much lower) by including the background pixels. It is therefore possible that the true accuracy of this method is much lower.
Any clarifications from the authors would be much welcome.
Sincerely
Karl Daggfeldt