Abstract
X-ray coronary angiography has been a gold standard in the clinical diagnosis and interventional treatment of coronary arterial diseases for decades. In angiography, a sequence of images is obtained, a few of which are suitable for physician inspection. This paper proposes an automatic algorithm for the extraction of one or more frames from an angiogram sequence, which is most suitable for diagnosis and analysis by experts or processors. The algorithm consists of two stages: In the first stage, the background and illumination in the angiogram sequence are omitted. By analyzing the histogram of the sequence, a feature is attributed to each frame. These features, determining the visibility of the vessel tree, are clustered by a fuzzy c-means method. In the second stage, the cardiac phase for each frame is specified. Using the results of both stages, the best frames in an angiogram sequence are obtained. To evaluate the proposed method, it has been tested on angiogram sequences from several patients. The results demonstrate the accuracy of the method. The performance and speed of our method indicate its usefulness in clinical applications.
Keywords: Algorithms, coronary angiography, coronary vessels, humans, lighting, X-rays
INTRODUCTION
Coronary artery disease (CAD) occurs when the coronaries are occluded and become constricted, which leads to the heart being deprived of oxygen and other nutrients and eventually arresting. According to the most recent statistics by the World Health Organization, CAD is one of the most prevalent causes of death in the world and affects an increasing number of people.[1] X-ray angiography is considered as the gold standard for the assessment of stenosis in the coronary arteries. In this procedure, a catheter is inserted into one of the two major veins, and a radiopaque contrast agent (i.e., dye) is injected into the bloodstream to make the arteries in the heart visible in X-ray images. Coronary angiography enables the radiologist or cardiologist to see any narrowing of the blood vessels (i.e., stenosis). Angiogram analysis is performed for both diagnostic and interventional purposes. Each sequence of coronary angiograms includes two or three cardiac cycles. However, according to blood flow and cardiac motion patterns, the vessel tree is not properly visible in all of the images in a sequence. Although it is rather easy for a physician to tell when the contrast medium appears in a fluoroscopy scene and makes judgments accordingly, an automatic contrast inflow detection algorithm is desired for many computer-assisted interventions. A few examples include: (1) In a stent enhancement application such as,[2] the algorithm needs to discard frames with contrast medium since they could undermine the visibility of stent enhancement. For this purpose, the algorithm needs to determine whether there is contrast injection during the acquisition. If there is, the algorithm has to estimate at which frame the contrast inflow begins to appear and only precontrast frames should be processed. (2) To improve the overlay of a three-dimensional (3D) model when contrast agent is present,[3] the system has to know the presence of contrast medium so that it can register the vessel/aorta with a presegmented vessel/aorta model. (3) To provide automatic road mapping,[4] the algorithm needs to know when the contrast agent appears in the fluoroscopy to disable the overlay. All these interventional procedures can be streamlined with a robust and automatic contrast inflow detection algorithm such that the physician can obtain desired information with less interactions or constraints.
Robust contrast inflow detection in a large scale is challenging. Little work has been done in the literature. Ideally, sharp changes of the overall intensity histograms along the temporal domain can be used as potential candidates indicating when contrast inflow appears. Nevertheless, this works poorly in practice since moving diaphragm and other darker regions can often appear in cardiac X-ray images and mislead the algorithm. Condurache et al.[4] proposed a method which applies morphological operations and a difference of Gaussians filter to generate a vessel map. A threshold is then applied to the histogram of the vessel map to decide whether contrast medium appears. Based on our experience, the main challenge of automatic contrast inflow detection lies in the high variability of data, especially when the images are often acquired using low-dose radiation where the contrast medium is weakly detectable by a traditional filter. Previous methods[4] were evaluated only on a limited number of data sets and are empirically set parameters. They can hardly be generalized to work in different clinical applications. In a study by Chen et al.[5] a learning-based framework for automatic detection contrast inflow is proposed. In a study by Syeda-Mahmood et al.,[6] two different methods of frame selection are described, one based on Frangi filter and the other based on detecting parallel curves formed from edges in angiography images. Furthermore, in the reconstruction of the 3D structure of coronary arteries, the images at end-diastole (ED) are preferred because they correspond to the most relaxed and stable state during heart motion and consequently reduce superimpositions and potential asynchronism.[7,8] We propose that the frames which show the full-vessel tree and are in the ED state can be considered as the best frames for diagnosis and analysis by experts or computer methods. Hence, it is desirable to develop an automatic mechanism for selecting one or more of these images from the angiogram set.
In this paper, we have proposed an algorithm. It consists of two stages. In the first stage, the visibility of the vessel tree in each frame is obtained by using the histogram information as its feature, and the features are clustered by the fuzzy c-means method. In the second stage, the cardiac phases for each frame are obtained by acquiring the vessels’ displacements in consecutive frames. Then, the frames in which both the vessel tree is full and the cardiac phase is ED are selected as the best frames.
This method has less complexity than the method of contrast inflow detection mentioned before and is proper for clinical application. In addition, to the best of our knowledge, this is the first paper to extract the best frames in angiography videos by combination of visibility of the vessel tree and the cardiac phases.
The remainder of this paper is organized as follows: The proposed method is described in Section 2. Experimental results on real angiography images are presented and assessed in Section 3. Section 4 is devoted to conclusions.
PROPOSED METHOD
For angiogram analysis, the best frames should have two properties: (1) the complete vessel tree should be clearly visible and (2) the frames must be in the ED state. Therefore, the following proposed method consists of two stages.
First Stage
In this stage, the visibility of the vessel tree in each frame is determined by enhancing the frames and using the histogram feature.
For enhancing each radiograph, we use the method proposed in the study by Dehkordi et al.[9] In this technique, a temporal Fourier transformation is applied in the logarithm domain to angiograms to omit the background and compensate the illumination.
Let T denote the number of frames in a cardiac cycle. Each image is defined by function I(x, y, t), where t = 0:T − 1 is the frame number and x, y are spatial coordinates. The image intensity, at each pixel, is the product of illumination (the amount of source illumination incident on the scene being viewed) and reflectance (the amount of illumination reflected by the objects in the scene). We denote illumination by L(x, y, t) and reflectance by R (x, y, t). Since angiograms are grayscale images,
I(x, y, t) = R(x, y, t) . L(x, y, t) (1)
where L(x, y, t) represents the illumination which can be seen as equivalent to the intensity of the incident X-ray energy passed to the film while R(x, y, t) represents the reflectance or equivalently represents the change in body absorption of the penetrating X-ray energy.
Taking the logarithm of Eq. 1
log I(x, y, t) = log R(x, y, t) + log L(x, y, t) (2)
Since illumination variations lie mainly in the low-frequency band, they can be reduced by low-pass filtering. On the other hand, static or slowly moving organs, such as ribs, spines, and lung, comprising the background in an X-ray are eliminated in the same process. Accordingly, using a temporal Fourier transformation in the logarithm domain and omitting low-frequency components, artifacts, and illumination nonuniformity can be removed as needed, as presented in the study by Dehkordi et al.[9] Given a sequence of T frames of images I(x, y, t), 0 ≤ t ≤ T – 1, the discrete Fourier transform in the logarithm domain are obtained as
Replacing Eq. 2 in Eq. 3, we find:
If
Then
FI(x, y, k) = FR(x, y, k) + FL(x, y, k), 0 ≤ k ≤ T – 1 (6)
Applying high-pass temporal filtering, H(x, y, k) = (1 – e−βt), Eq. 6 leads to
F̄I(x, y, k) = FI(x, y, k)(1 − e−βk)
= FR(x, y, k)(1 − e−βk) + FL(x, y, k)(1 − e−βk) (7)
= F̄R(x, y, k) + F̄L(x, y, k),
in which β is a positive constant. In coronary angiograms, β = 9 is appropriate. The inverse Fourier transform of Eq. 7 is obtained as following:
The first and second terms in Eq. 8 remove the background and illumination, respectively. Therefore, a new sequence of images Ĩ(x, y, t)= log Ĩ(x, y, t), 0 ≤ t ≤ T − 1 is obtained in which the artifacts are corrected.
In Figure 1, the histograms of the sequences I(x, y, t) and Ĩ(x, y, t) are compared. Figure 1a corresponds to the 4th, 14th, and 25th frames in I(x, y, t) sequence with no, slight, and full contrast agent, respectively. Figure 1b shows the histograms of the three frames Ĩ(x, y, 4),Ĩ(x, y, 14), Ĩ(x, y, 25) approximately exhibiting Gaussian distributions. Since the histogram is a probability distribution function, this observation is consistent with the central limit theorem, and therefore, according to the central limit theorem and the sum of random variables in logarithm, quasi-Gaussian shape of the histogram is acceptable. For more accuracy, the measures of fit of three histograms for a Gaussian function, obtained by MATLAB curve fitting toolbox, are shown in Table 1. According to Table 1, it can be concluded that distribution of the histogram of each frame in sequence Ĩ(x, y, t), 0 ≤ t ≤ T − 1 has a quasi-Gaussian. It is observed that the mean of the distribution of the histograms are about 150; as a result, for comparing the histograms and obtaining feature, theirs means are set to 150. Since the filled arteries are darker than the image background, smaller values in the histogram indicate the presence of the contrast agent. Therefore, in frames with higher contrast, the stretch the distribution is further to the left. For a better demonstration, the logarithmic scale of the set histograms in Figure 1b is shown in Figure 1c.
Figure 1.
Histograms of three frames, (a) 4th (black), 14th (red) and the 25 (green) of the sequence I (x, y, t). (b) Corresponding Ĩ(x, y, t) frames. (c) The logarithmic scale of the histograms in (b)
Table 1.
Goodness of fit for Gaussian function
We propose the feature to be the intensity on the left extent of each histogram that makes zero the histograms. In Figure 2a, the features are depicted for a sequence of 39 frames of 2D angiograms. As shown in Figure 2a, the maximum features are related to contrast free frames; the median features are related to contrast flowing into or being washed, and the minimum features are related to the whole vessel tree filled by contrast agent.
Figure 2.
The features of a sequence and corresponding frames. (a) The green, red, and blue rectangles are corresponding to categories 1, 2, and 3, respectively. (b) Classification of features in (a)
Therefore, the proposed features for the three frames must be classified into three clusters: (1) contrast free, (2) contrast flowing into or being washed, (3) the whole vessel tree filled by contrast agent. We chose the fuzzy c-means clustering algorithm for this purpose.
Fuzzy c-means clustering
In fuzzy clustering, each sample has a specific possibility of belonging to each cluster. This method, developed by Dunn,[10] and improved by Bezdek,[11] is frequently used in pattern recognition. Let x = (x1, x2, x3,…,xN) be a finite collection of n elements to be partitioned into C clusters according to a given criterion. Fuzzy clustering is based on the minimization of the following objective function:
where m is any real number >1, uij is the membership degree of xi to cluster j, xi is the ith d-dimensional measured data, cj is the d-dimension of center of the cluster, and ||*|| is any norm expressing the similarity between measured data and the center.
Fuzzy partitioning is carried out through an iterative algorithm through updating uij and cj by
The final step is reached with maxij(|uij(k+1)–uij(k)|)<ɛ, where ɛ is a termination factor selected between 0 and 1 and k is the iteration step. This procedure converges to a local minimum or a saddle point of Jm.
The classification result of the data in Figure 3a is shown in Figure 4b, where C = 3 and v1= max(X), v2= min(X), and v3 = (v1 + v2)/2 are used for initialization.
Figure 3.
The control points correspondence in two frames. (a) The control points of frame number 16. (b) The corresponding control points of frame number 16 in frame number 17 (green dots)
Figure 4.
The diagram D(n) for a sequence. (a)The maximums and minimums of D(n) in each cardiac cycle are corresponding to frames in end-diastolic and end-systolic states, respectively. The frame number 15 marked by red arrow is in end-diastolic state and the frame number 20 marked by blue arrow is in end-systolic state. (b) The blue and red rectangles show the frames in cluster one and two, respectively
In Figure 2b, the frames in categories 1, 2, and 3 are specified by green, red, and blue rectangles, respectively.
Cardiac Phase
Recently, developments have been made in electrocardiogram (ECG)-gated ED and end-systole (ES) frame selection to circumvent the severe image ambiguities in X-ray angiography by synchronizing the ECG signal with the X-ray image data. Such ECG-gated methods, however, are not robust because unlike in a healthy patient, when a patient's ECG is pathological, it is difficult to indicate the correct ED and ES frames because the QR- and T-waves of the ECG signal do not have good correspondence with ED and ES. This fact greatly limits the application of such ECG-based approach as most patients undergoing left ventricular angiography have irregular ECG signal. Moreover, since such kind of approaches requires strict synchronization between the ECG signal and the X-ray image data, the record delay, or noise may introduce additional error and lead to wrong detection results. Accordingly, there is a need for an improved system and method for finding ED and ES frames in an angiography series. In this stage, the cardiac phase in each frame is solely determined from the image sequence information, without any auxiliary mean such as an ECG signal. This goal is achieved by acquiring the vessels’ displacements in consecutive frames through the two steps described in the following.
Selection of control points
Strong edges in an image are proper landmark points. To find these points, the gradient image is obtained by applying the first derivative Gaussian filter with σ = 1. The calculated values are normalized in the range (0, 1). Selecting a proper threshold, a zero is attributed to any pixel falling under the threshold. The resulting image is called IG.
The local maxima of IG longer than the threshold are sorted in descending order. From the top of the sorted list, each two points with an Euclidean distance less than a certain amount, Dmin, are chosen as control points.
By decreasing Dmin the computational time of the template matching described in the following method increases. The reason is that the smaller Dmin leads to an increase in the number of increments of selective control points.
Displacement of control points
Template matching is used to determine the corresponding control points in the frames. Based on the previous studies,[12,13] the energy of the histogram of differences (EHD) measure provides an appropriate similarity measure in coronary angiography and is applied in the present study. It is defined as
where H(g) is the normalized histogram of differences between the mask and live images, which can be written as
where g ∈ (−G, G) ⊂ Z and δ(x, y) is the Kronecker delta function.
One of the most influential parameters in a template matching algorithm is the template size. Smaller images impose less computational burden at the cost of poorer resemblance quality. Based on the study by Buzug and Weese[14] with histogram-based similarity measures such as EHD, the size of 50 × 50 pixels is found to be optimum, for both computational burden and confidence level.
In addition, optimization procedures can be used to reduce the amount of computation needed for template matching. In the proposed model, the simple convergence hill-climbing method is used as.[15]
The results of executing one algorithm on two consecutive frames in a sequence are shown in Figure 3. The red dots are the control points of frame number 16, and the green dots are the corresponding control points of frame number 16 in frame number 17. According to the cardiac motion pattern, if the control points in a frame are closer, the heart is in the systolic mode, and if farther apart, in the diastolic mode. Therefore, the sum average of the distances between control point's pairs in each frame is considered as the frame state feature.
Let (P1n, P2n,…, PNn) denotes the positions ([x, y] coordinates) of N control points in frame n. Hence, the state feature is
EXPERIMENTAL RESULTS
We have applied the above technique to several angiogram sequences. These sequences were obtained from Sina Cardiac Hospital, Isfahan, Iran. Their format is DICOM with 512 pixel × 512 pixel frames. Each sequence includes two or three cardiac cycles.
Figure 4 depicts D(n) (n = 10,..,39) for one of the sequences. It includes thirty frames filled with contrast agent (selected according to stage 1 in section 2). Evidently, the local maxima and minima indicate ED and ES frames, respectively.
Figure 4a shows frame number 15 and number 20 marked by arrows and have maximum and minimum of D(n) in first cardiac, respectively. As seen in Figure 4, these frames are corresponded to ED and ES states, respectively.
According to Figure 4a, the frames can be placed into two clusters: (1) The frames in ED and close to it and (2) The frames in ES and close to it. Each sequence consists of several cardiac cycles which include systole and diastole phases. The movements in the two cycles can be different. Thus, the local minimum and maximum values in each cycle can be different. As a result, if the clustering is done in each cycle, the results will be better. Hence, first the number of local maxima, N, is calculated, and then the diagram is clustered by the Fuzzy c-means method. The number of clusters is 2 × N and the N clusters with larger centers, including the frames in the first class. In Figure 4b, the result of clustering Figure 4a is shown so that the blue and red rectangles show the frames in cluster one and two, respectively.
Selection of Best Frames
Finally, the frames in which the entire vessel tree is filled and the cardiac phase is ED are selected as the best frames. Therefore, following the procedure outlined, the best frames are obtained for analysis. In Figure 5, the frames (n = 10,.., 39) are shown according to the contrast feature, and cardiac phase, the horizontal axis is cardiac phase feature D(n) and the vertical axis is contrast feature. According to clustering results in Figures 2 and 4, the frames (13, 15, 16, 23, 24, 25, 26) drown in ellipse in Figure 5 are the best frames.
Figure 5.
The location of best frames: The blue circles are shown the location of frames according to contrast feature (vertical axis) and cardiac phase feature (horizontal axis) for a sequence of images. The best frames are in ellipse
To evaluate the performance of the proposed method quantitatively, we randomly selected 20 X-ray coronary angiogram sequences from 15 patients obtained from Sina cardiac Hospital, Isfahan, Iran. The information of the patients and sequences is in Appendix A. The best frames were selected manually by two experts at this center as ground truth. The determination results of the best frames by the proposed method and the visual method is shown in Table 2. The numbers seen in each row in Table 2 are according to the number of the best frame of a sequence determined by expert and the proposed method, respectively. As can be seen, the proposed method determines the best frames accuracy. In some sequences, such as the row number 6, the difference between of the best frames selected by the proposed method and experts is more. This happens due to C-arm position during angiography. Thus, in some of the frames with heart motion, some parts of the vessel tree are out of sight. As a result, for achieving more precise results during the heart motion, it is better than the doctor takes the images in a way that the whole vessel tree located in the frames.
Table 2.
Determination of the best frames in 10 sequences of angiograms
We used missed detection and false detection. The twenty sequences include 870 frames as 84 frames are best frames. Missed detection denotes that the best frame is misclassified as a nonbest frame and missed-detection rate (MDR) is the number of missed detection over the total number of the best frames. False detection denotes a nonbest frame misclassified as a best frame. False-detection rate (FDR) is the number of false detection over the total number of nonbest frames.
Table 3 shows the MDR and FDR for all twenty angiogram sequences with the proposed method.
Table 3.
Performance of the best selection
DISCUSSION
In this paper, a novel and automatic method was proposed to select the best frames which are most suitable for diagnosis and analysis by experts or digital processing in 2D angiogram sequences. To our knowledge, this is the first method to extract the best frames in angiography videos by combining the visibility of the vessel tree and the cardiac phases. Therefore, this method consists of two stages. In the first stage, the visibility of vessel tree is determined by enhancing the frames and using the histogram feature. The distribution of histogram of frames after enhancing becomes quasi-Gaussian and appropriate for comparison. For comparing the histograms and identification features, the means of the histograms are set to 150 and the intensity on the left extent of each histogram that makes the histogram zeros is calculated as feature. Accordingly, in the view of image processing, the visibility of vessel tree has three states, so the features are classified into three categories. In the second stage, the cardiac phases for each frame are obtained by acquiring the vessels’ displacements in consecutive frames. The control points are selected on strong edges in a frame. Then, the corresponding control points in the next frames are obtained by template matching. According to the cardiac motion pattern, if the control points in a frame are closer, the heart is in the systolic mode, and if farther apart, in the diastolic mode. Therefore, the sum average of the distances between control point's pairs in each frame is considered as the frame state feature. As seen in Figure 3, a few of the control points can be placed in area with no strong edge, but their effect on the feature is negligible because their displacement in two frames is very low. Finally, the frames in which the entire vessel tree is filled and the cardiac phase is ED are selected as the best frames. By By applying the proposed method to twenty angiogram videos including 870 frames, it is seen that this method determines the best frames with high accuracy and is proper for clinical application. Accordingly, for achieving these precise results, it is better that the videos were taken in a way that the whole vessel tree was located in the frames.
CONCLUSIONS
A novel and automatic method was proposed to select the best frames for diagnosis and analysis by experts or digital processing in 2D angiogram sequences. This method consists of two stages. In the first stage, the visibility feature of vessel tree is determined for each frame from its histogram, and then these features are clustered by fuzzy c-means method. Based on this feature, the cardiac phase for each frame is obtained in the second stage. Finally, frames with filled entire vessel tree and ED cardiac phase are selected as the best frames.
This method makes automatic selection of frames with higher contrast and determination of the corresponding cardiac phases possible.
Financial Support and Sponsorship
Nil.
Conflicts of Interest
There are no conflicts of interest.
Acknowledgments
The authors wish to sincerely thank the specialists at Sina Cardiac Hospital, Isfahan, Iran, for providing the images and manually selecting the best frames. Their expertise is much appreciated.
Appendix A: Patients information and images
REFERENCES
- 1.WHO. [Last accessed on 2016 Jun]. Available from: http://www.who.int/mediacentre/factsheets/fs310/en/
- 2.Koolen JJ, Veer MV, Hanekamp CE. Medicamundi: Clinical Applications; 2005. A stent boost image enhancement:First clinical experience; pp. 4–8. [Google Scholar]
- 3.John M, Liao R, Zheng Y, Nöttling A, Boese J, Kirschstein U, et al. System to guide transcatheter aortic valve implantations based on interventional C-arm CT imaging. In: Jiang T, Navab N, Pluim JP, Viergever MA, editors. Medical Image Computing and Computer-Assisted Intervention 2010. LNCS. Springer: Heidelberg; 2010. pp. 375–82. [DOI] [PubMed] [Google Scholar]
- 4.Condurache A, Aach T, Eck K, Bredno J. Bildverarbeitung fur die Medizin. Springer-Verlag Berlin Heidelberg: Berlin; 2004. Fast detection and processing of arbitrary contrast agent injections in coronary angiography and fluoroscopy; pp. 5–9. [Google Scholar]
- 5.Chen T, Funka-Lea G, Comaniciu D. Medical Image Computing and Computer-Assisted Intervention 2011, Part I, LNCS 6891. Berlin, Heidelberg: Springer-Verlag; 2011. Robust and fast contrast inflow detection for 2D X-ray fluoroscopy; pp. 243–50. [DOI] [PubMed] [Google Scholar]
- 6.Syeda-Mahmood T, Beymer D, Wang F, Mahmood A, Lundstrom RJ, Shafee N, et al. Automatic Selection of Keyframes from Angiogram Videos. IEEE International Conference on Pattern Recognition. 2010 [Google Scholar]
- 7.Blondel C, Malandain G, Vaillant R, Ayache N. Reconstruction of coronary arteries from a single rotational X-ray projection sequence. IEEE Trans Med Imaging. 2006;25:653–63. doi: 10.1109/TMI.2006.873224. [DOI] [PubMed] [Google Scholar]
- 8.Çimen S, Gooya A, Grass M, Frangi AF. Reconstruction of coronary arteries from X-ray angiography: A review. Med Image Anal. 2016;32:46–68. doi: 10.1016/j.media.2016.02.007. [DOI] [PubMed] [Google Scholar]
- 9.Dehkordi MT, Sadri S, Doosthoseini AM. Removing artifacts in coronary angiograms. Pisa, Italy: CARS; 2012. [Google Scholar]
- 10.Dunn JC. A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. J Cybern. 1973;3:32–57. [Google Scholar]
- 11.Bezdek JC. Pattern Recognition with Fuzzy Objective Function Algorithms. New York: Plenum Press; 1981. [Google Scholar]
- 12.Meijering EH, Zuiderveld KJ, Viergever MA. Image registration for digital subtraction angiography. Int J Comput Vis. 1999;31:227–46. [Google Scholar]
- 13.Zitová B, Flusser J. Image registration methods: A survey. Image Vis Comput. 2003;21:977–1000. [Google Scholar]
- 14.Buzug TM, Weese J. Image registration for DSA quality enhancement. Comput Med Imaging Graph. 1998;22:103–13. doi: 10.1016/s0895-6111(98)00012-3. [DOI] [PubMed] [Google Scholar]
- 15.Nejati M, Sadri S, Amirfattahi R. Nonrigid image registration in digital subtraction angiography using multilevel B-spline. Biomed Res Int 2013. 2013:236315. doi: 10.1155/2013/236315. [DOI] [PMC free article] [PubMed] [Google Scholar]