Automatic classification of symmetry of hemithoraces in canine and feline radiographs

Abstract.

Purpose

Thoracic radiographs are commonly used to evaluate patients with confirmed or suspected thoracic pathology. Proper patient positioning is more challenging in canine and feline radiography than in humans due to less patient cooperation and body shape variation. Improper patient positioning during radiograph acquisition has the potential to lead to a misdiagnosis. Asymmetrical hemithoraces are one of the indications of obliquity for which we propose an automatic classification method.

Approach

We propose a hemithoraces segmentation method based on convolutional neural networks and active contours. We utilized the U-Net model to segment the ribs and spine and then utilized active contours to find left and right hemithoraces. We then extracted features from the left and right hemithoraces to train an ensemble classifier, which include support vector machine, gradient boosting, and multi-layer perceptron. Five-fold cross-validation was used, thorax segmentation was evaluated by intersection over union (IoU), and symmetry classification was evaluated using precision, recall, area under curve, and F1 score.

Results

Classification of symmetry for 900 radiographs reported an F1 score of 82.8%. To test the robustness of the proposed thorax segmentation method to underexposure and overexposure, we synthetically corrupted properly exposed radiographs and evaluated results using IoU. The results showed that the model’s IoU for underexposure and overexposure dropped by 2.1% and 1.2%, respectively.

Conclusions

Our results indicate that the proposed thorax segmentation method is robust to poor exposure radiographs. The proposed thorax segmentation method can be applied to human radiography with minimal changes.

1. Introduction

Thoracic radiography is an important diagnostic in companion animal medicine, routinely used for diagnosing cardiopulmonary disorders as well as the staging of neoplasms in dogs and cats. Radiographs must be of diagnostic quality to provide clinical utility to the veterinarian. The radiograph quality heavily depends on the operator’s experience in acquisition and patient-related factors. Positioning errors are common, especially those where the patient is rotated.¹ In the authors’ experience, radiographs from general practices are inappropriately positioned $\sim 30\%$ of the time. Correcting positioning errors during image acquisition can improve diagnostic quality and is considered to be a key element in reducing errors in radiograph interpretation. While it is common practice to inspect the image quality visually, the day-to-day challenges sometimes make this difficult to perform effectively. Therefore, a method for digital positioning error recognition would be valuable in clinical veterinary practice.

The ventrodorsal (VD) and dorsoventral (DV) radiographs are common projections of the thorax and are acquired in all thoracic radiograph studies. Patient positioning is aimed at collimating the animal so that the anatomy of the target organ is accurately captured in the radiograph. In a properly positioned VD or DV radiograph, the hemithoraces look nearly symmetrical,^1,2 and severe deviation from this symmetry indicates that the patient has rotated during the positioning process. Generally, positioning errors are expected to be noticed before image acquisition or corrected through staff training and experience. However, in practice, this is often not the case. The author’s experience in veterinary practice indicates that technical staff are often inadequately trained in radiographic positioning, resulting in errors. Challenges with positioning also exist in human medical imaging, despite dedicated professionals for image acquisition.³ In veterinary medicine, the challenge is greater as most personnel taking radiographs are veterinary technicians with diverse responsibilities and no specialized training. Henceforth, improving radiographic positioning is necessary, and technology can play a crucial role. Figure 1 illustrates symmetric and asymmetric radiographs. In the asymmetric radiographs Fig. 1(b), the shape and size of the left and right hemithoraces highly differ, whereas in the symmetric radiographs Fig. 1(a), these two regions are almost identical.

Fig. 1.

Examples of symmetric (a) and asymmetric hemithoraces (b) in canine and feline radiographs.

In recent years, deep learning methods have been extensively applied to canine radiographs for classification tasks. For instance, Yoon et al.⁴ used the bag of words combined with convolutional neural networks (CNNs) to detect normal and abnormal canine thoracic radiographs and used cardiomegaly as a metric of abnormality. Tonima et al.⁵ proposed an automatic sorting of canine radiographs by view and region. They heavily considered prediction time and memory limits in their work and proposed a novel lightweight CNN architecture based on ideas from SqueezeNet⁶ and Depthwise CNNs.⁷ Most recently, Banzato et al.⁸ utilized DenseNet-121⁹ and ResNet-50¹⁰ to classify the lateral thoracic radiographs into eight different findings, such as cardiomegaly, alveolar pattern, and bronchial pattern. In their work, they removed radiographs with positioning errors or poor image quality and used the whole radiograph as the input to the network without any prior segmentation. ResNet-50 achieved better classification results, and for four classes, such as cardiomegaly and pleural effusion, the area under curve (AUC) of their model was similar to or higher than that reported in similar studies on humans.

The problem of lung symmetry detection has already been studied in human chest radiography.^11,12 Santosh et al.¹² utilized lung symmetry in human chest radiography to detect pulmonary abnormalities. They extracted different features based on edge, texture and shape, and combined random forest, multi-layer perceptron (MLP) and Bayesian network models to classify lungs into symmetric or asymmetric classes. Their study did not consider chest segmentation, and the chest segmentations of both hemithoraces were provided beforehand. Furthermore, they did not examine the varying exposure conditions that may result from poor image acquisition.

In the present study, we propose a novel method to segment hemithoraces by segmenting the ribs and the spine from canine and feline VD and DV radiographs. We then utilized the segmented ribs and fit an active contour to form the thorax region. The segmented spine was then used as a symmetry line to divide the segmented thorax into left and right hemithoraces contours. To compare left and right hemithoraces for making the final symmetry classification decision, we extracted seven features that captured shape and size from both sides. Finally, we utilized these features for training support vector machines (SVMs), gradient boost classifier (GBC),¹³ and MLP models and made the final classification decision by majority voting.

To the best of our knowledge, this is the first research study proposing a method for segmenting hemithoraces and classifying symmetry in canine and feline radiography. We evaluated our approach on a dataset of 900 canine and feline radiographs from DV and VD views acquired at the Ontario Veterinary College (OVC) and labeled by an expert veterinary radiologist. Additionally, to prove that exposure setting or obstruction of the lungs does not affect the output of our model, we extensively compared the segmentation of the thorax in these scenarios with the ground-truth segmentation to study the drop in performance.

The main contributions of the present work are:

• •proposing a novel thorax segmentation method by segmenting the ribs and the spine and then fitting a snake to the rib region. This thorax segmentation method could be adopted for human chest radiography.
• •Proposing a method to classify symmetry in canine and feline radiography, which could also be used for image acquisition quality control.
• •Performing extensive analysis on the performance of the proposed method under different exposure settings to test its robustness to these extreme conditions and provide a detailed comparison.

2. Method

The pipeline for our symmetry classification method is shown in Fig. 2. Our method automatically segments left and right hemithoraces and then uses them to make classification decisions. This section first explains the hemithoraces segmentation method, followed by feature extraction and classification. The proposed method for the segmentation of the ribs is not sensitive to the appearance of the lung in the radiograph and would function properly as long as the first sternal ribs and the last pair of floating ribs are presented on both left and right hemithoraces.

Fig. 2.

Automated symmetry assessment pipeline. First, a U-Net model masks the ribs and spine, then an active contour is fitted to the ribs to identify the thorax region. After, the thorax mask is divided into left and right regions, which form left and right hemithoraces. Finally, features are extracted from both hemithoraces for the final classification.

2.1. Hemithoraces Segmentation

We started by segmenting the ribs and the spine using a U-Net¹⁴ model. This architecture consists of a series of convolutional blocks in an encoder-decoder fashion. Each encoder layer extracts features and downsizes the image by half until the bottleneck layer is obtained. After this layer, the decoder stream or the upstream starts, where on each step, we upsampled the input features to finally reach an image with the same width and height as the input. The main point in the upstream section is the presence of skip connections¹⁰ to avoid vanishing gradients and improve gradient flow. Here, we concatenated features on the same level from the encoder and decoder before upsampling. Finally, U-Net generates a mask with the same width and height as the input image. We used pre-trained weights from ImageNet¹⁵ to increase training speed and faster convergence.

After segmenting the spine and ribs to detect left and right hemithoraces, we used active contours (deformable snakes)¹⁶ to find the thorax region. Since the segmented ribs can be considered as the approximate shape of the thorax boundary, we used the active contour method as they are designed to solve problems where the approximate shape of the boundary is known and it preserves the topology. Specifically, the top-most rib, bottom-most rib, and the endpoints on the left and right of each rib approximately specify the thorax region for our method. An elastic snake is a contour represented parametrically as $v(s)=(x(s),y(s))$, where $x(s)$ and $y(s)$ are coordinates along the contour and $s\in [\mathrm{0,1}]$, which will be used to minimize an energy function consisting of internal energy $(E_{\text{internal}})$ and external energy $(E_{\text{external}})$. The internal energy generally controls the shape of the snake itself and consists of elasticity and stiffness as described in the following equation:

$$E_{\text{internal}}(v(s))=\alpha (s){\left|\frac{dv}{ds}\right|}^2+\beta (s){\left|\frac{d^{2}v}{d{s}^2}\right|}^2,$$ (1)

where the elasticity term ${|\frac{dv}{ds}|}^2$ controls the length of the snake, and stiffness term ${|\frac{d^{2}v}{d{s}^2}|}^2$ controls its curvature or how much it is allowed to bend to fit existing boundaries. External energy describes how well the curve matches the image data locally. A simple external energy function was described based on a first-order gradient of the image $I$ as

$$E_{\text{external}}(v(s))=-{|\nabla I(x,y)|}^2.$$ (2)

The total energy of the snake is the sum of its external and internal energy

$$E_{\text{snake}}={\int }_0^1{E}_{\text{internal}}(v(s))+E_{\text{external}}(v(s))\mathrm{d}s.$$ (3)

To optimize this function using gradient descent, we needed an initial guess. In this study, we used a rectangle as the initial shape, which is minimal and covers all four corners of the segmented ribs outputted by the U-Net model. After segmenting the thorax, we used the spine as the symmetry line to divide the thorax mask into left and right regions. This step is represented in Fig. 2.

2.2. Shape Feature Extraction

After segmenting left and right hemithoraces separately, we focused on extracting distinctive features from both contours to be used in an ensemble of SVM, GBC, and MLP. The low-level shape features¹⁷ and shape histograms that were extracted include:

• •Area: for a segmented hemithorax region $s\in R^2$, $s=(x_i,y_i{)}_{i=1,\dots ,N}$, it returns a scalar specifying the number of pixels inside the segmented hemithorax.
• •Perimeter: for a segmented hemithorax region $s\in R^2$, $s=(x_i,y_i{)}_{i=1,\dots ,N}$, it returns the number of pixels over the region’s boundary.
• •Centroid horizontal distance ($c_{xo}$): the center of mass of the segmented hemithorax $s\in R^2$ $s=(x_i,y_i{)}_{i=1,\dots ,N}$ or its centroid $c_o=(x_o,y_o)$ can simply be expressed as $x_o=\sum _{i=0}^N\frac{x_i}{N}$, $y_o=\sum _{i=0}^N\frac{y_i}{N}$. To compare the centroid horizontal distance of the left and right hemithoraces, we found the distance on the $x$-axis from each centroid to the closest point on the spine.
• •First-rib-width ($w_x$): this feature compares the distance of the first rib endpoints on the left and right hemithoraces to the spine midline in pixels. As shown in Fig. 3, to calculate this value, we found the top left point on the left hemithorax and the top right point on the right hemithorax, which represents the end point of the first left ribs and first right rib, respectively and calculated their euclidean distance to the spine midline. Red arrows show this distance on both the right and left sides in Fig. 3, and green dotted circles delineate the endpoints.
• •
  Contour density plot: suppose $X=\{x(i,j)|1\le i\le h,1\le j\le w\}$ of size $w\times h$ denote the image containing segmented left or right hemithorax region where $x(i,j)$ denotes the graylevel value at location $(i,j)$ and we can calculate the horizontal and vertical histogram of the image using the following equation:

  $$\begin{gathered} H_j=\sum _i^{w}x(i,j) \\ {W}_i=\sum _j^{w}x(i,j). \end{gathered}$$ (4)

Fig. 3.

The first-rib-width ($w_x$) index is calculated by measuring the distance in pixels, along the $x$-axis, between the spine midline (shown by the yellow line) and the endpoints of first ribs on left and right hemithoraces (shown by green dotted circles). Subscripts $r$ and $l$ represent the feature measured in the right and left hemithoraces, respectively.

After calculating vertical and horizontal histograms of both left and right hemithoraces using the above equation (Fig. 4), we used Jenson–Shannon divergence (JSD) and intersection to compare the vertical and horizontal histograms between the left and right regions. The JSD for two probability distributions of $H_1$ and $H_2$ is defined as

$$\begin{gathered} \mathrm{JSD}(H_1,H_2)=\frac{1}{2}\mathrm{KL}(H_1,M)+\frac{1}{2}\mathrm{KL}(H_2,M) \\ \text{where}M=\frac{1}{2}(H_1+H_2) \\ \mathrm{KL}(H_1,H_2)=\sum _X{H}_1(X)\log \frac{H_1(X)}{H_2(X)}, \end{gathered}$$ (5)

where KL represents kullback–leibler divergence. The intersection between two histograms can be defined as

$$d(H_1,H_2)=\sum _X\min (H_1(X),H_2(X)).$$ (6)

Fig. 4.

Horizontal and vertical histograms of a hemithorax. We compute this histogram for both left and right hemithoraces and then find their similarity using JSD and intersection as defined in Eqs. (5) and (6).

• •
  Intersection over union (IoU): to measure how two contours match each other, we could find their overlap. However, we could not simply flip the left hemithorax and find its IoU with the right one since hemithoraces do not always happen in the center of the radiograph. Henceforth, we registered the left hemithorax to the right hemithorax using translation and then we found IoU using the following equation:

  $$\mathrm{IoU}=\frac{\text{left}\cap \text{right}}{\text{left}\cup \text{right}}.$$ (7)

To quantify symmetry, we used a similarity index $\text{sim}(r_L,r_R)$ for six extracted features from left and right hemithoraces, where $r_L$ and $r_R$ are features in the left and right hemithoraces, respectively. We did not use the similarity index for the IoU feature since it is already in the range of [0,1]. For low-level shape features, such as area, perimeter, centroid horizontal distance, and first-rib -width, we defined the similarity index as below, which is a ratio in the range of [0,1]. The similarity index of 1 and 0 represents pure symmetry and asymmetry, respectively

$$\mathrm{sim}(r_L,r_R)=\frac{\min (r_L,r_R)}{\max (r_L,r_R)}.$$ (8)

2.3. Classification of Symmetry

After encoding the radiograph with seven features and calculating their corresponding similarity index, we used an ensemble of three classifiers, SVM, MLP, and GBC, for canine and feline radiograph symmetry classification.

SVM aims to find a hyperplane in an $N$-dimensional space that distinctly classifies the data points. It transforms data using a kernel function to a space where data can be separated using a line. The choice of the kernel is of utmost importance because it has a direct impact on the performance of the model. Common kernels for training an SVM model include radial basis functions (RBF) and polynomial functions of different degrees. To find the best kernel, we used five-fold cross-validation.

MLP uses a series of connected neurons to pass information in a feed-forward fashion to make classification decisions. To tune the MLP model for the best classification decision, we then utilized errors made on each sample and updated each neuron’s weight and bias using an optimization technique, such as gradient descent. The MLP has several hyperparameters to tune, such as learning rate, regularization term, number of hidden units, network depth, and more. In the present study, we used five-fold cross-validation to find the best set of hyperparameters.

GBC uses the same dataset to generate many decision trees, each to decrease previous errors. At each step, decision trees were built using the errors our model had made thus far. The final decision for a GBC is the sum of the decisions of all the decision trees it generated, each multiplied by a factor called the learning rate. In this model, the learning rate, maximum number of decision trees, and maximum depth in each decision tree are among the many hyperparameters that were tuned by five-fold cross-validation.

We used an ensemble of the three above-mentioned methods to make the final symmetry classification decision. Each model’s hyperparameter was tuned independently and then merged to form the final ensemble model.

3. Experiments

3.1. Dataset

Since there was no public dataset available for this task, a dataset of 900 VD and DV radiographs of dogs (890 radiographs) and cats (10 radiographs) was retrieved from the OVC database. Radiographs were acquired from various computed radiography and direct digital radiography systems at referral practices (primary care clinics) and at OVC. Images were provided to the OVC in either JPEG or DICOM format and stored in the picture archiving and communication system provided by AGFA,¹⁸ with the patient file and any onsite imaging. To gather the dataset of this study, radiographs were exported as JPEG and anonymized either by removal of patient data from the images prior to exportation or through blurring data burned into the image as was present in many images from referral practices. Among 900 radiographs, 270 represented an asymmetric thorax, whereas the rest of them contained a symmetric thorax. Note that we did not exclude overexposed or underexposed radiographs for training and evaluating our model.

3.2. Implementation Details

Our proposed method was implemented in Python, and our deep learning models were implemented using the Pytroch library.¹⁹ The model was trained on an NVIDIA RTX3090 24GB GPU and an AMD Ryzen 9 5950X CPU @ 3.4 GHz ×16. All models were trained with a batch size of 2 and resized to size $1024\times 1024$. We used the AdamW optimizer²⁰ with an initial learning rate of $1{\mathrm{e}}^{-4}$. We used early stopping to avoid overfitting. Our model was evaluated using stratified five-fold cross-validation with a fixed split across all experiments. We initialized our model weights with a pre-trained model on ImageNet.¹⁵ Considering hyperparameters, for training segmentation model loss, we used focal loss²¹ with $\alpha =0.8$ and Tversky loss²² with $\alpha =2.0$ and $\beta =2.0$. For the MLP symmetry classification model, we used 50 neurons in three hidden layers, Adam solver and ReLu, as the activation function. For the GBC model, we used 100 estimators, each with a maximum depth of 3 and a learning rate of 0.1. We used an RBF kernel with $\gamma =0.01$ and $C=0.1$ for the SVM symmetry classification model.

3.3. Experimental Design

This section describes the performance evaluation of the proposed thorax segmentation method under poor exposure settings or when part of the lung is completely obscured or missing. We applied a set of transformations on validation radiographs on each fold and performed three tests: underexposure test, overexposure test, and obstruction test. The exposure settings of properly exposed radiographs (as labeled by an expert veterinary radiologist) were synthetically altered by performing a gamma transformation using the equation below, where $\gamma$ was sampled using a uniform distribution in the range of [0.2, 0.5] and [2.0, 5.0] to simulate underexposed and overexposed radiographs, respectively

$$I(x,y)=I{(x,y)}^{\gamma }.$$ (9)

After applying the gamma transformation, we added gaussian noise with $\mu =0$ and $\sigma \in [\mathrm{0,10}]$. The results following the application of these transformations on two different radiographs are shown in Fig. 5. To obscure the lungs for the obstruction test, we used ground-truth segmentation for the thorax, achieved by fitting an active contour to the ground-truth ribs and fitted an ellipse to it. Then, we randomly divided major and minor axis lengths by values in the range [2,4] and overlapped them with the original image to simulate obscured lungs. Figure 5 shows the results of applying this process to two different radiographs. Note that, for this experiment, we only applied these transformations to the validation set of each fold.

Fig. 5.

(a)–(d) Examples of two normal radiographs with their corresponding synthetic overexposed, underexposed, and obscured transformation.

To evaluate the performance of the proposed method, we applied a direct segmentation method for segmenting hemithoraces and compared its outcome with the proposed method using the IoU metric between the ground-truth left and right hemithoraces and the models’ prediction. In the direct segmentation approach, we directly utilized hemithoraces masks, which we obtained by fitting active contour to ground-truth rib labels, to train a deep learning model that directly outputs both hemithoraces by feeding a radiograph as an input. The U-Net model and the same training scheme with slight modifications were used for the direct segmentation method.

As a baseline model, we also trained a classification model that directly classified each radiograph into symmetric and asymmetric classes. We used EfficientNet²³ with Adam as the optimizer, a learning rate of $5{\mathrm{e}}^{-4}$, batch size of 4, and image size of $1024\times 1024$. We used the same split for folds to train the baseline model.

4. Results

4.1. Thorax Segmentation

A side-by-side comparison of the prediction of the proposed model and the direct segmentation method is provided in Fig. 6. In the first row, the radiograph is underexposed and obscured. Even though part of the ribs is completely obstructed, our model could still recover the entire shape of hemithoraces, whereas the direct segmentation method failed to recover the whole shape. This is because the active contour model aligned the shape to the edges of the image and was less sensitive to the object’s internal structure. The same pattern also happened for images in the second and third rows. The proposed model effectively segmented the hemithoraces when the boundary of the ribs is visible in the radiograph.

Fig. 6.

(a)–(d) Comparison between the hemithoraces segmentation outputs of the proposed model and a conventional direct segmentation method on three sample radiographs obstructed synthetically.

As can be seen in Table 1, our proposed method achieved a slightly higher IoU compared to the direct segmentation method when using normal radiographs. Note that here, the term normal refers to radiographs within the validation set that are not overexposed or underexposed as labelled by a radiology expert. However, the IoU difference between the proposed and direct segmentation methods was higher with 2.04% and 1.16% when we used the synthetically underexposed and overexposed radiographs, respectively. The most significant difference between our proposed and direct segmentation method occurred when we obscured part of the hemithoraces with a difference of 6.1%. Additionally, the IoU of our model only dropped by 3.31% when we obstructed lungs, whereas the IoU of the direct segmentation method dropped by 8.35% after introducing the obstruction (Table 1). Overall, our proposed method worked better than the direct segmentation approach, mainly when the radiograph provided was either overexposed or underexposed.

Table 1.

Comparison between the conventional direct segmentation and proposed segmentation methods using normal, underexposed, overexposed, and obscured radiographs.

Radiograph type	IoU		Difference
	Proposed model	Direct segmentation
Normal	$94.72\pm 1.43$	$93.66\pm 0.98$	1.06
Underexposed	$91.89\pm 0.95$	$89.85\pm 1.17$	2.04
Overexposed	$92.57\pm 1.05$	$91.41\pm 1.52$	1.16
Obscured	$91.41\pm 1.71$	$85.31\pm 2.02$	6.1

4.2. Symmetry Detection

Table 2 provides the SVM, GBC, and MLP model results across five-folds, with MLP resulting in the best performance. Employing an ensemble of three models and majority voting increased precision, recall, F1, and AUC compared to base model values (Table 2). The baseline model results for F1 and AUC were lower by 4.8% and 4.3%, respectively, in comparison to the ensemble model. Additionally, the confusion matrix for the ensemble model is provided in Fig. 7. As can be seen, across all test examples in all folds, our model predicted 209 asymmetric cases correctly out of 270 available cases. Additionally, among all 237 predicted cases as asymmetric, only 28 radiographs were incorrectly classified.

Table 2.

Results of symmetry classification using the SVM, MLP, GBC, ensemble model, and baseline model.

Model	Metric
	Precision (%)	Recall (%)	F1 (%)	AUC
SVM	$81.36\pm 4.63$	$74.98\pm 5.02$	$78.03\pm 4.32$	$80.25\pm 3.98$
MLP	$85.41\pm 4.25$	$76.33\pm 3.15$	$80.61\pm 3.43$	$84.37\pm 3.05$
GBC	$82.19\pm 5.65$	$75.65\pm 4.72$	$78.78\pm 5.15$	$83.12\pm 4.92$
Ensemble model	$88.31\pm 5.25$	$77.68\pm 5.51$	$82.82\pm 5.04$	$85.28\pm 4.73$
Baseline	$79.45\pm 4.58$	$76.65\pm 4.37$	$78.02\pm 4.91$	$80.98\pm 5$

Fig. 7.

Confusion matrix for ensemble model.

Considering symmetry detection prediction times, evaluations on the validation set on each fold showed that the average time to output an active contour on a radiograph of size $1024\times 1024\text{pixels}$ is $3\pm 0.5\mathrm{s}$. Additionally, inference of ribs and spine segmentation on GPU took $0.01\pm 0.005\mathrm{s}$, whereas it took $12\pm 1\mathrm{s}$ on CPU.

The proposed active contour method could be applied in two manners to mask the right and left hemithoraces. The first approach (one-snake) includes applying the active contour to the entire segmented ribs to create the thorax region and then dividing the masked thorax using the spine method. The second method (two-snakes) is to first divide the segmented ribs into the left and right partitions using the spine and then fit an active contour to each side separately to mask the left and right hemithoraces regions. We compared the classification result of these two approaches in Table 3, where the one-snake performed better than the two-snakes in both F1 and AUC metrics, in addition to computational advantage. In fact, in the two-snakes approach, two active contour models should be applied, one on the left and one on the right rib partitions, whereas in the one-snake method, we only need to apply the active contour once.

Table 3.

Masking the left and right hemithoraces using two approaches: (1) the proposed method where the thorax was obtained by applying an active contour to the entire ribs, and the resulted region then was divided into the left and right hemithoraces and (2) two-snakes method where the left and right ribs were treated by two active contours separately.

Model	Metric
	F1	AUC
(1) One-snake	$82.82\pm 5.04$	$85.28\pm 4.73$
(2) Two-snakes	$79.25\pm 4.81$	$82.69\pm 5.04$

Besides the computational efficiency of the proposed method, fitting active contour to all of the ribs can automatically fill any internal artifacts. This is investigated in Fig. 8, where the middle region of the ribs was manually obscured, and the hemithoraces were masked using the one-snake and two-snakes methods. Results of the two-snakes method showed that the segmented hemithoraces are significantly affected by the obstruction of the ribs, and this model could not fully recover the thorax shape, as can be seen in Fig. 8(d). However, our proposed model recovered the thorax shape, and missing rib regions did not affect the output, as shown in Fig. 8(e).

Fig. 8.

(a)–(e) Comparison between the results of our one-snake model and the two-snakes method.

5. Discussion and Conclusion

In this study, we introduced a thorax segmentation method for canine and feline VD and DV radiographs and utilized it to classify the symmetry of hemithoraces. The experimental results suggest that the proposed method for segmenting the thorax performs well under poor exposure settings and even when parts of the lungs are obscured. This ability came from the nature of fitting an active contour to a shape in which our snake is attracted to the image gradient and maintains the topology of the segmentation.

In thoracic radiography, the air-filled lungs are lucent regions (dark) relative to the soft tissue and bone within and surrounding the thoracic cavity. For symmetry classification, as studied by Santosh et al.,¹² one possible solution is to label lungs based on their opacity (dark regions inside the thorax) in both VD and DV radiographs instead of segmenting the whole hemithoraces, as we proposed in this work. Then, use the masks to train an end-to-end segmentation model using these labels as supervision; we call this model a direct segmentation of lungs. However, this method has two major drawbacks.

• •Thoracic diseases can alter the expected opacity of the lungs. For example, patients with pneumonia or heart failure will have an increased opacity of their lungs and patients with pleural effusion will have increased opacity in the pleural space. Increased opacity in these regions may make it hard for the direct segmentation method to segment lungs due to the absence of the dark regions in the thorax that are typically indicative of lungs. Two examples of this scenario are represented in Fig. 9(a).
• •Lungs in underexposed radiographs may appear too bright or have increased noise. Two examples of underexposure are provided in Fig. 9(b).

Fig. 9.

Examples of radiographs that the direct segmentation method would fail to segment the thorax. (a) The left lung lobes are affected by an increased opacity due to bronchopneumonia. (b) Radiographs are underexposed, leading to a more white and grainy appearance of the pulmonary parenchyma.

An ensemble of MLP, SVM, and GBC models allowed us to analyze and look at the features from different perspectives as a result of using different learning processes that improved the outcome. GBC, similar to a random forest, creates an ensemble of trees, each of which is weighed by a weight metric. The MLP uses a series of connected neurons in which each layer transforms input data to a new space before making the final classification decision. The SVM tries to find a decision boundary between the points by first finding the distance between each data point using an RBF and then minimizing SVM loss. One limitation of our methods was that radiographs with an obscured spine were not tested in our model. However, since the spine’s shape is relatively easy to estimate, even if a part of it is not visible, we can utilize the segmented part to recover an estimate of its full shape. Another drawback of the proposed method is that fitting an active contour might be time-consuming and proportional to the image’s resolution. In fact, in active contour fitting, we have to consider all pixels and iteratively optimize the objective function. Thus, a higher resolution radiograph with a higher number of pixels requires more steps to optimize the active contour.

Considering the baseline model, due to the extensive information contained in canine radiographs, it can be challenging for a purely deep learning model to effectively focus on relevant features, especially given the limited amount of available data. Upon investigating misclassified radiographs, we observed that the baseline model struggled with classifying radiographs where part of the thorax is cut outside of the image (it might happen regularly due to poor collimation) or when the radiographs cover the whole body and not the thorax only. From the author’s perspective, our proposed method achieved higher results since we extracted task-specific features from the contours of the hemithoraces.

In this study, the asymmetry of the thorax implied improper positioning of the patients, but it could be due to a disease condition, such as scoliosis. Asymmetry analyses have been widely used to diagnose, measure, and monitor human musculoskeletal conditions, specifically scoliosis, which is evaluated from torso radiographs^24–29 and pulmonary abnormalities such as tuberculosis.¹² To improve the model’s generalization, disease-related asymmetric thorax radiographs could be included in the training dataset. Moreover, the method’s potential could be studied in diagnosing and monitoring diseases affecting canine musculoskeletal morphology or causing pulmonary abnormalities.

The utilized dataset of this study included both canine and feline thoracic radiographs. Since canine and feline thoracic radiographs are very similar in appearance and structure, authors believe training and testing the model on only canine radiographs would not significantly increase the model’s performance. The current dataset lacks sufficient feline radiographs (only 10 examples), and a larger dataset exclusively consisting of feline radiographs is necessary to fully understand the impact of mixing canine and feline radiographs on model performance. Moreover, since the variation of musculoskeletal anatomy between the male and female radiographs are minimal in canine and feline radiographs, the author believes that gender inequality in the training dataset might not impact the final model performance. However, further investigation using a dataset containing an adequate number of radiographs of each gender may answer the concern related to the gender inequality of the dataset.

In future works, we will utilize the proposed method for symmetry classification to develop an automatic quality classification for canine and feline thoracic radiographs, where asymmetric left and right hemithoraces are an indication of poor positioning. Therefore, the developed model can be incorporated into a larger model, which analyzes and classifies the quality of a given thoracic radiograph.

Biographies

Peyman Tahghighi received his BSc degree in computer science from Kharazmi University, Iran, in 2016 and his MSc degree in computer engineering at the University of Tehran, Iran, in 2021. He is a second-year PhD student in biomedical engineering at the University of Guelph with a research focus on applications of artificial intelligence in veterinary radiology. His research interests include machine learning, deep neural networks, and image processing.

Nicole Norena graduated from the University of Guelph in 2021 with a bachelor of science in animal biology and a minor in neuroscience. She is now a second-year MSc student in the Department of Clinical Studies at the Ontario Veterinary Studies. Her research interests include translational research and veterinary radiology. Her research project is focused on investigating and proposing a novel active learning model for the segmentation of canine thoracic radiographs.

Eran Ukwatta received his PhD from Western University in 2013. He went on to hold a position as a postdoctoral fellow at Johns Hopkins University in Baltimore, Maryland, United States, as well as a multi-center postdoctoral fellow at Johns Hopkins University and the University of Toronto. Between 2016 and 2018, he was an assistant professor at Carlton University, still maintaining a position as an adjunct research professor. He joined the School of Engineering at the University of Guelph in 2018, where he is an associate professor.

Ryan B. Appleby graduated from Ontario Veterinary College (OVC) and completed a rotating internship at the OVC HSC. Following a rotating internship, he completed a 1-year diagnostic imaging internship at Veterinary Imaging Center of San Diego. He then completed a residency in diagnostic imaging at NC State University. Currently, he is an assistant professor of diagnostic imaging in the Department of Clinical Studies at OVC. His residency research focused on contrast-enhanced ultrasound and shear wave elastography of chronic kidney disease in cats.

Amin Komeili is an associate professor in the Department of Biomedical Engineering with a joint appointment in the Department of Mechanical and Manufacturing Engineering and an adjunct appointment in the Faculty of Kinesiology. His expertise includes soft tissue biomechanics, biomedical imaging for disease diagnosis, and biomedical device design. He aims to conduct translational research, advancing basic findings into therapeutics and facilitating knowledge transfer between clinicians and the research environment. He collaborates with industry partners specializing in AI and wearable biodevices for human and animal applications.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Code, Data, and Materials Availability

Data may be made available upon request. Code is available at https://github.com/PeymanTahghighi/Symmetry-Classification.