Fracture detection of distal radius using deep-learning-based dual-channel feature fusion algorithm

Abstract

Purpose

Distal radius fracture is a common trauma fracture and timely preoperative diagnosis is crucial for patient's recovery. With the rise of deep-learning applications in the medical field, utilizing deep-learning for diagnosing distal radius fractures has become a significant topic. However, previous research has suffered from low detection accuracy and poor identification of occult fractures. This study aims to design an improved deep-learning model to assist surgeons in diagnosing distal radius fractures more quickly and accurately.

Methods

This study, inspired by the comprehensive analysis of anteroposterior and lateral X-ray images by surgeons in diagnosing distal radius fractures, designs a dual-channel feature fusion network for detecting distal radius fractures. Based on the Faster region-based convolutional neural network framework, an additional Residual Network 50, which is integrated with the Deformable and Separable Attention mechanism, was introduced to extract semantic information from lateral X-ray images of the distal radius. The features extracted from the 2 channels were then combined via feature fusion, thus enriching the network's feature information. The focal loss function was also employed to address the sample imbalance problem during the training process. The selection of cases in this study was based on distal radius X-ray images retrieved from the hospital's imaging database, which met the following criteria: clear anteroposterior and lateral X-ray images with a diagnosis of distal radius fractures confirmed by experienced radiologists. The exclusion criteria encompassed poor image quality, the presence of severe multiple or complex fractures, as well as non-adult or special populations (e.g., pregnant women). All cases meeting the inclusion criteria were labeled as distal radius fracture cases for model training and evaluation. To assess the model's performance, this study employed several metrics, including accuracy, precision, recall, area under the precision-recall curve, and intersection over union.

Results

The proposed dual-channel feature fusion network achieved an average precision (AP)50 of 98.5%, an AP75 of 78.4%, an accuracy of 96.5%, and a recall of 94.7%. When compared to traditional models, such as Faster region-based convolutional neural network, which achieved an AP50 of 94.1%, an AP75 of 70.6%, a precision of 91.1%, and a recall of 92.3%, our method shows notable improvements in all key metrics. Similarly, when compared to other classic object detection networks like You Only Look Once version 4 (AP50=95.2%, AP75=72.2%, precision=91.2%, recall=92.4%) and You Only Look Once version 5s (AP50=95.1%, AP75=73.8%, precision=93.7%, recall=92.8%), the dual-channel feature fusion network outperforms them in precision, recall, and AP scores. These results highlight the superior accuracy and reliability of the proposed method, particularly in identifying both apparent and occult distal radius fractures, demonstrating its effectiveness in clinical applications where precise detection of subtle fractures is critical.

Conclusion

This study found that combining anteroposterior and lateral X-ray images of the distal radius as input for deep-learning algorithms can more accurately and efficiently identify distal radius fractures, providing a reference for research on distal radius fractures.

1. Introduction

Distal radius fracture (DRF) is a prevalent orthopedic fracture type, with millions of people worldwide suffering from this type of hand injury each year.¹ Radiologists typically rely on visual examination to detect fractures in X-ray samples. However, in previous diagnoses, clinicians often face challenges such as strong subjectivity and heavy reliance on experience. In addition, the poor quality of images produced by radiological equipment often complicates diagnoses. This ultimately leads to misdiagnoses and missed diagnoses, resulting in patients not receiving timely treatment.²

Analyzing medical images is often challenging and time-consuming. The exceptional learning capabilities of deep-learning are promising for addressing these issues. As a branch of machine learning, deep-learning has achieved significant success in medical image processing owing to its powerful self-feature extraction capabilities. For instance, Esteva et al.³ classified skin diseases using a single deep convolutional neural network (CNN). Relying solely on pixels and disease labels as input, they utilized a dataset of 129,450 clinical images. Ultimately, the model achieved expert-level performance. Similarly, deep-learning also achieved a series of successful applications in the field of fracture detection. Olczak et al.⁴ selected 5 publicly available deep neural network models that had been pre-trained on non-medical datasets. They applied these models to wrists, hands, and ankle joints using transfer learning methods to automatically detect fractures. Kim et al.⁵ utilized the InceptionV3 network model, which was pre-trained on a non-medical dataset. They retrained the top layer of this network with manually labeled wrist X-ray images to achieve a binary classification of whether wrist X-rays were fractured. The final diagnostic accuracy of the model significantly exceeded that of conventional machine learning methods. For hip fractures, Cheng et al.⁶ employed Densely Connected Convolutional Networks-121 as the backbone network and pre-trained it on a limb dataset to achieve limb part recognition. Kitamura et al.⁷ developed an ankle fracture detection system by forming an ensemble of CNN, which utilized small samples, were trained from Scratch, and combined multiple views. Furthermore, several studies have been conducted on images of fractures in various body parts, including dental roots,⁸ ribs,⁹ proximal humerus,¹⁰ scaphoid,¹¹ etc.

However, most of the aforementioned studies focused on classification tasks and cannot mark the specific location of fractures with bounding boxes, making them unsuitable for DRF detection tasks that require both classification and localization.¹² Object detection is more challenging than merely diagnosing the presence of fractures in images; in addition to determining whether a fracture exists, it also involves pinpointing its exact location in the image. To achieve object detection for DRFs, Yahalomi et al.¹³ utilized visual geometry group-16 as the backbone network for Faster region-based convolutional neural network (Faster RCNN) to detect DRFs, ultimately achieving a classification accuracy of 96% on the test set. Gan et al.¹⁴ employed Inception-v4 as the backbone network for Faster RCNN in detecting DRFs. Zech et al.¹⁵ demonstrated the capability of the Faster RCNN in accurately identifying pediatric wrist fractures with high precision, utilizing a dataset that comprised only a few hundred patients. Min et al.¹⁶ developed a 2-stage strategy for the detection of DRFs. In their approach, You Only Look Once version 5 (YOLO v5) was utilized to pinpoint the location of DRFs, subsequently complemented by the application of EfficientNet-B3 for classifying the type of fracture at the identified site. Meanwhile, Anttila et al.¹⁷ employed the segmentation algorithm U-net in detecting DRFs.

Although the aforementioned studies achieved relatively accurate detection results on DRF images, most of them only employed anteroposterior (AT) X-ray images of the patients as network inputs, neglecting the information from the lateral (LT) X-ray images of the distal radius. This omission prevented the network from acquiring the inherent connection between the AT and LT X-ray images in representing fracture semantic information. Consequently, the network's feature information was relatively limited, leading to poor detection of fractures, especially occult fractures. In addition, ignoring the LT X-ray images of the distal radius also resulted in a significant waste of information in the DRFs dataset. However, the simplistic incorporation of LT X-ray images into the training dataset can potentially lead to a decline in network performance.¹⁸

In response to the aforementioned issues, the main contributions of this study are presented as follows: (1) A dual-channel feature fusion network model is proposed for detecting and locating DRFs. This model enriches the network's feature information by adding a Residual Network (ResNet) 50 model with a Deformable and Separable (DAS) attention mechanism to the existing Faster RCNN framework as a semantic information extraction channel for LT images. (2) An effective feature fusion module is designed to help the network integrate different detailed features and texture structures of the AT and LT X-ray images of DRFs, achieving a complete feature representation of the patient's fracture information. (3) Extensive experiments have demonstrated that the method proposed in this study effectively addresses the current issue of low detection accuracy, achieving an accuracy of 98.1%.

2. Methods

2.1. Network architecture

In most previous DRF detection tasks, only AT X-ray images were employed as input, leading to a reduction in the amount of data available for training the model and a lack of rich feature information in the X-ray images themselves. This resulted in the network learning fewer features consequently, the confidence in the target class information within the region proposal boxes at the classification and regression layer was relatively low. In object detection networks, the confidence level of target class information in the region proposal boxes plays a crucial role in the division of positive and negative samples within the network. If the confidence level is low, the network struggles to obtain a sufficient number of positive samples for training, and the loss gradient of the network is dominated by the more numerous negative samples, thereby degrading the model's performance.

To address this, this study adopts a feature fusion network, as illustrated in Fig. 1, which merges the feature information of LT images with that of AT X-ray images, thereby enriching the network's feature information. Fig. 1A presents the detection channel for AT X-ray images. This channel takes the AT image of the distal radius as input and outputs the classification and location information of fractures in the AT X-ray images. Fig. 1B illustrates the feature extraction channel for LT X-ray images. To match the feature dimensions of both channels, the last layer of ResNet50 is discarded, retaining only up to the average pooling layer. This approach ensures that the model extracts features with high-level semantic information and fine granularity while reducing the number of parameters. The feature information from the LT X-ray images, extracted via the semantic information channel, is then fused with the feature information of the AT X-ray images in the detection channel's classification branch, before being sent to the classification layer for detection.

Fig. 1.

Dual-channel input network models. (A) Detection channel of anteroposterior image; (B) Feature extraction channel of lateral image.

Conv: Convolution, RPN: region proposal generation network; ROI: region of interest; GAP: global average pooling; FC: fully connected; Bbox reg: bounding box regression; DAS: deformable and separable.

2.2. AT X-ray image detection channel

This study adopts Faster RCNN as the main network for the AT image detection channel. Compared to other representative object detection networks such as single shot multibox detector (SSD),¹⁹ single-shot refinement neural network for object detection (RefineDet),²⁰ and scale normalization for image pyramids (SNIP),²¹ Faster RCNN offers advantages such as high detection accuracy, wide applicability, and strong robustness. It primarily comprises a feature extraction network (backbone), a region proposal generation network (RPN), a region of interest pooling, and a classification and regression layer.

The specific algorithmic process involves initially inputting an image, from which the main feature extraction network obtains a shared feature layer. The RPN network then generates proposal boxes using this shared feature layer. Subsequently, the boxes are decoded to determine the areas to be cropped from the shared feature layer. The RPN loss function is as follows:

$$\mathrm{L}\left(\left\{{\mathrm{p}}_{\mathrm{i}}\right\},\left\{{\mathrm{t}}_{\mathrm{i}}\right\}\right)=\frac{1}{{\mathrm{N}}_{\text{cls}}}\sum _{\mathrm{i}}{\mathrm{L}}_{\text{cls}}\left({\mathrm{p}}_{\mathrm{i}},{\mathrm{p}}_{\mathrm{i}}^{\ast }\right)+\mathrm{\lambda }\frac{1}{{\mathrm{N}}_{\text{reg}}}\sum _{\mathrm{i}}{\mathrm{p}}_{\mathrm{i}}^{\ast }{\mathrm{L}}_{\text{reg}}\left({\mathrm{t}}_{\mathrm{i}},{\mathrm{t}}_{\mathrm{i}}^{\ast }\right)$$ (1)

where the classification loss function is:

$$L_{cls}=-\left[p_i^{\ast }\log \left(p_i\right)+\left(1-p_i^{\ast }\right)\log \left(1-p_i\right)\right]$$ (2)

where $p_i^{\ast }$ denotes the predicted probability that the i-th anchor is the target; $p_i$ is the ground truth corresponding to the i-th anchor. The regression loss function is:

$${\mathrm{L}}_{\text{reg}}\left({\mathrm{t}}_{\mathrm{i}},{\mathrm{t}}_{\mathrm{i}}^{\ast }\right)=\sum _i{smooth}_{L_i}\left(t_i-t_i^{\ast }\right)$$ (3)

where $t_i$ denotes the bounding box regression parameter that predicts the i-th anchor; $t_i^{\ast }$ is the ground truth corresponding to the i-th anchor. The region of interest pooling layer then adjusts the scale of the cropped feature layer and performs further convolution. Finally, the predictor decodes the feature information to obtain the final prediction results. The entire detection process is illustrated in Fig. 2.

Fig. 2.

Faster RCNN algorithm process.

Faster RCNN: Faster region-based convolutional neural network; RPN: region proposal network; NMS: non-maximum suppression; img: image; ROI: region of interest; BBox: Bounding Box; roi_cls_loc: region of interest classification and localization; roi_score: region of interest score.

2.3. LT X-ray image semantic channel

Clinicians typically require both AT and LT X-ray images to determine the presence of radius fractures in wrist images, indicating that both views are equally important in verifying fractures. Sometimes, fractures that are not visible in the AT view can be distinctly identified in the LT view, as illustrated in Fig. 3. Although the LT view may confuse representing the location of fractures, with the radius and ulna often overlapping, the semantic information it provides is consistent with the AT view, which represents the same classification information (whether the patient has a fracture). Therefore, extracting semantic information from LT X-rays to assist the model in determining the presence of fractures is crucial.

Fig. 3.

AT and LT X-ray images of the wrist of the patient with a fracture. (A) AT X-ray image of the wrist; (B) LT X-ray image of the wrist.

AT: anteroposterior; LT: lateral.

Here, ResNet50²² is utilized as the semantic information extraction channel for LT X-ray of the distal radius. ResNet50 comprises 4 sets of residual blocks, an average pooling layer, and a fully connected layer. Its short-circuit mechanism and identity mapping function in the residual blocks effectively address issues such as gradient vanishing, gradient explosion, and overfitting that emerge with increased network depth.

To improve the expressive capability of the fracture feature information output from the semantic information extraction channels, this study introduces the DAS²³ attention mechanism based on ResNet50, which dynamically adjusts the position of the convolution kernel by combining depth-separable convolution and deformable convolution while maintaining computational efficiency, to precisely focus on the important regions within the image, as shown in Fig. 4. The specific process is that, firstly, depth-separable convolution is performed on the output feature map of each residual block, reducing the number of channels from $C$ to $\alpha \times C$, where $0<\alpha <1$. After that, instance normalization and the Gaussian error linear unit (GELU) activation function are used to obtain the compressed features, as shown in equation (4):

$$X_c=GELU(InstanceNorm\left(X\ast W_1\right)$$ (4)

where $W_1$ is the weight matrix of the depth separable convolution. After that, deformable convolution dynamically adjusts the position of the convolution kernel to capture the salient regions in the image, the equation is as follows:

$$deform\left(p\right)=\sum _{k=1}^K{w}_k\cdot w_p\cdot X\left(p_{ref,k}+\Delta p_k\right)$$ (5)

where $K$ is the size of the kernel with weights identical to those of the regular kernel in the residual block, which is convolved at a fixed reference point $P_{ref}$. $\Delta p$ is a trainable parameter that helps the kernel find the most relevant features. $w\_p$ is a trainable parameter between 0 and 1. The values of $\Delta p$ and $w\_p$ depend on the features upon which the kernel function acts. After that, layer normalization and the Sigmoid activation function are performed on the output feature map to generate the attention weight map. After passing through this deformable convolution, the number of channels of the feature map is changed from $\alpha \times C$ again to $C$:

$$A=\sigma \left(LayerNorm\left(deform\left(X_c\right)\right)\right)$$ (6)

Where $\sigma \left(LayerNorm\left(deform\right(X_c\left)\right)\right)$ denotes the Sigmod function. Each element of the attention weight map normalized by the Sigmod function has a value between 0 and 1. The magnitude of these values determines which parts of the feature mapping are amplified or attenuated. Finally, the original feature map is multiplied point by point with the attention weight map to obtain the enhanced feature map $X_{out}$, calculated as follows:

$$X_{out}=X\otimes A$$ (7)

Fig. 4.

The overall structure of the DAS.

DAS: Deformable and Separable; X: feature map of the lateral image processed by the ResNet block; Xc: feature of the X processed by the depth separable convolution; DSC: depth separable convolution; CD: deformable convolution.

2.4. Feature fusion

Owing to the differences in feature information between the AT and LT X-ray images of the distal radius, simply adding the extracted features from the LT images to those from the AT images can cause fluctuations in network performance. Therefore, this study designs a feature fusion method using an attention mechanism, as illustrated in Fig. 5. This approach allows the network to utilize the differences in high-level semantic features between the AT and LT X-ray images of the distal radius, thereby extracting potential information about the fracture.

Fig. 5.

Feature fusion module.

F: the feature vector post-fusion; A: the feature vector output by the detection channel classification branch; B: the feature vector output by the semantic information channel; M: the attention map; C: chanel; H: height; W: width.

The feature fusion operation in this method is performed on a per-pixel basis. Subtracting the feature information of the LT image from that of the AT image helps provide contrast information and suppress responses to attributes irrelevant to fracture identification and localization. Adding the resulting features back to the AT image features assists the network in recognizing the same structural information of the distal radius in both AT and LT images. In addition, weighting the AT feature map with the attention map obtained after encoding the feature vectors of both channels enables the network to learn the differences in high-level semantic features between the AT and LT images. The presence of a weighting factor allows the network to dynamically balance the additional noise interference caused by integrating the semantic information of the LT image.

$$M\left(A,B\right)=Sigmod\left(A+\lambda \left(A-B\right)\right)$$ (8)

$$F=M\otimes A$$ (9)

Where $F$ denotes the feature vector post-fusion, $A$ signifies the feature vector output by the detection channel classification branch, $B$ represents the feature vector output by the semantic information channel, $\lambda$ indicates the weight coefficient, and $M$ constitutes the attention map.

2.5. Loss function

In the original Faster RCNN network, a binary cross-entropy loss function is utilized as the classification loss function, and expressed as:

$$L=-\text{ylog}{\mathrm{y}}^{\prime }-\left(1-y\right)\log \left(1-y^{\prime }\right)=\{\begin{array}{l}-\log y^{\prime },y=1\\ -\log \left(1-y^{\prime }\right),y=0\end{array}$$ (10)

where $y^{\prime }$ denotes the predicted probability of the class, and $y$ represents the true label, which assumes values of 1 and 0. For binary classification problems, this formula can be reformulated as follows:

$$L=\frac{1}{N}\left({\sum }_{y_i=1}^m-\log \left(y^{\prime }\right)+{\sum }_{y_i=0}^n-\log \left(1-y^{\prime }\right)\right)$$ (11)

where $m$, $n$ and $N$ denote the number of positive samples, the number of negative samples, and the total number of samples, respectively.

When there is an imbalance in the sample distribution, such as a dominance of negative samples, the distribution of the loss function will lean towards the negative samples, and vice versa. Studies²⁴ have shown that this skewness in the loss function causes simple samples to dominate during gradient updates in the model, leading to poorer performance in classes with fewer samples. In DRF images, the complex background contains bone seams that are structurally similar to fracture lines. Moreover, the narrow and long DRF lines occupy a negligible number of pixels in the overall image background, which can easily lead to sample imbalance during training. Therefore, this study adopts focal loss to replace the original cross-entropy loss function. The formula for focal loss is expressed as:

$$L_{fl}=\{\begin{array}{l}-\alpha {\left(1-y^{\prime }\right)}^{\gamma }\log y^{\prime },y=1\\ -\left(1-\alpha \right){y^{\prime }}^{\gamma }\log \left(1-y^{\prime }\right),y=0\end{array}$$ (12)

For simple samples with high prediction probabilities, after being modulated by ${\left(1-y\right)}^{\lambda }$, their contributed loss gradient becomes significantly smaller, causing the network to focus more on learning difficult positive samples. The same applies to negative samples. Hence, focal loss can address the issue of sample imbalance on the level of learning difficulty, while the balancing factor $\alpha$ serves to balance the weights of positive and negative samples.

3. Results

3.1. Dataset

The experimental data for this study consisted of a total of 641 DRF samples, of which 413 were obtained from the clinical data of Shanghai Sixth People's Hospital between March 2015 and September 2020, while 228 samples were derived from clinical images at Lingang Campus of Shanghai Sixth People's Hospital between June 2018 and August 2023. Each sample comprised 2 directional wrist X-ray films of the patient: 1 in the AT view and the other in the LT view. The dataset was augmented to a total of 5128 images via techniques such as random directional flipping, contrast enhancement, and horizontal flipping, thereby ensuring an equal distribution of AT and LT wrist X-ray images. During the random flipping operation, the image is rotated counterclockwise by an angle selected randomly from 0 to 270°, where the upper left corner of the image acts as the center of rotation, as illustrated in Fig. 6. The image contrast is enhanced through the Contrast Limited Adaptive Histogram Equalization,²⁵ as shown in Fig. 7. The horizontal flipping process is demonstrated in Fig. 8.

Fig. 6.

Random directional flipping.

Fig. 7.

Contrast enhancement.

Fig. 8.

Horizontal flipping.

The dataset was partitioned into a training set and a test set at a 5:1 ratio, comprising 4274 and 854 images for training testing tests, respectively. The image annotation was performed with LabelImg software under the supervision of experienced physicians. The annotation data were stored in Pascal VOC2007 format, a standard frequently employed in Faster RCNN networks, as XML files. The main annotation process is shown in Fig. 9. Since the lateral view images were employed exclusively to provide semantic information, they were classified into fracture and non-fracture categories without the use of BoundingBox annotations.

Fig. 9.

Example of labeling process.

3.2. Evaluation metrics

Here, 4 key evaluation metrics were employed for the fracture detection task: accuracy, precision, recall, and mean average precision (mAP). In object detection, the intersection over union (IoU) metric is crucial for the calculation of mAP, as expressed in equation (13).

$$IoU=\frac{A\cap B}{A\cup B}$$ (13)

IoU is the ratio of intersection to union between the predicted bounding box A and the true bounding box B. In the evaluation algorithm used for fracture detection, an IoU value greater than 0.5 is considered a correct prediction of the fracture location, while an IoU value less than 0.5 is regarded as a prediction error.²⁶ Here, a more stringent IoU threshold of 0.75 is also adopted for evaluating mAP. Since this study focuses on only one type of fracture, the average precision (AP) is equivalent to the mean average precision (mAP). The specific formulas for calculating accuracy, precision, and recall are expressed as:

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\times 100\%$$ (14)

$$Precision=\frac{TP}{TP+FP}\times 100\%$$ (15)

$$Recall=\frac{TP}{TP+FN}\times 100\%$$ (16)

where TP, TN, FP, and FN denote the number of true positives, true negatives, false positives, and false negatives, respectively. Accuracy is represented by the accuracy rate, Precision by the precision, and Recall by the recall rate. The AP value is equal to the area under the precision-recall curve.

3.3. Implementation details

To train our network model, DAS-ResNet50 is employed as the semantic information extraction channel for the LT view, and Faster RCNN for the detection channel of the AT view. The input comprises both AT and LT wrist X-ray images of patients.

To guarantee a fair comparison, all experiments were conducted using Pytorch on a single NVIDIA 3080Ti graphics processing unit. The specific details of the experimental environment are shown in Table 1. A uniform batch size of 4 was established for all training sessions, and stochastic gradient descent was selected as the optimizer for the model. The initial learning rate was established at 0.005, and the weight decay coefficient was maintained at 0.0005.

Table 1.

Experimental environment configuration.

Name	Environment configuration
Operating system	Unbuntu20.04
CPU	I7 12700
GPU	NVIDIA RTX 3080Ti
Memory	32 G
GPU graphics memory	12 G
Python version	3.8
Algorithm framework	Pytorch 1.10.0

CPU: central processing unit; GPU: graphics processing unit.

3.4. Experimental results

Our network, trained on 4274 images, achieved an accuracy rate between 97% and 98%, exemplifying its robust learning capability. As illustrated in Fig. 10, every loss index of the network model optimally converged through continuous iterations across the designated 35 training epochs. Ultimately, upon testing on a dataset comprising 854 images, the network attained AP50, AP75, precision, and recall values of 98.5%, 78.4%, 96.5%, and 94.7%, respectively. The final accuracy rate achieved was 98.8% (Fig. 11).

Fig. 10.

Overall loss of the model during training.

Loss: overall loss; Loss_classifier: loss for classifier; Loss_box_reg: loss for bounding box regression; Loss_objectness: loss for objectness prediction; Loss_rpn_box_reg: loss for region proposal network box regression.

Fig. 11.

Precision-recall curves for different models.

Cascade RCNN: Cascade Region-based Convolutional Neural Network; YOLO: You Only Look Once; SSD: single shot multibox detector; SPP: spatial pyramid pooling.

4. Discussion

4.1. Comparative experiments and quantitative analysis

To ascertain the effectiveness of the proposed dual-channel input network for detecting DRFs, this study conducted comparative experiments with 7 classic object detection networks. The network developed in this study employs a dual-channel input, incorporating both AT and LT images of the distal radius. In contrast, the comparative networks utilize a single input, comprising solely the AT images of the distal radius. Furthermore, to evaluate the impact of incorporating LT images on the performance of the 5 comparative networks, a further set of inputs combining both AT and LT images of the distal radius was tested. The results obtained from these experiments are presented in Table 2.^19,27, 28, 29, 30, 31

Table 2.

Test results and comparison with other deep-learning architecture.

Input	Method	AP50 (%)	AP75 (%)	Precision (%)	Recall (%)
AT
	Faster RCNN	94.1	70.6	91.1	92.3
	SPP27	91.4	69.2	90.2	90.6
	Cascade RCNN28	96.8	74.6	94.5	93.7
	YOLOv429	95.2	72.2	91.2	92.4
	YOLOv5s30	95.1	73.8	93.7	92.8
	YOLOv831	95.4	74.1	94.1	93.1
	SSD19	93.6	69.7	90.7	91.8
AT + LT
	Faster RCNN	90.4	66.3	87.5	88.7
	SPP27	86.8	64.1	85.2	86.6
	Cascade RCNN28	92.4	69.5	90.9	91.4
	YOLOv429	90.6	68.4	88.8	89.8
	YOLOv5s30	94.2	71.4	92.2	91.
	YOLOv831	94.4	72.1	92.9	91.8
	SSD19	89.6	65.2	86.8	89.3
AT + LT	Ours	98.5	78.4	96.5	94.7

AT: anteroposterior; LT: lateral; AP50: average precision at IoU = 0.50; AP75: average precision at IoU = 0.75; Cascade RCNN: Cascade region-based convolutional neural network; YOLO: You Only Look Once; SSD: single shot multibox detector; SPP: spatial pyramid pooling.

Table 2 shows that the network designed in this study achieved optimal results across all evaluation metrics. Compared to other networks that use AT images as input, the AP50 value of our network outperformed Cascade RCNN (96.8%) and YOLO v8 (95.4%), achieving 98.5%. For the AP75 metric, the model demonstrated superior performance, achieving 78.4%, exceeding Cascade RCNN's 74.6% and Faster RCNN's 70.6%. These results indicate that the model maintains high detection accuracy under stricter IoU thresholds, highlighting its capability to accurately localize fracture points.

In terms of precision, our model reached 96.5%, representing a notable improvement over SDD (90.7%) and YOLO v5s (93.7%), thereby minimizing false positives in positively labeled cases. Additionally, the model achieved a recall of 94.7%, outperforming YOLO v4 (92.4%) and YOLO v8 (93.1%), and demonstrating the greatest improvement over the spatial pyramid pooling network (94.7% vs. 90.6%). This demonstrates that the model effectively identifies most actual fracture cases, thus reducing the risk of missed detections.

Table 2 clearly illustrates that the performance of the 5 comparative networks declined to varying degrees upon the inclusion of LT images of the distal radius in the inputs. This performance decline can be attributed to the substantial differences between AT and LT images of the distal radius. The representation of fracture features in these 2 image types is not uniform; fractures may be more conspicuous in LT images and less discernible in AT images, or vice versa. Consequently, despite the increased data volume from adding both AT and LT images of the distal radius to the inputs of the other 5 networks, the conflicting feature information from the 2 image types adversely affected their synergy, thereby degrading network performance. The proposed network improves its performance by incorporating additional channels tailored for extracting features from the LT images. Utilizing a feature fusion module that assigns weights, the network derives differential information between AT and LT images via contrastive learning. This approach allows the network to extract fracture information from LT images, thereby enriching the network with additional semantic information and further improving its performance.

To improve the visual representation of the experimental results, 3 test result examples were arbitrarily selected. The prediction boxes produced by various networks are illustrated in Fig. 12. The numbers positioned above the boxes denote confidence levels, which represent the probability of a fracture. The greater the value, the more superior the network's detection performance. The first column presents the original images, whereas the other columns exhibit the images as predicted by various networks. From the illustrations, it is apparent that the method introduced in this study attained a 99% confidence level in the prediction boxes. In the identification of occult DRFs as illustrated in the first and second images of the original pictures in Fig. 12. The highest prediction probability of other networks was 87%, with the lowest at 71%. In contrast, the proposed method consistently maintained a 99% confidence level in predictions. This indicates that the proposed dual-channel feature fusion network exhibits optimum detection accuracy and recall rate. The evaluation of the model training results indicates that, compared to networks using only the AT view of the distal radius as input, the dual-channel input network introduced in this article demonstrates more accurate target recognition and enhanced performance in detecting occult fractures in images.

Fig. 12.

Prediction results of different networks.

RCNN: region-based convolutional neural network; YOLO: You Only Look Once; SPP: spatial pyramid pooling; SSD: single shot multibox detector.

4.2. Ablation experiments and qualitative analysis

To assess the effectiveness of the incorporated DAS-ResNet50 extra channel, feature fusion module, and focal loss relative to the original Faster RCNN, we conducted ablation studies. The base model, Faster RCNN, without additional modules, is a single-channel network; hence, it exclusively processes images of the AT view of the distal radius. The results of these experiments are presented in Table 3.

Table 3.

Results of ablation experiments.

Input	Improvement name	ResNet50	Feature fusion	DAS	Focal loss	AP50 (%)	AP75 (%)	Accuracy (%)	FLOPs (G)	Parameters (M)
AT	0	✕	✕	✕	✕	94.1	70.6	96.2	61.61	82.32
AT + LT	1	✓	✕	✕	✕	93.5	69.2	95.5	104.33	84.54
	2	✓	✓	✕	✕	96.3	74.7	96.8	107.48	84.61
	3	✓	✓	✓	✕	97.8	77.8	98.2	108.23	86.24
	4	✓	✓	✓	✓	98.5	78.4	98.8	108.23	86.24

AT: anteroposterior; LT: lateral; DAS: Deformable and Separable; AP: average precision; FLOPs: floating point operations per second.

Table 3 demonstrates that relative to the original Faster RCNN network, incorporating an LT view semantic information extraction channel via ResNet50 resulted in decreased AP value and accuracy. This reduction occurs because directly integrating the extracted LT view semantic information into the Faster RCNN network leads to performance degradation. This phenomenon is attributed to the varying contribution levels of the different features in the AT and LT views of the radius towards the final result, accompanied by noise from other levels. This outcome corroborates the aforementioned issue that merely adding the feature information from the 2 channels may lead to a decrease in performance. The inclusion of the feature fusion module successfully mitigates the noise disturbance emerging from the integration of LT view feature information. It allows the network to establish a more efficient semantic information flow via contrastive learning, thus obtaining richer feature information. The enhancements in AP value and accuracy verify the efficacy of the feature fusion module. Relative to using only ResNet50 as the LT view semantic information extraction channel, the integration of the DAS module resulted in an increase in the AP50 value and accuracy by 1.5% and 1.4%, respectively. The reason is that the DAS module can dynamically adjust the convolution kernel positions, enabling it to effectively adapt to local deformations as well as the complex structures present in the image.

Although the proposed network, incorporating the DAS-ResNet50 channel, a feature fusion module, and focal loss, increases the floating-point operations relative to the original Faster RCNN, it only adds 2.29 M parameters. As a result, the AP50, AP75, and accuracy reach 98.5%, 78.3%, and 98.8%, respectively, which represents an acceptable trade-off between performance and model complexity. This level of performance exceeds that of the original Faster RCNN algorithm, thereby conclusively validating the feasibility of the improvements. Fig. 13 illustrates the precision-recall curve for the enhanced modules.

Fig. 13.

Precision-recall curves for different improvement modules.

Fig. 14 presents the feature visualization results of the network following the addition of each module, where the red boxes in the original images highlight the areas of interest at the distal end of the radius. As observed in Fig. 14, owing to the presence of normal gaps between wrist bones, the area of focus for the basic Faster RCNN network is predominantly concentrated around the wrist area. However, upon integrating the various proposed modules, the network's focus increasingly shifts towards the distal end of the radius, thus exhibiting stronger resistance to interference from normal gaps between other bones and verifying improved robustness. In summary, the network, after integrating various modules, demonstrates enhanced effectiveness in detecting DRFs.

Fig. 14.

Visual comparison of feature maps.

DRFs often initially present with minor swelling or pain, which can be easily overlooked. Timely and accurate diagnosis can mitigate further trauma and discomfort. The model proposed in this paper performs well in detecting both overt and occult DRFs, facilitating early intervention to prevent disease progression and minimize the risk of misdiagnosis.

Despite the model's strong performance in DRF detection, certain limitations must be addressed. First, the dataset in this study predominantly originates from a single geographic area with a limited sample size, potentially introducing selection bias. Zech et al.³² highlighted that models trained on data from a single source might exhibit reduced generalizability across diverse populations and environments. Therefore, the model's applicability to other regions and patient groups remains unclear. Additionally, the model might exhibit overfitting on specific patterns in the current dataset, which could limit its generalizability across varied healthcare settings. Second, the X-ray images in the dataset primarily stem from similar types of imaging devices, leaving the robustness of the model in handling images from different devices unverified. Variations in imaging protocols and equipment across hospitals could impact model performance, and without thorough testing on a range of devices, it remains challenging to validate its consistency across diverse clinical environments.³³ Lastly, the model is evaluated based on static images, which cannot fully replicate the dynamic changes encountered in actual clinical practice. Factors such as patient positioning, image quality, and artifacts can influence diagnostic accuracy, which are not comprehensively accounted for in this study.

To overcome these limitations, several potential solutions should be explored in the future research. Firstly, collecting more diverse data, especially from different regions, healthcare centers, and equipment types, could improve the model's robustness. Additionally, incorporating dynamic imaging simulations or X-ray video sequences could better capture the variability encountered in real-world situations. Furthermore, expanding the dataset with images that simulate variations in positioning, image quality, and artifacts could also enhance the model's resilience.

This study, informed by radiologists' clinical experience, which necessitates 2 directional X-ray images of the radius for fracture diagnosis, introduces a dual-channel feature fusion network for detecting DRFs. By efficiently integrating the information from AT and LT X-ray images of the distal radius via feature fusion, the proposed method gathers more comprehensive feature information for learning, relative to conventional networks. The network underwent training on 2137 pairs of images and subsequently underwent validation on another set of 427 pairs. Relative to other networks, this method significantly improves the ability to identify DRFs, achieving an accuracy and AP50 value of 98.8% and 98.5%, respectively. Hence, it can potentially reduce the workload of radiologists and assist in diagnosing DRFs.

The proposed method offers valuable clinical applications, particularly as a diagnostic aid in emergency departments where rapid and accurate fracture detection is essential for prompt treatment. By improving detection accuracy, especially in complex or subtle cases, this method can help reduce the risk of missed fractures, contributing to improved patient outcomes. Additionally, deploying this model could alleviate radiologists' workload, enabling them to dedicate more attention to complex cases and thereby enhancing the overall efficiency of patient care.

CRediT authorship contribution statement

Jin Li: Conceptualization, Formal analysis, Methodology, Validation, Visualization, Writing – original draft. Hao-Jie Shan: Data curation, Supervision. Xiao-Wei Yu: Conceptualization, Supervision, Writing – review & editing.

Ethical statement

Not applicable.

Funding

This research was funded by the Shanghai Innovative Medical Device Application Demonstration Project (grant Nos. 23SHS02800) and Shanghai 2022 “Science and Technology Innovation Action Plan" Technical Standards Project (grant Nos. 22DZ2203600)

Declaration of competing interest

No competing financial interests exist.