Deep learning-based automated angle measurement for flatfoot diagnosis in weight-bearing lateral radiographs

Abstract

This study aimed to develop and evaluate a deep learning-based system for the automatic measurement of angles (specifically, Meary’s angle and calcaneal pitch) in weight-bearing lateral radiographs of the foot for flatfoot diagnosis. We utilized 3960 lateral radiographs, either from the left or right foot, sourced from a pool of 4000 patients to construct and evaluate a deep learning-based model. These radiographs were captured between June and November 2021, and patients who had undergone total ankle replacement surgery or ankle arthrodesis surgery were excluded. Various methods, including correlation analysis, Bland–Altman plots, and paired T-tests, were employed to assess the concordance between the angles automatically measured using the system and those assessed by clinical experts. The evaluation dataset comprised 150 weight-bearing radiographs from 150 patients. In all test cases, the angles automatically computed using the deep learning-based system were in good agreement with the reference standards (Meary’s angle: Pearson correlation coefficient (PCC) = 0.964, intraclass correlation coefficient (ICC) = 0.963, concordance correlation coefficient (CCC) = 0.963, p-value = 0.632, mean absolute error (MAE) = 1.59°; calcaneal pitch: PCC = 0.988, ICC = 0.987, CCC = 0.987, p-value = 0.055, MAE = 0.63°). The average time required for angle measurement using only the CPU to execute the deep learning-based system was 11 ± 1 s. The deep learning-based automatic angle measurement system, a tool for diagnosing flatfoot, demonstrated comparable accuracy and reliability with the results obtained by medical professionals for patients without internal fixation devices.

Introduction

Flatfoot, characterized by a collapsed inner foot arch, can stem from various causes including congenital defects, injuries, inflammation, and certain lifestyle habits¹. Diagnosing flatfoot involves various methods, including visual examination (assessment of the shape of the arch and foot structure and overpronation gait analysis), observation of the patients’ walking pattern, footprint examination and foot pressure measurement by applying pressure to the foot’s surface^2,3, and radiographic analysis⁴. Visual measurements are subjective, relying on the doctor's assessment and often exhibiting ambiguities in diagnostic criteria. Radiographic methods offer a relatively high accuracy, as they employ quantitative criteria to evaluate the angles of key bone parts in lateral X-ray images taken during weight-bearing. However, measuring these angles requires specialized knowledge and can be time intensive. Furthermore, variations in the observed values often result from the subjectivity of different observers. To address these challenges, a system that enhances efficiency by accurately and swiftly measuring these parameters is needed⁵.

In recent years, rapid advancement in deep learning technology has resulted in an increase in research leveraging this technology in the field of medical imaging. These studies have achieved exceptional performance across various domains, including lesion classification, organ segmentation, and image enhancement^6–8. To diagnose flatfoot accurately, it is essential to measure various radiographic parameters, which necessitates an initial detection of critical landmarks. This initial step is particularly distinctive and pivotal for improving performance when applying deep learning technology. In this context, we developed a deep learning-based algorithm capable of autonomously measuring key radiographic parameters, such as the lateral Meary’s angle and calcaneal pitch, from weight-bearing lateral radiographs. We performed a comparative analysis of the algorithm’s measurements and those obtained by a clinician to evaluate the diagnostic accuracy and effectiveness of the algorithm.

Material and methods

This retrospective study received approval from the Institutional Review Board of Kyonggi University (KGU-20230216-HR-098). All methods adhered to the ethical principles outlined in the Declaration of Helsinki. The data utilized in this study was de-identified to safeguard the personal information of patients. Consequently, the Institutional Review Board of Kyonggi University waived the necessity for informed consent.

Study population and datasets

In this study, we used the artificial intelligence training dataset⁹ provided by the National Information Society Agency for the diagnosis and monitoring of foot and ankle diseases. This dataset is publicly available and encompasses multi-modal data, including gait analysis data, two-dimensional and three-dimensional X-ray images, and gait videos from a total of 4,000 patients. These images and videos were captured between June and November 2021. Patients who had undergone total ankle replacement surgery or ankle arthrodesis surgery were excluded from the dataset. Consequently, we employed 3,960 lateral radiographs from either the left or right foot to construct a deep learning model and evaluate its performance (see Fig. 1). To ensure a clear demarcation between model development and performance evaluation, we prepared two distinct datasets without overlapping samples. One dataset was designated for the development of deep learning-based algorithms, whereas the other was dedicated to validating the diagnostic efficacy of the developed system. For the development of the deep learning model, the radiographs were converted from digital imaging and communications in medicine (DICOM) format to joint photographic experts group (JPEG) format, which comprises RGB channels. The converted images had an average resolution of 2100 × 1800 pixels, and all the images underwent anonymization before being used for model training and algorithm implementation. In the context of developing the ROI detection and segmentation model, the training data were partitioned into a training set (n = 3048, 80%), a validation set (n = 381, 10%), and a test set (n = 381, 10%) to ensure their independence from one another. The test dataset was utilized to evaluate the mean average precision (mAP) and Dice similarity coefficient (DSC) index of the ROI detection and segmentation model. Each radiograph's segmentation mask and landmarks were annotated by a board-certified clinician. The coordinates of a minimum-sized square bounding box surrounding the clinician-annotated mask were utilized as the bounding box data essential for ROI detection training. The demographic profile of this subset was as follows: the mean age $\pm$ standard deviation was 27 $\pm$ 25 years, with an age range spanning from 0 to 92 years. The gender distribution was nearly balanced, with 1907 male and 1903 female patients.

Figure 1.

Flow chart of the datasets.

The additional evaluation dataset used to validate the performance of the developed system comprised 150 radiographs of the foot. The mean age $\pm$ standard deviation in this dataset was 44 $\pm$ 19 years, with an age range of 9 to 84 years. The male-to-female ratio was 94:56. A clinician meticulously annotated the necessary landmarks required for calculating the lateral Meary's angle and calcaneal pitch. Furthermore, these angles were calculated using the annotated landmarks and served as the reference standard. Table 1 provides a concise summary of the demographic information of the patients included in the datasets used in this study.

Table 1.

Data and patient characteristics.

	Training set	Validation set	Test set	Angle validation
No. of patients	3048	381	381	150
No. of male patients	1540	177	190	94
No. of female patients	1508	204	191	56
Mean age (year)	26.54	27.8	28.58	44.04
Age range (year)	0–92	2–82	1–81	9–84

Reference standard for landmark and angle

This paper proposes a solution designed to automatically measure the Meary's angle and calcaneal pitch, two widely employed radiographic parameters in the diagnosis of flatfoot, based on weight-bearing lateral radiographs. Meary’s angle specifically denotes the angle between the lateral talus and the first metatarsal, serving as an indicator of the collapse of the longitudinal arch. In a normally aligned weight-bearing foot, the longitudinal axis of the talus aligns with the longitudinal axis of the first metatarsal, forming a straight line. A diagnosis of flatfoot is typically made when this angle exceeds 4°¹⁰.

The landmarks used for computing the longitudinal axis of the talus encompass several key points: the topmost point of the talus (Talus-1), a point where a perpendicular line drawn down from Talus-1 to the bounding box of the talus intersects the contour of the talus (Talus-2), and two points located within the neck of the talus (Talus-3, Talus-4)^10–12. Furthermore, landmarks associated with the first metatarsal include points forming a line connecting the proximal dorsal region of the diaphysis and the plantar margins (Meta-1, Meta-2) and points forming a line connecting the distal dorsal of the diaphysis region and the plantar margins (Meta-3, Meta-4)^10,11,13.

To measure the calcaneal pitch, we used two landmarks on the calcaneus: the anterior plantar point of the calcaneal tubercle (Cal-1) and the plantar prominence of the calcaneocuboid joint (Cal-2)^10,11,14. Figure 2 shows the types and precise locations of these landmarks, which were employed to determine Meary's angle and calcaneal pitch. The radiographic parameters are summarized in Table 2, demonstrating that they fall within the normal range.

Figure 2.

Landmarks of the talus, first metatarsal, and calcaneus required to measure Meary’s angle and calcaneal pitch.

Table 2.

Normal and abnormal ranges of Meary’s angle and calcaneal pitch.

Parameter	Normal range
Meary’s angle	0°–4°
Calcaneal pitch	17°–32°

Development of the deep learning-based automatic measurement system

The system proposed in this paper comprises three distinct stages, as illustrated in Fig. 3. ROI detection (Stage 1): The initial stage involves the detection of the ROI where the critical landmarks are situated, specifically the regions of the talus, calcaneus, and the first metatarsal. This task was accomplished using the Yolov5¹⁵ model for ROI detection. Segmentation (Stage 2): In the second stage, three segmentation models were employed to extract three masks corresponding to the talus, calcaneus, and first metatarsal. Each model used the ROI identified in the first stage as the input and produced the respective segmentation mask for each bone structure. All the segmentation models are based on FCBFormer¹⁶. While the structure of the model and hyperparameters remain consistent across all regions, these models were trained on distinct datasets. Landmark Detection (Stage 3): The third stage incorporates an image processing algorithm to extract the outlines of the masks and subsequently detect the essential landmarks required for angle measurements. A total of 10 landmarks were identified, and these landmarks are instrumental in measuring the angles of the talus, calcaneus, and first metatarsal. The Supplementary Method document comprehensively describes the model development, training processes, and details regarding the landmark detection algorithm.

Figure 3.

Illustration of the proposed deep learning-based automated angle measurement system using weight-bearing lateral radiographs.

Statistical analysis

To evaluate the performance of the ROI detection and segmentation models, we employed key metrics such as the mAP and Dice score. Moreover, the accuracy of the landmark detection algorithm and the reliability and precision of the automatically computed radiographic parameters were evaluated to determine the diagnostic capabilities of the proposed automated flatfoot angle measurement system. The accuracy of the landmark detection algorithm was assessed using the mean radial error (MRE) and success detection rate (SDR) of the automatically identified landmarks, as defined in Eq. 1.

$$MRE=\frac{1}{n}\sum _{i=1}^n\sqrt{{\left(p_1^i,-,q_1^i\right)}^2+{\left(p_2^i,-,q_2^i\right)}^2}\times ps$$ 1

where $n$ represents the number of data points, $p_1^i$ and $p_2^i$ denote the x and y coordinates of the ground truth landmark for the $i$-th data point. Furthermore, $q_1^i$ and $q_2^i$ represent the x and y coordinates of the landmark predicted by the system for the $i$-th data point, and $\mathit{ps}$ signifies the pixel spacing.

Equation 2 is used to compute the mean absolute error (MAE), mean square error (MSE), and root mean square error (RMSE) between the radiographic parameters measured by the clinician and those measured automatically. In this equation, $n$ signifies the number of data points, $y_i$ denotes the ground truth angle measured by the clinician, and $x_i$ denotes the automatically measured angle.

$$\begin{array}{cc}\hfill \mathit{MAE}& =\frac{1}{n}\sum _{i=1}^n\left|y_i,-,x_i\right|\hfill \\ \hfill MSE& =\frac{1}{n}\sum _{i=1}^n{\left(y_i,-,x_i\right)}^2\hfill \\ \hfill RMSE& =\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(y_i,-,x_i\right)}^2}\hfill \end{array}$$ 2

The Pearson correlation coefficient (PCC), intraclass correlation coefficient (ICC), and concordance correlation coefficient (CCC) were used to evaluate the agreement between the values measured by the clinician and those automatically measured values. Additionally, the Bland–Altman plot was used to check the similarity between these values and the presence of bias. Finally, the paired T-test was used to compare the average difference between the two groups and confirm the presence of errors in the measured angles. A p-value of less than 0.05 was considered indicative of a significant difference between the automatic and manual calculations.

Ethic statement

This retrospective study received approval from the Institutional Review Board of Kyonggi University (IRB No. KGU-20230216-HR-098). All methods adhere to the ethical standards outlined in the Helsinki Declaration. The Institutional Review Board of Kyonggi University waived the need for informed consent because the data used in this retrospective study were already fully de-identified to protect patient confidentiality.

Results

Performance of ROI detection and segmentation

To detect the ROI encompassing the talus, calcaneus, and first metatarsal, we utilized a YOLOv5-based ROI detection model. The performance of this model was quantified using the mAP, which yielded a high accuracy of approximately 0.996 at mAP 0.5 and approximately 0.919 at mAP 0.5:0.95. Furthermore, we evaluated the performance of the segmentation model, FCBFormer, which was used to extract the segmentation masks for each region. The evaluation used the Dice score, yielding an average performance measurement of 0.970. Table 3 provides a comprehensive summary of the performance metrics for each deep learning model used in our study.

Table 3.

Performance of deep learning models for ROI detection and segmentation.

	mAP 0.5	mAP 0.5:0.95	DSC
Talus	0.996	0.905	0.965
Calcaneal	0.998	0.966	0.981
1st metatarsal	0.994	0.887	0.963
Mean of ROI precision	0.996	0.919	–
Mean of mask DSC	–	–	0.970

mAP: mean average precision, DSC: dice similarity coefficient, ROI: region of interest.

Performance of landmark detection

Accurate detection of landmarks on the talus, calcaneus, and first metatarsal is crucial for measuring the radiographic parameters necessary to diagnose flatfoot. MRE, which quantifies the distance error between the predicted and actual landmarks selected by the clinician, yielded an average value of 2.539 mm. This result indicates that the proposed system attained a remarkably high level of accuracy in detecting the landmarks required for angle measurements. Furthermore, the algorithm's performance was evaluated using the SDR, an indicator that assesses the amount of error that falls within specific thresholds. On average, the SDR was measured as follows: 45% within 2 mm, 67% within 3 mm, and 82% within 4 mm. These results underscore the algorithm’s capacity to provide mostly accurate outcomes. Table 4 presents the MRE for each landmark, whereas Table 5 lists the SDR for each region.

Table 4.

Mean radial error for 10 landmarks.

Landmark	Mean radial error (mm)
Talus-1	2.29
Talus-2	2.96
Talus-3	3.15
Talus-4	2.55
Cal-1	2.82
Cal-2	2.29
Meta-1	2.26
Meta-2	2.19
Meta-3	2.58
Meta-4	2.3

Table 5.

Success detection rate within different distances at the 10 landmarks.

	Success detection rate ($\le 2$ mm)	Success detection rate ($\le 2.5$ mm)	Success detection rate ($\le 3$ mm)	Success detection rate ($\le 4$ mm)
Talus	0.399	0.52	0.623	0.771
Calcaneus	0.503	0.616	0.676	0.823
1st metatarsal	0.5	0.6	0.711	0.853

Performance and reliability of angle measurements

The automatically measured Meary’s angle and calcaneal pitch exhibited a high degree of agreement and correlation with the values measured by the clinician (Meary’s angle: PCC = 0.964, ICC = 0.963, CCC = 0.963; calcaneal pitch: PCC = 0.988, ICC = 0.987, CCC = 0.987). Additionally, the MAE, MSE, and RMSE were computed for each angle to evaluate the accuracy of the automatically measured angles. Furthermore, a paired T-test was conducted using the values measured by the clinician, revealing that there was no statistically significant difference between the two sets of measurements (Meary’s angle: p-value = 0.632; calcaneal pitch: p-value = 0.055). Table 6 presents an overview of the correlation coefficients, paired T-test results, and error indices for each angle.

Table 6.

Correlation coefficients (PCC, ICC, and CCC), error indices (MAE, MSE, and RMSE), and results of paired T-tests measured by a clinician and those measured using the proposed deep learning-based system.

Parameter	PCC	ICC	CCC	Paired T-test (p-value)	MAE	MSE	RMSE
Meary’s angle	0.964 (< .001)	0.963	0.963	0.632	1.59	3.75	1.93
Calcaneal pitch	0.988 (< .001)	0.987	0.987	0.055	0.63	0.67	0.82

PCC: Pearson correlation coefficient, ICC: intraclass correlation coefficient, CCC: concordance correlation coefficient, MAE: mean absolute error, MSE: mean square error, RMSE: root mean square error.

A p-value < 0.005 in Pearson correlation coefficient indicates a strong correlation between the automated measurements and reference standards.

Bland–Altman plots were generated for each angle to evaluate the agreement and bias between two sets of angle values: those measured by the clinician and those measured automatically using the system (see Fig. 4). These plots visually illustrate the distribution of the measurement differences for each region of the centerline, which represents the average difference between the two measurements. Therefore, they enable us to examine any systematic tendencies in the values automatically measured using the system compared to those measured by the clinician. Consequently, the Bland–Altman plots produced for all radiographic parameters confirm a high level of agreement between the values obtained using the system and those obtained by the clinician. Moreover, no significant bias between the two sets of measurements was observed.

Figure 4.

Bland–Altman plots of the reference standards and measurements using the deep learning-based system for each radiographic parameter. X-axis represents the mean of two measurements, and the y-axis represents the difference between two measurements. (a) Meary’s angle. (b) Calcaneal pitch.

Lastly, we conducted time measurements at each processing stage of the system, including the time taken to produce the final results. These time measurements were conducted separately for scenarios where only the CPU was utilized and where the CPU operated in conjunction with the GPU. When only the CPU was used, the ROI detection process averaged 0.95 s, whereas the mask segmentation time for the segmentation model was recorded at an average of 10.49 s. In the cases involving GPU calculations, the ROI detection process took an average of 0.84 s, and the segmentation model's processing time averaged 9.63 s. Moreover, the landmark detection algorithm exclusively employed the CPU and was completed at an average of 0.24 s. Consequently, the total processing time for the system averaged 11.68 s when utilizing only the CPU and 10.71 s when both the CPU and GPU were engaged. The time needed to load the deep learning model remained similar in both cases, with only a minor difference observed during inference in the ROI detection and segmentation stages. Table 7 provides a comprehensive breakdown of the processing time for each proposed stage and the overall processing time.

Table 7.

Comparison of the average and total measurement times of each stage of the proposed deep learning-based system obtained with and without GPU.

Device	1st step	2nd step	3rd step	Total
CPU only	0.95 s	10.49 s	0.24 s	11.68 s
CPU + GPU	0.84 s	9.63 s	0.24 s	10.71 s
CPU + GPUa	0.47 s	2.57 s	0.24 s	3.28 s

^aIndicates the time measured after uploading the deep learning model in advance.

The results of angle measurements conducted on weight-bearing lateral radiographs using the developed deep learning-based system are shown in Fig. 5.

Figure 5.

Example outputs of the developed system. (a) Normal patients. (b) Patients with flatfoot.

Discussion

This study focused on the creation of an automated system designed to measure Meary's angle and calcaneal pitch, which are critical parameters for diagnosing flatfoot from lateral radiographs of the foot. Precise landmarks situated on the talus, calcaneus, and first metatarsal are required for measuring these angles. To this end, we employed a deep learning-based cascaded structure for landmark detection within the proposed system. The deep learning model utilized for ROI detection and segmentation exhibited outstanding performance, as evidenced by an mAP 0.5 of 0.996 and a Dice score of 0.970, signifying excellent accuracy. Moreover, we employed an additional evaluation dataset to compare the angles automatically computed using the system with those manually measured by the clinician. The results of this comparative analysis underscore a high level of agreement and correlation, and statistical analysis revealed no significant differences between the two sets of angle measurements (Meary’s angle: PCC = 0.964, ICC = 0.963, CCC = 0.963, p-value = 0.632; calcaneal pitch: PCC = 0.988, ICC = 0.987, CCC = 0.987, p-value = 0.055).

Several studies have successfully applied deep learning to automatically and consistently measure various radiographic parameters related to flatfoot diagnosis, demonstrating improved performance over conventional methods. FlatNet¹⁷ is a fully automated deep learning model designed to detect 25 landmarks comprising the talus, calcaneus, and first metatarsal bones in the weight-bearing lateral radiographs of the foot. This model achieved an average distance difference of 0.84 $\pm$ 0.73 mm between the predicted and ground-truth landmarks. However, the above study is limited in that it was conducted exclusively on 19-year-old males, whereas our study utilized radiographs from a demographically diverse population of patients aged 9 to 84 years, with a male-to-female ratio of 0.63:0.37. Nonetheless, should their methodology demonstrate comparable consistency and reliability when validated across a broader demographic spectrum, it is anticipated that their approach could be used in the detection of various anatomical landmarks required for automated measurements, including flatfoot diagnosis.

Ryu et al.¹⁸ assessed the effectiveness of using semantic segmentation with active learning to improve the accuracy and efficiency of automated measurements for various flatfoot-related angles. The above study and the present study involved the automatic measurement of calcaneal pitch. The Bland–Altman analysis of this common parameter showed that our method initially demonstrated a smaller mean difference (0.13) compared with that reported in the study by Ryu et al. (0.24) without active learning. However, with the active learning, their method achieved a better mean difference (0.3) than that obtained in the present study. Furthermore, the 95% limits of agreement (LoA) ranged from − 0.76 to 0.82 for their method with active learning, indicating narrower limits of agreement compared with that obtained using our method (95% LoA: − 1.5 to 1.7). These findings suggest that incorporating active learning, as demonstrated in their approach, could further enhance the accuracy and precision of our method, ultimately leading to more reliable clinical assessments of flatfoot conditions. Koo et al.¹⁹ proposed a methodology for angle measurement in lateral foot radiographs using deep learning technology to ensure accurate flatfoot diagnosis (Meary’s angle: ICC = 0.955, p-value < 0.0001; calcaneal pitch: ICC = 0.984, p-value < 0.0001). Their research involved the use of an image segmentation model to generate mask images for the talus, calcaneus, and the first metatarsal, which is similar to the method proposed in this study. However, the two studies differ in terms of how these masks were employed to measure the angles. In particular, the talus and the first metatarsal have varying shapes, posing a challenge in determining the reference points of the landmarks for them.

Koo et al.¹⁹ opted to use the iterative closest point (ICP) method to detect landmarks, directly relying on points on the outline. They computed the similarity between the predicted mask and the reference mask image using ICP and selected 10 reference masks with the highest similarity. Subsequently, they calculated the distances between the outline of the predicted mask and the landmarks set on the selected reference masks. The location of the point with the shortest distance was then designated as the final landmark. This approach allows for accurate predictions by referencing available reference data, particularly when the shape of the bone is ambiguous. However, it may encounter difficulties in cases where reference data similar to the predicted mask are unavailable or when the number of data points is limited. In contrast, our study used a landmark detection algorithm based on medical definitions to measure landmarks for the talus, calcaneus, and first metatarsal. This approach enhanced the overall performance of the angle measurement system. Moreover, the system's ability to efficiently measure numerous images in approximately 11 s enables swift determination of whether a foot exhibits flatfoot characteristics.

This study has several limitations that should be acknowledged. First, while deep learning technology was utilized to extract accurate landmarks, the present study used 3,810 weight-bearing lateral foot X-ray images for model development and performance verification. Although this is relatively high compared with those used in previous studies, enhancing the generalization performance of the developed algorithm and ensuring its stability would require the use of a more extensive and diverse set of X-ray images from multiple institutions for both model development and performance verification.

Second, the exclusion of data from patients who underwent artificial joint surgery limited the training and testing of the model for diverse patient groups, making diagnosis difficult. To address this challenge related to patient group diversity, a more sophisticated deep learning model should be developed, and additional datasets containing diverse patient groups should be acquired.

Conclusion

The developed deep learning-based flatfoot angle measurement system demonstrates the ability to minimize errors compared with the values measured by clinicians, thereby improving accuracy and reliability. This automated approach, which swiftly obtains measurements without human intervention, not only reduces the workload of clinicians but also assists them in devising appropriate treatment plans, thereby enhancing treatment effectiveness.

Future research directions should prioritize the development of a more comprehensive flatfoot angle measurement system by expanding the diversity and quantity of training data and improving the model's feature learning capabilities. Furthermore, prospective research should encompass diverse patient groups, including those with artificial joints or foot deformities to facilitate further advancements in this field.

Author contributions

Guarantors of integrity of entire study, W.-J.N., B.-D.L., and M.L.; conceptualization, M.L. and B.-D.L.; literature review, W.-J.N., and B.-D.L.; supervision, B.-D.L.; software, W.-J.N. and B.-D.L.; formal analysis, W.-J.N., B.-D.L., and M.L.; clinical studies, M.L.; writing—original draft preparation, W.-J.N., B.-D.L., and M.L.; writing—review and editing, W.-J.N., B.-D.L., and M.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (No. 2020R1A6A1A03040583). This work was also supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (grant No. NRF-2022R1A2C1007169).

Data availability

The dataset used in this study is publicly available at https://aihub.or.kr/ (accessed on January 12, 2024).

Competing interests

The authors declare no competing interests.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-69549-3.