Identification of auricular acupoints using a convolutional neural network

Abstract

Background

The accurate identification of acupoints is an essential task in acupuncture therapy. Recent advancements in artificial intelligence (AI) have led to the exploration of automated landmark detection systems, which may provide more accurate and reliable acupoint detection. This study investigated the efficiency of an AI model in predicting the shenmen, lung, and mouth auricular acupoints and compared its performance to placements made by a practitioner of traditional Korean medicine.

Methods

Ear images from 39 individuals were captured from three different angles. The mask region-based convolutional neural network (Mask R-CNN) model was utilized to isolate the ear region, followed by landmark detection using a CNN model trained on resized images to predict three auricular acupoints. Model reliability was enhanced by treating each acupoint as a separate prediction coordinate. Acupoint distribution was also estimated using a kernel density estimation method.

Results

Centroids of auricular acupoints predicted by the CNN model showed deviations of < 3 pixels from traditional placements by the practitioner. Kernel density estimation showed that CNN predictions led to narrower acupoint distributions compared with those placed by the practitioner, suggesting higher consistency in CNN model predictions across different images.

Conclusions

The AI-driven approach showed significant potential in improving both the accuracy and consistency of auricular acupoint identification. These findings support the integration of AI into acupuncture practice as a reliable tool for enhancing clinical accuracy and precision of acupoint location.

1. Introduction

Acupoints are areas of the body that can be stimulated by the insertion of an acupuncture needle.¹ The accurate location of acupoints is crucial in clinical acupuncture practice.² Advances in acupuncture research over the past few decades have produced no conclusive findings about the anatomical or physiological properties of acupoints,³ and the location of acupoints frequently varies among practitioners.⁴ However, the use of assistive devices has shown less variation among acupoint locations determined by different practitioners.⁵ For example, acupoint locations were found more precisely using laser-assisted instruments, which offered visual guidance.⁶ Nonetheless, it remains challenging to ensure efficient, identical acupoint localization.

Auricular acupoints have been extensively utilized in complementary and alternative medicine as a micro-acupuncture system.7, 8, 9 Auriculotherapy, a practical method to treat illnesses, has demonstrated benefits in the treatment of acute and chronic pain, as well as insomnia.¹⁰ Considering that auricular acupoints were developed by observing the resemblance of the ear to an inverted fetus, it has been hypothesized that these acupoints may be easily identified according to the somatotopic organization of the ear.¹¹ The microsystemic characteristics of auricle reflect the entire body, forming the foundation for its diverse therapeutic effects.¹² Recently, transcutaneous auricular vagus nerve stimulation (taVNS) has gained popularity as a non-invasive treatment method. It involves applying an electrical current to the cutaneous receptive field created by the auricular branch of the vagus nerve.¹³ To ensure the quality of auricular acupoint research, the international standardization efforts have been focused on the nomenclature, location methods, and consistency between names and locations of auricular acupoints.¹⁴ However, the validity of auricular acupoint localization has not been studied.

Numerous studies have attempted to locate acupoints using computer vision and machine learning approaches. An automatic facial acupoint positioning system was developed by analyzing facial features using image processing techniques.¹⁵ A deep learning approach has been proposed for the automatic, accurate location of acupoints on the back based on prior information.¹⁶ Using augmented reality (AR) technology, an assisted acupuncture system can map out acupoint locations and offer visual assistance during treatments.¹⁷ AR can also be used to display acupoints on an image of the human body.¹⁸ A new method for acupoint localization was developed using landmark points and a three-dimensional morphable model constructed based on a polygonal grid; the results showed remarkably high accuracy and an acceptable reaction time.¹⁹ A deep convolutional neural network-based technique for detecting acupoints was developed to predict the positions of acupoints surrounding joints.²⁰ A smartphone-based automated acupoint positioning system for laser acupuncture was developed using computer vision and machine learning techniques²¹; an AR-based acupoint identification system was able to discover and visualize facial acupoints and auricular zone maps in real time.²² Despite the development of these numerous innovative methods, few studies have confirmed the localization of auricular acupoints.

A convolutional neural network (CNN) is a type of deep learning model, which is itself a subset of machine learning, a core area within the broader field of artificial intelligence (AI).²⁰ In this study, we localized auricular acupoints using the mask region-based convolutional neural network (Mask R-CNN) to evaluate the effectiveness of the CNN model in predicting auricular acupoint locations. We compared the model’s performance with that of acupoint placements detected by a practitioner of traditional Korean medicine. A kernel density estimation method was applied to analyze the distributions of auricular acupoints identified by both methods.

2. Methods

2.1. Dataset

2.1.1. Data source for auricular acupoints

Three auricular acupoints (shenmen: P1, lung: P2, and mouth: P3) were selected to test the localization of auricular acupoints using the CNN. These auricular acupoints have been widely used for smoking cessation therapy.^{23, 24} For instance, shenmen (P1), located at the lower edge of the crus of the helix, near the lateral edge of the triangular fossa, is an important point in clinical practice for treating anxiety, insomnia, pain, and other conditions. These effects are closely associated with the abundant presence of the auriculotemporal nerve and the auricular branch of the vagus nerve.¹² An experienced practitioner of traditional Korean medicine (DY) marked these points on the right ear in 39 participants, according to recommendations based on an international standard.²⁵ Participants were recruited from Aug 16 to August 26, 2024. All participants received a detailed explanation of the study and provided written informed consent prior to its commencement. All experiments were conducted in accordance with guidelines issued by the human subjects committee and were approved by the institutional review board of Kyung Hee University, Seoul, Republic of Korea (approval no. KHSIRB-24–263).

2.1.2. KHU-EAR dataset construction

The KHU-EAR datasets include the baseline characteristics of the participants with a mean age of 22.3 years, consisting of 20 females and 19 males, all of whom are Korean. The dataset comprised high-resolution images of the right ear from 39 individuals, captured from three random angles, resulting in a total of 117 images. The dataset was divided into two sets. Set A included right ear images with the three auricular acupoints marked in black pen, and Set B included right ear images without markings. Images in each set were taken at three different angles, with no direct image matching between sets. The images included portions of the face besides the ear, and exhibited various external items such as glasses, hats, earrings, piercings, masks, and different hair colors.

2.2. Auricular acupoint image processing

2.2.1. Ear region extraction

An extraction model may fail to correctly detect the target in an image due to noise such as glasses, hair, earrings, masks, or piercings.²⁶ To eliminate these factors, we first detected, cropped, and extracted ear regions from the images²⁷ for use as landmark detection inputs.

Our prediction technique is based on the Mask R-CNN model, renowned for its efficacy in instance segmentation tasks.²⁸ The Mask R-CNN model is an extension of Faster R-CNN, which adds a branch for predicting segmentation masks on each region of interest (ROI), in parallel with existing branches for classification and bounding box regression. This method allowed for precise object localization, with simultaneous generation of a high-quality segmentation mask for each instance, making it ideal for tasks involving object detection and pixel-level segmentation.²⁸

To train the model, we manually labeled the ear region of each image in the KHU_EAR dataset. The training data consisted of image data and JSON-formatted polygon coordinate data representing the ear region. Using the trained model, we retrieved and cropped the ear from each image. The cropped images were then used to determine ear landmarks and provide input for further model training.

2.2.2. Image augmentation

The KHU_EAR dataset contains 117 high-resolution images of the right ear from 39 different people, taken at three random orientations. However, 117 images may be insufficient for effective landmark detection model training, because limited datasets can result in poor model generalization to new data. To address this issue and improve model robustness and performance,²⁹ we artificially expanded the dataset size and diversity by collecting ear images from various angles.

During data augmentation, it is critical to ensure that landmarks are accurately detected in rotated images.²⁷ As a pilot experiment, we rotated images at several angles ranging from –50° to +50° The results showed that our landmark detection model performed well in the specific angle range of –20° to +20° Consequently, we augmented the number of images by rotating the images at 5° intervals within this range, specifically at –20°, –15°, –10°, –5°, +5°, +10°, +15°, and +20° We adjusted the landmark coordinates in the rotated images to correspond to the rotation angle as follows. The original coordinates of the landmarks were represented as $\left(x,y\right).$ After applying rotation by angle $\theta$, the new coordinates $\left(x^{\prime },y^{\prime }\right)$ were calculated using the following transformation equations:

$$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right).$$

This transformation ensured that the landmarks maintained their relative positions within the rotated image. The equations for the new coordinates are as follows:

$$x^{\prime }=x\cos \theta -y\sin \theta$$

$$y^{\prime }=x\sin \theta +y\cos \theta .$$

By applying these equations, we were able to precisely change the landmark coordinates to match those of the rotated image. After data augmentation, we expanded the original dataset of 105 images (12 images were excluded from initial 117 images because the landmarks were obscured or blurred) with landmarks to 945 images, producing a 9-fold increase in dataset size.

2.3. Auricular acupoint identification

Auricular acupoints were identified in two steps: ear landmark detection, followed by the prediction of three acupoints based on the identified landmarks (Fig. 1). Our auricular acupoint identification model was trained on a 64-bit Windows 10 operating system using a 13th generation Intel Core i7–13,700 K central processing unit, with 64 GB of random-access memory, and an Nvidia RTX 4070Ti graphics card. Model performance was assessed using R² and mean square error (MSE). The training process consisted of 20 epochs, each with 33 iterations. The optimal model performance was observed at iteration 660.

Fig. 1.

Overview of the ear landmark detection and acupoint prediction pipeline. After the input of an image of an individual ear, the ear region was extracted using the mask region-based convolutional neural network (Mask R-CNN) model, which segmented the ear, enclosed each segment within a border, and then cropped and resized each ear segment. Next, data augmentation was performed by rotating the cropped ear images at different angles to enhance the size of the training dataset. The augmented images were entered into the CNN model, which predicted 55 ear landmark points that were used to predict individual acupoints. Separate neural networks were utilized to predict the coordinates of acupoints P1 (shenmen), P2 (lung), and P3 (mouth), which were then concatenated to determine the final acupoints. The final output consisted of the predicted acupoints marked on the ear image. Spatial distribution of acupoints determined by a practitioner of traditional Korean medicine was compared to those predicted by the CNN model.

2.3.1. Ear landmark detection

We explored various public repositories on GitHub related to ear landmark detection and selected a project aligned with our objectives (https://github.com/kbulutozler/ear-landmark-detection-with-CNN). This CNN model takes 224 × 224 images as input and produces 55 predicted landmark points as outputs. The model comprises a series of convolutional layers designed to extract detailed features from the input images, followed by dense layers to predict the landmark points.

The architecture includes six convolutional layers with different filter sizes (16, 32, 64, 128, 256, and 512) and kernel sizes (3 × 3 and 5 × 5), interspersed with max-pooling layers (2 × 2) to reduce the spatial dimensions. Batch normalization was conducted after the fourth and sixth convolutional layers to stabilize learning; dropout layers (0.3 and 0.5) were used to prevent overfitting. The network concludes with a fully connected layer of 1024 neurons and an output layer of 110 neurons to predict the 55 landmark points on the ear, each represented by x and y coordinates.

2.3.2. Prediction of three auricular acupoints

In our initial prediction model, we utilized a simultaneous prediction approach to obtain the coordinates for P1, P2, and P3. This method resulted in consistent geometrical patterns that did not accurately align with the actual coordinates, thereby reducing prediction accuracy. For example, the shapes and lengths of the areas between coordinates P1, P2, and P3 frequently followed a predictable pattern that differed from their actual positions.

To enhance accuracy, we redesigned the model to predict each coordinate individually. Thus, rather than relying on a single model that simultaneously predicted all three coordinates, we used three separate neural networks, where each was dedicated to the prediction of a single coordinate (P1, P2, or P3). The independently predicted coordinates were then merged to estimate the final points.³⁰ This modification mitigated the introduction of artificial geometrical patterns and improved the overall landmark detection accuracy (Fig. 1).

2.4. Data analysis

The distributions of the auricular acupoints were estimated using a kernel density estimation method in R v4.0.2 (R Core Team, Vienna, Austria). This approach allowed us to estimate the probability density function of a random variable. The three auricular acupoints were obtained by the traditional practitioner and the CNN model; their distributions were analyzed using a color map representing the probability density of the acupoints. Distance errors between the two approaches were statistically analyzed using the independent-sample Student’s t-test, with a significance threshold of P < 0.05. Violin plots were drawn using Raincloud Plots (https://gabrifc.shinyapps.io/raincloudplots).

3. Results

3.1. Accuracy of acupoint location by the practitioner and CNN model

We detected no significant differences in acupoint locations between the two methods; errors of < 3 pixels were evident between the predicted and actual x and y coordinates (Table 1). The CNN model estimates of the x coordinates for P1, P2, and P3 deviated from those of the practitioner-determined acupoints by 0.2, 1.2, and 1.9 pixels, respectively. The y coordinate deviations were also negligible, at 0.5, 0.7, and 1.8 pixels, respectively. These findings highlight the capacity of the model to predict acupoint locations with high accuracy and low error (Fig. 2).

Table 1.

Coordinates and error distances between auricular acupoints (P1: shenmen, P2: lung, and P3: mouth) determined by the convolutional neural network (CNN) model and a practitioner of traditional Korean medicine. SD, standard deviation.

		P1	P2	P3
Practitioner	Center of mass (x, y coordinates)	126.9, 67.9	151.7, 108.7	139.4, 139.0
	Error distance (mean ± SD)	15.44 ± 8.52	18.49 ± 9.30	22.84 ± 12.79
CNN model	Center of mass (x, y coordinates)	127.1, 68.4	150.5, 109.4	141.3, 140.8
	Error distance (mean ± SD)	10.89 ± 6.51	13.55 ± 6.98	14.82 ± 7.99

Fig. 2.

Accuracy of acupoint prediction by the CNN model (right) compared with acupoint determination by a practitioner of Korean traditional medicine (left). Locations are indicated as follows: P1 in blue (practitioner) and red (CNN), P2 in cyan (practitioner) and green (CNN), and P3 in brown (practitioner) and yellow (CNN). CNN-based acupoint predictions are clustered around practitioner-determined landmarks, demonstrating the consistent and accurate performance of the model.

3.2. Acupoint distributions determined by the practitioner and CNN model

Kernel density estimation showed that the CNN model produced narrower acupoint distributions than those determined by the practitioner. Across all point sets, mean error distances of the CNN model were substantially lower (P1: 10.89 pixels, P2: 13.55 pixels, P3: 14.82 pixels) than those of the practitioner-based distributions (15.44, 18.49, and 22.84 pixels, respectively). Student’s t-tests comparing points P1–3 between the two methods showed that the mean error distance was significantly lower for acupoints predicted by the CNN model than for acupoints determined by the practitioner (P1: t = 6.29, P < 0.01; P2: t = 6.48, P < 0.01; P3: t = 10.26, P < 0.01; Fig. 3).

Fig. 3.

Spatial distribution of acupoints determined by a practitioner of traditional Korean medicine and those predicted by the CNN model. Auricular acupoint distributions were determined using a kernel density estimation method in R v4.0.2. The probability density map revealed that the CNN model (blue) predicted a narrower acupoint distribution relative to that determined by the practitioner (red); a significantly lower mean error distance was predicted by the CNN model for all point sets, relative to the mean distance error calculated for the practitioner (P1: t = 6.29, P < 0.01; P2: t = 6.48, P < 0.01; P3: t = 10.26, P < 0.01).

4. Discussion

This study found no significant differences between auricular acupoint locations determined by a practitioner of traditional Korean medicine and those predicted by the CNN model, indicating that our AI model displayed high accuracy. The CNN model predictions showed a narrower distribution of auricular acupoints compared with the distribution obtained by the practitioner, suggesting that the CNN model had higher precision. Our findings demonstrate the potential of the AI-driven approach for increasing both the accuracy and precision of acupoint localization.

The high accuracy of our CNN model was comparable to that of the traditional Korean medicine practitioner. Several previous studies have investigated the use of machine learning models for acupoint localization, demonstrating increases in accuracy and consistency. For example, Liu et al.¹⁶ developed a back acupoint localization method using deep learning, achieving an accuracy rate of 86.3 %. In another study, a deep residual network was used to accurately locate facial acupoints, achieving high concordance with traditional methods and error margins within 5 pixels.³¹ A recent acupoint localization study developed a deep learning-based approach to correct for variation between the left and right ears; Sun et al.³² trained a neural network that incorporated a direction normalization module and a self-collected dataset of 91 manually annotated landmark points to achieve an average normalized mean error of 0.051 ± 0.002, successfully identifying auricular areas related to the digestive system based on these landmarks.³² An AR-based acupoint detection method was developed to locate and display auricular acupoints in real time.²² Our AI-driven technique revealed that such approaches have high potential for improving the accuracy of auricular acupoint localization.

Our adapted Mask R-CNN model exhibited high accuracy that was comparable to that attained by an experienced practitioner. Kernel density estimation revealed a smaller distribution for CNN model-predicted points than that obtained by the practitioner. Kernel density estimation is a spatial analysis method that considers relative feature locations and yields population inferences based on finite data samples.³³ In our previous study, kernel density estimation revealed smaller distributions for acupoint PC5 for laser device placement compared with those obtained for naked-eye placement.⁶ In the current study, the distributions of CNN-predicted acupoints also showed lower variability than those obtained by the practitioner. Thus, the kernel density approach is a useful method for measuring acupoint distributions.

The newly proposed auricular acupoint prediction model offers several advantages over other methods, particularly in enhancing the accuracy and reliability of acupoint localization. The Mask R-CNN model for ear region extraction confers precise identification and segmentation of ear regions, effectively minimizing noise from external factors such as glasses and hair. The application of image augmentation techniques enhances model robustness, enabling it to generalize more effectively across diverse datasets. Additionally, the approach of predicting each acupoint individually rather than simultaneously addresses the issue of potential artificial geometrical patterns, leading to improved accuracy in landmark detection. Our stepwise and systematic methodology significantly improved the precision of acupoint prediction, leading to more reliable and consistent outcomes. Our model exhibited high accuracy commensurate with acupoints identified by an experienced practitioner, achieving distance errors of < 3 pixels compared with the x and y coordinates of the latter. Three pixels corresponds to 0.54 mm, ensuring that the estimation of acupoints is both safe and effective for their localization. Furthermore, our CNN model showed greater precision than the practitioner in locating auricular acupoints, with lower variability in the acupoint distributions. Our CNN model may be used to locate auricular acupoints coupled with computer vision and machine learning approaches, due to its accuracy and consistency in localizing acupoints.

The three target acupoints used in this study (shenmen, lung, and mouth) have been extensively utilized for smoking cessation therapy.^{23, 24} It is crucial to simultaneously identify multiple acupoints because pressure is typically applied to combinations of acupoints.^{34, 35} However, rather than using a single model to simultaneously predict all three coordinates, we used three different neural networks, where each was responsible for the prediction of a single coordinate. The separately estimated coordinates were then combined to estimate the final points. This approach will allow the AI model to predict combinations of acupoints simultaneously. Our method can be extended to other acupoint combinations, such as those involved in treating insomnia, obesity, and pain.¹⁰ Future studies should evaluate model performance in the automatic identification of acupoints based on images of other body parts.

This study has several limitations. First, the dataset was limited in both size and ethnic diversity. To enhance the generalizability of the results, it is essential to include samples from a broader and more diverse range of images. Second, our model was trained and tested using internally collected data. Future studies should incorporate external validation to thoroughly evaluate the robustness of our model. Lastly, the system should be designed for healthcare professionals or patients to localize acupoints in real-time, thereby improving clinical convenience and intelligence.

In conclusion, our CNN model demonstrated good accuracy in locating auricular acupoints, consistent with the accuracy exhibited by a traditional medicine practitioner. Our AI-driven technique showed great potential for increasing acupoint location accuracy and precision.

Author contributions

Conceptualization: YC and JK; DY and IL collected the data; Formal analysis: JK and YK; Writing - original draft: JK and YC; Writing - review and editing: JK, YK, DY, IL, and YC. All authors read and approved the final manuscript.

Declaration of competing interest

YC is an editorial board member of this journal but this role had no influence on the editorial process or decision. The authors declare that they have no other competing interests.