Reproducibility and reliability of three-dimensional soft tissue landmark identification using three-dimensional stereophotogrammetry

Abstract

Objective:

To evaluate the intraexaminer repeatability and interexaminer reproducibility of soft tissue landmarks on three-dimensional (3-D) stereophogrammetric images.

Materials and Methods:

Thirty-four stereophotogrammetric images were taken and 19 soft tissue points were identified. The images were obtained using the 3-DMD Face (3-DMD TM Ltd, Atlanta, Ga) system. Two examiners marked 34 images manually with a mouse-driven cursor 4 weeks apart. Intraexaminer marking differences were calculated and classified as <0.5 mm, 0.5–1 mm, and >1 mm. Intraclass correlation coefficients were calculated for intraexaminer reliability. A paired-samples t-test was used to evaluate the difference between the examiners. Interexaminer reproducibility was evaluated by kappa analysis. Statistical significance was set at P < .05.

Results:

Only one landmark (labiale superior) had an intraexaminer marking difference less than 0.5 mm. Existing landmarks had an intraexaminer difference less than 1 mm, but higher than 0.5 mm. The intraclass correlation coefficients (ICCs) indicated good intraexaminer repeatability for both observers. The ICC range for examiners 1 and 2 was 0.986–1.000 and 0.990–1.000, respectively. Kappa scores showed good interexaminer agreement, especially on the z-axis.

Conclusions:

Except labiale superior, the soft tissue landmarks used in this study were shown to have moderate reproducibility, but the difference between the landmarks was less than 1 mm, and they had clinically acceptable reproducibility.

INTRODUCTION

Improvement in the facial appearance is one of the most important factors for patients seeking orthodontic treatment. With the paradigm shift from hard tissue to soft tissue, the key determinant in orthodontic diagnosis and treatment success now lies behind soft tissue positions. This paradigm shift necessitates using three-dimensional (3-D) imaging, because traditional two-dimensional cephalometry places emphasis on hard tissue landmarks, because their reproducibility is better than soft tissue landmarks.1 The rapid development of 3-D imaging systems has enabled us to diagnose facial soft tissues in detail and open a new window for 3-D soft tissue evaluation in orthodontic planning and treatment results. These 3-D imaging techniques are CBCT,2–4 facial surface laser scanning,5,6 and 3-D stereophotogrammetry.7

It is now possible to obtain viable and objective results in clinical trials using such advanced 3-D stereophotogrammetry, being one of the current 3-D imaging methods.8 Especially in cases where 3-D cone beam computerized tomography is contraindicated, 3-D facial scanning seems a good alternative to record and analyze facial soft tissues.9

One use of stereophotogrammetry is the analysis of proportions and distances between soft tissue points of the face. An examiner marks the soft tissue landmarks manually using a mouse-driven cursor. Since it is not possible to palpate soft tissue points on 3-D photographs, the repeatability of the measurements of such soft tissue points—especially pogonion and zygion—is open to discussion.10

There are several studies on repeatability and reliability of 3-D soft tissue facial landmarks,8–13 but conflicting results have been reported in these studies and no consensus has been reached. The aim of this study was to evaluate the intraexaminer repeatability and interexaminer reproducibility of soft tissue landmarks on 3-D stereophotogrammetric images. For the present study, the null hypothesis assumes that there is no difference in the intraexaminer reliability (no statistically meaningful inter- or intraexaminer consistency) or interexaminer reproducibility (absolute agreement) of soft tissue landmarks on 3-D stereophotogrammetric images.

MATERIALS AND METHODS

Thirty-four stereophotogrammetric images taken from the patients (17 male, 17 female) of the same race who applied to postgraduate orthodontics clinics were used. Of the 34 patients, 9 had Class I, 20 had Class II, and 5 had Class III malocclusion. The mean age of the patients was 13.1 ± 1.3 years. The ethics committee of Izmir Katip Celebi University approved the study (No. 54). Each patient's parent(s) signed an informed consent before inclusion in the study.

Inclusion criteria were as follows:

• no history of previous orthodontic treatment,

• no craniofacial or neuromuscular deformities,

• no history of facial trauma or plastic surgery intervention,

• being in the healthy percentage of the child mass index.

Stereophotogrammetric images were obtained using the 3DMD Face (3DMD TM Ltd, Atlanta, Ga) system. This system consisted of six cameras in total: two infrared cameras and one color camera in 2 sets. The cameras were positioned at predetermined angles and distances. 3DMDvultus software creates an image from the records it obtains through the six cameras from two separate points on the x, y, and z coordinates. The software is also used for image processing, mapping, and measurements and is calibrated with the camera system to obtain flawless 3-D images for processing.

Definitions of the 19 soft tissue points used in the study (Figure 1) are given in Table 1. Both examiners had undergone special training for image capture, processing, and measurement. Prior to the study, the examiners had annotated more than 100 scans. To prevent memory bias, two examiners independent of each other marked all soft tissue landmarks on 34 images 4 weeks apart.

Figure 1.

Soft tissue points used in this study.

Table 1.

Soft Tissue Landmarks Used in This Study.

Marking Procedure

The 3-D photographs were adjusted according to the 3-D coordinate system suggested by Plooji et al.8 To evaluate the landmark coordinate data, both examiners separately marked the soft tissue points in the frontal view, and then these markings were verified in profile and top-down views. The coordinates of the soft tissue points on the x, y, and z planes were extracted and recorded. The marking procedure was repeated after 4 weeks by the same examiners and the second coordinates were recorded. The software was designed to automatically calculate the Euclidean distance of the landmark positions; the difference in landmark positions was also recorded separately for both examiners. The 19 landmarks were classified according to the reading differences.12 The classifications were

• highly reproducible landmarks wherein differences between readings were less than 0.5 mm,

• moderately reproducible landmarks wherein differences between readings were more than 0.5 mm, but less than 1 mm,

• poorly reproducible landmarks wherein differences between readings showed values greater than 1 mm.

Statistical Analysis

IBM SPSS version 22.0 (SPSS Inc, Chicago, Ill) was used for statistical analysis. Descriptive statistics included the mean, standard deviation, and minimum and maximum values. Intraexaminer reliability was calculated using intraclass correlation coefficients. A paired-samples t-test was used to assess the difference between examiners. Interexaminer reproducibility was evaluated by kappa analysis. Statistical significance was set at P < .05.

RESULTS

Intraclass correlation coefficients (ICCs) for the examiners are given in Table 2. The ICC values indicate good intraexaminer repeatability for both observers. The ICC range for examiners 1 and 2 was 0.986–1.000 and 0.990–1.000, respectively.

Table 2.

Intraexaminer Correlation Coefficients

Descriptive statistics of the data and interexaminer reproducibility assessments for the samples are shown in Table 3. All the landmarks were reproducible to less than 1 mm. The lowest reproducibility value was recorded for zygion (Zyg) point for both examiners, but this point was also classified as a moderately reproducible landmark (<1 mm). The landmark labiale superior (Lab-Sup) was the most reproducible landmark for both examiners.

Table 3.

Differences Between First and Second Examiner

Considering the difference between the two examiners, right alare landmark (AL-R) showed the lowest (0.110 mm) and left endocanthion (EN-L) showed the highest reproducibility (0 mm). No significant statistical difference was found between the markings of the first and second examiner. Thus, all the landmarks used in this study have been found to show high intra- and interexaminer consistency according to the results of the paired-samples t-test.

Table 4 shows the kappa scores of each annotation point on the x, y, and z coordinates. Highest consistency was observed on the z-axis for all the annotation points.

Table 4.

Kappa Scores

The null hypotheses testing intraexaminer consistency and interexaminer agreement failed to be rejected.

DISCUSSION

Because of the paradigm shift from hard to soft tissue, 3-D facial soft tissue evaluations have become necessary. These facial scanning systems are continuously being developed, and the major concern about these new systems is reproducibility and reliability of the measurements obtained from the facial scans. The aim of this study was to assess the reproducibility and reliability of 19 anthropometric soft tissue landmarks marked on 3-D stereophotogrammetric images.

The most error-prone landmarks are those that are more leniently defined, such the points defining the jaw line.13 These landmarks are relatively harder to determine visually, so require other methods such as palpation.9 They are located on convex surfaces, and the operator should control their localization on the x-, y-, z-axes separately. Examples such as zygion, gonion, and tragion have been shown to have unacceptable reproducibility.8,10 In this study, zygion, pogonion, nasion, and glabella had the highest intraexaminer variability (less than 1 mm), and the landmarks were classified as moderately reproducible.

Landmarks with well-defined borders or edges were shown to have higher degrees of reproducibility than the ones located on gently curving slopes.12 The reproducibility and reliability of landmarks that are near orbits (endocanthion and exocanthion) and the mouth (labiale superior, labiale inferior, and cheilion) are considered moderate. These landmarks are easy to determine visually. Improvements in 3-D imaging systems' resolution have made determination of these landmarks easier.

Hajeer et al.8 identified a landmark as highly reproducible if the standard deviation (SD) from centroid was 0.5 mm or less. They evaluated intraexaminer reproducibility and found that 20 out of 30 landmarks showed high reproducibility. Gwilliam et al.10 found 12 out of 24 landmarks that were reproducible (intraexaminer data) assuming that the SD is less than 1 mm. They also evaluated the interexaminer data and reported that only cheilion and labiale superior were reproducible. In the current study, there was 1 landmark classified as highly reproducible and 18 landmarks moderately reproducible for each examiner.10 For the interexaminer data, all landmarks were found to be reproducible, and no statistically significant difference was found between two examiners.

Glabella, nasion, and pogonion showed poor reproducibility on the y-axis.12 The authors related this finding to good manipulation skills and clinical knowledge of the natural head position. In the present study, all facial scans were taken in the natural head position, which has been shown to be highly reproducible.14 Another standardization criterion for the scanning procedure was the selection of participants from the subjects within the healthy percentage according to body mass index. This may have led the examiners to encounter more standardized soft tissue contours and not deformations such as soft tissue sag.

At the beginning of the study, to prevent possible errors that might arise from differing landmark definitions by the examiners, 19 facial landmarks were clearly defined and, if there was a conflict, the examiners discussed the localization until a consensus was reached. These measures may have resulted in high interexaminer reproducibility. Also, we found high correlation coefficients for both examiners in all three planes of space. All landmarks showed a kappa score of 1.000 for the z-axis. In general, relatively poor kappa scores were found for the x-axes, so we suggest that clinicians be more cautious when placing landmarks on the x-axis.

Plooji et al.9 asserted that the actual 3-D distance or difference between two landmarks remains unclear if the landmark is defined separately on the x, y, and z coordinates, thus these landmarks should accurately be identified even in the 3-D mode. They evaluated the actual distance between two similar landmarks and calculated the mean differences, finding that this distance was less than 0.5 mm for 76% of the first examiner's recordings, 92% of the second examiner's recordings, and 80% of the interexaminer recordings. Contrarily, we found that one landmark (labiale superior) had an intraexaminer marking difference less than 0.5 mm. The other 18 landmarks had an intraexaminer difference less than 1 mm but higher than 0.5 mm. The different results should be addressed, especially since the same scanning system was used in both studies. The only valid explanation may be the experience of the examiners. If this assumption is correct, there is a learning curve for 3-D tracings, and experience is very important for correct 3-D landmark positioning.

Toma et al.12 found lower intra- and interexaminer reproducibility comparable with present results, but they assumed that landmarks have clinically acceptable reproducibility if the difference is less than 1 mm. Similar to our study, they found that labiale superior was the most reproducible landmark.

Familiarity with the software program is also likely to play a part in the examiners' ability to place landmarks.10 In addition to training in the use of the program, the examiners were experienced in using it and able to manipulate the images with full 3-D control. This skill is also thought to improve the reproducibility of landmark placement in this study.

Toma et al.12 reported that poorly reproducible landmarks were related to the eyes, and this was linked to acquisition of the image using a laser-based system. We did not encounter such a finding using the stereophotogrammetric scanning system, which may be account for the negative side of the laser-based systems.

Plooji et al.9 stated that the quality of 3-D photographs has improved, especially at the levels of the nasal base, ears, and hairline. With the advances in 3-D imaging and the software we used, we can say that landmark identification is becoming more accurate and that those used in this study showed high levels of reproducibility and accuracy in analyzing facial soft tissue on stereophotogrammetric images. However, clinicians should remember that some points, especially those located on gently curved slopes are more prone to error in landmark localization.

CONCLUSIONS

• All soft tissue landmarks were shown to have high reproducibility and reliability, both in inter- and intraexaminer comparison.

• Soft tissue landmarks used in this study were shown to differ less than 1 mm between repeated markings.