The reliability and reproducibility of an Android cephalometric smartphone application in comparison with the conventional method

Abstract

Objectives

To assess the reliability and reproducibility of linear and angular measurements of the cephalometric smartphone Android application OneCeph in comparison with the conventional method.

Materials and Methods

A total number of 22 landmarks were registered, and 26 skeletal and dental cephalometric parameters were measured on 30 pretreatment cephalograms. The measurements for both digital (OneCeph) and conventional tracings were performed twice with a 4-week interval. The reliability (intraexaminer error) was evaluated by using the Pearson correlation coefficient. The variation in measurements between the tracing techniques (reproducibility) was determined by paired t-test.

Results

The Pearson correlation coefficients of all cephalometric measurements for each tracing technique were ≥ 0.95. Significant differences between the two tracing techniques were detected in five measurements (SNB angle, N I to Pog linear measurement, U1-Apoint linear measurement, U lip to S line, and nasiolabial angle; P < .05).

Conclusions

Using 26 measurements to compare both tracing methods, all mean differences between the digital (OneCeph) and conventional methods were below 1 degree/1 mm, indicating that differences between the tracing methods were clinically insignificant. The U1-A point measurement was an exception for the digital method (OneCeph) with a clinically significant difference of 1.25 mm (P < .01); the difference was a result of wrongly measuring the distance from the A line to the incisor edge of the upper central incisor rather than the facial surface of the upper incisor. This leads to the conclusion that both tracing methods were reliable for daily clinical practice.

INTRODUCTION

Cephalometric radiographs are important tools for proper orthodontic diagnosis and treatment planning. Traditionally, cephalometric analysis is done manually. However, this technique is time consuming and it is subject to measurement and magnification errors.^1,2 Digital computer-assisted cephalometric programs can save time and eliminate measurement and magnification errors because these measurements are done automatically.³ Despite the advantages, however, computer-assisted cephalometric programs are still relatively expensive, and they need a computer, which is less portable in comparison with smartphones.

Recently, the usage of smartphones has spread in many fields, including dentistry. The ability to download custom-built software applications (apps) has created opportunities for orthodontists to use this technology in orthodontic treatment.⁴ A new released app for cephalometric analysis called “OneCeph” was launched in the Google Play store in 2016. The application was developed by Prof. Dr. M. Pavan Kumar, Kamineni Institute of Dental Sciences, Narketpally, Telangana, India. This free app has the ability to produce several cephalometric analyses in a short period.⁵ However, no previous study has verified the reliability (intraexaminer error) and reproducibility (agreement between two measurements of different methods) of this app. Thus, the aim of the present study was to determine the reliability and reproducibility of the cephalometric smartphone app OneCeph and compare it with the conventional hand tracing method.

MATERIALS AND METHODS

This study was conducted on 30 pretreatment lateral cephalograms. The cephalograms were selected from the digital archive of the Department of Orthodontics, School of Dentistry, İstanbul Yeni Yüzyıl University. The study was approved by the ethical committee of İstanbul Yeni Yüzyıl University. The inclusion criteria for radiographs were pretreatment cephalograms of individuals who were seeking orthodontic treatment, no pathology, no artifacts, and no gross craniofacial asymmetry and the dentition was in maximum intercuspation with the lips in a relaxed position.

All of the selected lateral cephalometric radiographs were taken by the same technician and cephalostat. The X-ray images were obtained digitally (PCH-2500 Digital Cephalometric Unit, Vatech, Hwaseong, Korea). The patients' heads were positioned with the Frankfort horizontal plane parallel to the floor, and the midsagittal plane was at a 90-degree angle to the X-ray beam. The radiographs were taken in centric occlusion and a light lip seal.

To perform manual hand tracing, the digital cephalometric images were printed out at a scale of 1:1. The radiographs were traced in daylight. The tracing was done using a 0.3 mm hard black (HB) lead pencil. Landmarks were identified by a single point. In case of superimposed bilateral anatomical structures and double images, the mid-point was chosen. After identifying the 22 selected landmarks and 7 planes (Figure 1), 26 linear and angular parameters (Table 1) were measured manually and digitally.

Figure 1.

Cephalometric landmarks used in the study: (1) sella (S), (2) nasion (N), (3) anterior nasal spine (ANS), (4) posterior nasal spine (PNS), (5) A point (A), (6) incisor superius (Is), (7) incisor inferius (Ii), (8) B point (B), (9) pogonion (Pog), (10) gnathion (Gn), (11) menton (Me), (12) gonion (Go), (13) condylon (Cd), (14) articulare (Ar), (15) orbitale (Or), (16) porion (Po), (17) mid-point between molar superioris (Ms) and molar inferioris (Mi), (18) subnasal (Sn), (19) S point (Steiner analysis), (20) labial superius (LS), (21) labial inferius (LI), (22) soft tissue pogonion (Pog'). (A) SN plane, (B) Frankfort plane (Po-Or), (C) maxillary plane (ANS-PNS), (D) bisecting occlusal plane (BOP), (E) mandibular plane (Go-Gn), (F) mandibular plane (Go-Me), (G) mandibular plane (tangent to lower border of mandible).

Table 1.

Cephalometric Measurements

S-N	Linear distance from sella turcica (S) to nasion (N)
S N-Ar	Angle between anterior cranial base (S-N) and articulare (Ar) represents cranial base flexure
SNA	Anteroposterior position of the maxilla relative to the anterior cranial base
N l to A	Linear measurement from nasion perpendicular line to A point
Co-A	Linear distance from condylion to A point, represents the effective mid-facial length
SNB	Anteroposterior position of the mandible relative to the anterior cranial base
N I to Pog	Linear measurement from nasion perpendicular line to pogonion (Pog)
Co-Gn	Linear distance from condylion to the gnathion represents the effective mandibular length
Go-Gn	Linear distance from gonion and gnathion, represents mandibular body length
SN-Go Gn	Angle between sella turcica-nasion (SN) line and the mandibular plane (Go-Gn)
FMPA	Angle between Frankfort (orbital-porion) and mandibular planes
Gonial angle (Ar-Go-Me)	Angle between mandibular plane (Go-Me) and ramal plane (Go-Ar)
ANB	Difference between SNA and SNB angles
Wits	Linear measurement between A point and B point projected onto the bisecting occlusal plane
ANS-Me	Linear distance from anterior nasal spine (ANS) to menton (Me) represents the lower anterior facial height
Jarabak ratio (S-Go/N Me)	Ratio between total posterior and anterior facial heights (sella-gonion and nasion-menton)
U1-A point	A line is constructed through point A parallel to nasion perpendicular and the distance measured to the facial surface of the upper incisor; it relates the upper incisor to the maxilla
LI-A Pog	Distance from the facial surface of the lower incisor to the line drawn through point A and pogonion; it relates the lower incisor to the mandible
IMPA	Angle between long axis of lower central incisor and the mandibular plane (tangent to lower border of mandible)
U1-NA	Angle between nasion-A point (NA) line and the long axis of upper incisor
U1-NA (mm)	Linear measurement from the tip of upper central incisor to NA line
L1-NB	Angle between nasion-B point (NB) line and long axis of lower incisor
L1-NB (mm)	The linear measurement from the tip of lower central incisor to NB line
Nasiolabial angle	Angle between upper lip and base of the nose
UL to S line	Linear measurement from most prominent point of upper lip to Steiner's S line
LL to S line	Linear measurement from most prominent point of lower lip to Steiner's S line

Cephalometric measurements were performed by importing high-quality JPEG images (resolution quality of 96 dpi and dimensions of 2272 × 2045 pixels at a 24-bit depth) of all digital cephalograms to the OneCeph app using a Samsung Galaxy A7 2018 smartphone (Samsung C&T Corporation, Seoul, South Korea). The conventional hand tracing and cephalometric smartphone OneCeph app measurements were performed twice by the same investigator with intervals of 4 weeks between the measurements.

Statistical Methods

Sample size calculation was performed by G*Power 3 and based on expecting a moderate level of correlation for all cephalometric measurements (r² = 0.5). Power analysis showed that a minimum of 29 radiographs were required at power, 0.80; β, 0.20; and α, 0.05.

Data analyses were carried out using SPSS software version 22 (SPSS Inc., Chicago, IL). The mean, standard deviation, and standard error of the difference between the repeated measurements for each method and between the two methods were calculated. The reliability of each method was defined using the Pearson correlation coefficient (r²). The levels of the correlation strength were the following: r² > 0.8 = high (strong), 0.5 ≤ r² ≤ 0.8 = moderate, r² < 0.5 = weak. The reproducibility was determined by comparing the paired measurements through t-tests. The level of statistical significance was set at P< .05.

RESULTS

The reliability (intraexaminer error) of the conventional and digital methods for all measurements was evaluated using Pearson correlation coefficients (r²). Overall, the reliability analysis (Table 2) showed high correlation between repeated measurements of the digital and manual methods. The greatest differences between the first and second tracing trials were 0.82 mm and 0.43 degrees for the digital method and 0.87 mm and 0.90 degrees for the conventional method.

Table 2.

Mean Differences, Standard Deviation and Correlation Coefficient (Intraexaminer Error) for Repeated Measurements of Digital and Conventional Tracing

Measurements	Digital Tracing		Conventional Tracing
	Difference, Mean ± SDa	Correlation Coefficientb	Difference, Mean ± SDa	Correlation Coefficientb
Cranial base
S-N	−0.01 ± 0.86	0.958	0.11 ± 0.81	0.958
S N-Ar	−0.42 ± 1.11	0.984	−0.31 ± 1.21	0.979
Maxilla
SNA	0.07 ± 0.58	0.990	−0.13 ± 0.65	0.986
N I to A	0.29 ± 0.58	0.981	0.14 ± 0.85	0.961
Co-A	−0.33 ± 1.79	0.932	−0.25 ± 1.40	0.957
Mandible
SNB	0.05 ± 0.42	0.994	−0.14 ± 0.46	0.993
N I to Pog	0.40 ± 0.93	0.988	−0.13 ± 1.92	0.949
Co-Gn	−0.82 ± 2.44	0.950	−0.87 ± 1.94	0.964
Go-Gn	0.20 ± 1.06	0.975	−0.08 ± 1.27	0.967
SN-Mn plane (SN-Go Gn)	0.35 ± 0.82	0.994	0.31 ± 1.21	0.986
FMPA	−0.06 ± 0.75	0.994	0.07 ± 0.72	0.994
Gonial angle (Ar-Go-Me)	−0.11 ± 0.98	0.992	0.18 ± 1.14	0.988
Intermaxillary relations
ANB	0.02 ± 0.34	0.993	−0.01 ± 0.55	0.985
Wits	−0.08 ± 0.58	0.991	−0.01 ± 0.63	0.990
ANS-Me	0.10 ± 1.05	0.989	−0.05 ± 0.77	0.993
Jarabak ratio (S-Go/N Me)	−0.33 ± 0.97	0.980	−0.16 ± 0.89	0.983
Dentoalveolar
U1-A point	0.003 ± 0.44	0.987	−0.36 ± 0.85	0.960
LI-A Pog	−0.08 ± 0.26	0.995	−0.26 ± 0.48	0.985
IMPA	−0.11 ± 1.7	0.981	−0.33 ± 1.60	0.981
UI-NA	−0.43 ± 1.33	0.981	0.32 ± 1.28	0.983
UI-NA (mm)	−0.11 ± 0.54	0.977	−0.08 ± 0.72	0.962
LI-NB	−0.30 ± 1.63	0.976	−0.20 ± 1.36	0.982
LI-NB (mm)	−0.12 ± 0.24	0.995	−0.15 ± 0.45	0.982
Soft tissue
Nasiolabial angle	0.41 ± 1.86	0.986	0.90 ± 2.56	0.976
U lip to S line	−0.20 ± 0.33	0.988	−0.09 ± 0.35	0.985
L lip to S line	−0.18 ± 0.26	0.996	−0.21 ± 0.35	0.993

^aSD indicates standard deviation.

^bPearson correlation coefficient (r² > 0.8, strong; 0.5 ≤ r² ≤ 0.8, moderate; r² < 0.5, weak).

The reproducibility (comparison between the digital and the conventional methods) showed statistically significant differences for five measurements (SNB, N I to Pog, U1-A point, U lip to S line, and nasolabial angle; P < .05, P < .01; Table 3). The conventional method had significantly higher means than the digital method for the following measures: SNB (P = .013), N I to Pog (P = .035), U1-A point (P = .001), and U lip to S line (P = .001). The digital method had a significantly higher mean than the conventional method for the nasolabial angle measurements (P = .002). However, the differences between the digital and the conventional methods regarding the rest of the sagittal linear and angular measurements were not statistically significant. The highest differences between sample means were 1.25 mm and 0.77 degrees.

Table 3.

Comparison of Digital and Conventional Tracing Measurements

Measurements	Digital Tracing, Mean ± SD	Conventional Tracing, Mean ± SD	Difference, Mean ± SD	t-Test P Value
Cranial base
S-N	65.09 ± 2.99	64.77 ± 2.83	0.32 ± 0.85	.052
S N-Ar	124.32 ± 6.04	124.28 ± 5.82	0.04 ± 1.00	.829
Maxilla
SNA	80.64 ± 3.97	80.84 ± 3.87	−0.20 ± 0.62	.090
N I to A	0.44 ± 2.87	0.54 ± 3.01	−0.11 ± 0.63	.367
Co-A	81.73 ± 4.59	81.74 ± 4.63	−0.02 ± 0.84	.923
Mandible
SNB	76.83 ± 3.81	77.04 ± 3.83	−0.21 ± 0.44	.013*
N I to Pog	−4.74 ± 6.00	−4.08 ± 6.07	−0.67 ± 1.65	.035*
Co-Gn	105.91 ± 6.84	106.07 ± 7.10	−0.15 ± 1.04	.425
Go-Gn	68.70 ± 4.54	68.62 ± 4.51	0.08 ± 0.66	.515
SN-Mn plane (SN-Go Gn)	34.16 ± 7.07	34.37 ± 6.95	−0.21 ± 0.97	.243
FMPA	27.49 ± 6.61	27.53 ± 6.35	−0.04 ± 0.60	.693
Gonial angle (Ar-Go-Me)	129.08 ± 7.49	128.93 ± 7.46	0.15 ± 0.72	.273
Intermaxillary relations
ANB	3.81 ± 2.95	3.80 ± 2.78	0.01 ± 0.52	.917
Wits	0.99 ± 4.35	1.00 ± 4.24	−0.01 ± 0.46	.905
ANS-Me	62.82 ± 6.50	63.00 ± 6.69	−0.18 ± 0.94	.303
Jarabak ratio (S-Go/N Me)	63.23 ± 4.85	62.98 ± 4.46	0.25 ± 1.04	.197
Dentoalveolar
U1-A point	3.30 ± 2.68	4.55 ± 2.68	−1.25 ± 1.30	.001**
LI-A Pog	2.03 ± 2.78	2.04 ± 2.77	−0.01 ± 0.30	.882
IMPA	94.15 ± 8.52	93.93 ± 8.24	0.22 ± 1.28	.341
UI-NA	22.07 ± 6.79	21.98 ± 6.87	0.08 ± 0.99	.648
UI-NA (mm)	3.08 ± 2.52	3.20 ± 2.26	−0.12 ± 0.73	.373
LI-NB	28.47 ± 7.50	28.38 ± 7.23	0.09 ± 1.16	.674
LI-NB (mm)	4.69 ± 2.21	4.60 ± 2.20	0.09 ± 0.33	.160
Soft tissue
Nasiolabial angle	106.47 ± 11.20	105.70 ± 11.17	0.77 ± 1.23	.002**
U lip to S line	−1.07 ± 2.11	−0.81 ± 2.02	−0.27 ± 0.32	.001**
L lip to S line	0.61 ± 2.84	0.69 ± 2.89	−0.08 ± 0.24	.088

^*P < .05; **P < .01.

DISCUSSION

Lateral cephalograms continue to be an essential part of orthodontic records. Conventionally, cephalometric analysis is done manually. However, the source of errors in conventional tracing could arise from the variation in landmark identification, measurement errors, and magnification errors.^1,6,7 To overcome the disadvantages of manual tracing, computerized cephalometric software products were introduced into clinical orthodontics. This computerized technique requires only digitizing the cephalometric landmarks on the cephalometric X-ray, and the calculation of measurements is done automatically.³ Thus, the digital technique eliminates most potential human errors except for landmark identification errors. Despite all the advantages, it is important to evaluate the reliability and reproducibility of new computerized software and to compare it with conventional tracing. Many studies have examined the reliability and reproducibility of computerized cephalometric software programs that are commercially available; however, as technology evolved, custom-built cephalometric software apps for smartphones have created useful opportunities for clinical practice.⁴

Thus, the aim of this study was to evaluate the reliability and reproducibility of the cephalometric smartphone application OneCeph, which runs on Android systems (Google LLC., Mountain View, Calif), and compare it with the manual tracing method. In examining the reliability and reproducibility of the OneCeph app, the use of cephalometric measurements was prioritized over landmark identification. Santoro et al.⁸ mentioned that any study that aims to evaluate the consistency and reliability of digital cephalometrics shall focus on the sources of errors and the use of measurements rather than landmarks because cephalometric measurements are the final product of the cephalometric tracing procedure and they are the basis of treatment planning. To minimize the errors in the digital technique, the grayscale of the digital images was kept at 24 bits. Ongkosuwito et al.⁹ showed that a grayscale less than 7 bits can lead to uncertainty for the reproducibility of cephalometric measurements.

Erkan et al.¹⁰ inferred that during the assessment of the reliability of computerized cephalometric software, intraexaminer error was much less than the interexaminer error. Therefore, this study was focused on intraexaminer errors (the reliability) for both linear and angular measurements. The reliability analysis (Table 2) showed a high correlation between the repeated measurements of the digital and manual methods (r² > 0.9 for all measurements), indicating that the operator had no difficulty in accurately repeating measurements in each technique. There was good agreement between the current findings and those of previous studies. For instance, Sayinsu et al.¹¹ found that the operator was consistent in repeated measurements, and all intraexaminer correlation coefficients were >0.90. AlBarakati et al. showed a high correlation between repeated measurements (the reliability) of both conventional and digital tracing techniques. In their study, all intraexaminer correlation coefficients for repeated measurements were at least 0.90, except for the maxillary length (ANS-PNS), which was not included in the current study.¹²

The number of studies that investigated the reproducibility of smartphone cephalometric apps are limited in the literature, and the available studies had targeted apps that run mainly on the iPhone (Apple Inc., Cupertino, Calif). Sayar and Kilinc¹³ examined the reproducibility of the CephNinja 3.10 app, which runs on Apple's IPhone operating system (IOS), in comparison with the hand-tracing method. They found that there were significant statistical differences for all measured parameters, but the differences were clinically insignificant. On the other hand, Aksakalli et al.¹⁴ investigated the accuracy of two cephalometric apps, CephNinja 3.3 and SmartCeph Pro 1.1, which run on iPad (Apple Inc.), and they compared those apps with the computerized Dolphin imaging software. The authors concluded that smartphone apps should be developed to provide more accurate results because most of the measurements differed significantly from the Dolphin imaging software. Although OneCeph is a smartphone app, comparison between the digital and conventional methods (the reproducibility) indicated statistically significant differences for only five measurements (SNB, N I to Pog, U1-A point, U lip to S line, and nasiolabial angle; P < .05, P < .01; Table 3). Similar findings were reported in previous studies that compared conventional tracing to digital tracing using software that runs on personal computers. AlBarakati et al.¹² reported significant differences in SNB measurements. A possible explanation for such a difference according to previous studies was that the nasion can be difficult to locate precisely when the nasofrontal suture is not clearly visualized.¹⁵ In addition, the B point is located on a curve, and, thus, it might show slightly greater errors of measurement. Similarly, the S point (U lip to S line) is located on a curve with wide radii, and it might increase the chance of error.¹ Celik et al.¹⁶ and Sayinsu et al.¹¹ reported significant differences in N I to Pog measurements, which might have arisen from the fact that the porion is an inconsistent cephalometric point.¹⁷ Despite the previously reported explanations regarding the differences in SNB, N I to Pog, and U lip to S line measurements between the tracing methods, the intraexaminer reliability in the present study was high for both tracing techniques. This might suggest that the landmark identification for the operator was relatively unchallenging. The inconsistency between conventional and digital measurements (SNB, N I to Pog, and U lip to S) might be related to the operator identifying some cephalometric points slightly differently when projected on a mobile touchscreen, even if they could be repeated consistently within each method. However, it is important to emphasize that the differences were below 1 mm or 0.5 degree; thus, they clinically might be considered insignificant.

The nasolabial angle measurements were inconsistent between the digital and the conventional methods. Sayinsu et al.¹¹ reported a similar result, and they explained the difference by the fact that the nasolabial angle is calculated based on landmarks that are located on curves with wide radii and therefore showing proportionally greater errors.¹ However, in the current study, the significant differences for nasolabial angle measurements between the two methods could have been attributed to the impractical technique for determining the tangent line of the nose in the OneCeph app. The app asks the user to place a point far away from the soft tissue profile of the nose as a part of constructing the tangent line of the nose (Figure 2); thus, the chance of error in correctly determining the tangent line of the nose increases, and the nasolabial angle is affected as a consequence.

Figure 2.

Placement of anterior point far away from soft tissue to construct the tangent line of the nose as part of nasiolabial angle measurement.

The U1-A point measurements were also inconsistent between the digital and conventional methods. The U1-A point measurement is defined as “A line constructed through point A parallel to nasion perpendicular and the distance measured to the facial surface of the upper incisor. It relates the upper incisor to the maxilla.”¹⁸ However, the distance in the OneCeph app is measured incorrectly: to the incisal edge of the maxillary central incisor (Figure 3) rather than the facial surface of the upper incisor.^18,19 Thus, there were significant differences in U1-A point measurements between the two tracing methods.

Figure 3.

For the U1-A point measurement in the OneCeph app, the distance is measured, incorrectly, to the incisal edge of the maxillary central incisor rather than to the facial surface of the upper incisor.

CONCLUSIONS

• Using 26 measurements to compare the two tracing methods, the results showed statistically significant differences for five measurements (SNB, N I to Pog, U1-A point, U lip to S line, and the nasiolabial angle; P < .05, P < .01). However, all mean differences between the digital (OneCeph) and conventional methods were below 1 degree/1 mm, indicating that differences between the tracing methods were clinically insignificant. However, the U1-A point measurement was an exception, with a clinically significant difference of 1.25 mm (P < .01); the difference was a result of the digital app incorrectly calculating the distance from the A line to the incisal edge of the upper central incisor rather than the facial surface of the upper incisor. This leads to the conclusion that both tracing methods were reliable for daily clinical practice.