Accuracy of a smartphone-based orthodontic treatment–monitoring application: A pilot study

Abstract

Objectives:

To investigate the reliability and accuracy of a monitoring system in patients undergoing orthodontic treatment with a rapid maxillary expander. Specifically, the amount of tooth movement calculated by the software was compared with the actual measurements taken on plaster models obtained during an in-office visit.

Materials and Methods:

Patients took intraoral video scans using the monitoring software's smartphone application (Dental Monitoring, Paris, France), immediately followed by impressions for plaster models. Intercanine and intermolar width measurements were calculated by the software and compared with those made on the plaster models. Data were analyzed using two one-sided t-tests for equivalence with equivalence bounds of ±0.5 mm. The significance level was set at .05.

Results:

Thirty sets of measurements were compared. The intercanine and intermolar measurement differences were on average 0.17 mm and −0.02 mm, respectively, and were deemed equivalent.

Conclusions:

The monitoring software seems to provide an accurate assessment of linear tooth movements.

INTRODUCTION

Digital technology has paved the way for a paradigm shift in multiple industries including retail, real estate, entertainment, education, and health care. The ability to gather, analyze, and disseminate large quantities of data rapidly through digital networks has revolutionized the world.1 Orthodontists are eager to incorporate new discoveries and inventions into their practices. Today, there is a growing segment of patients who are seeking to undergo orthodontic treatment with less frequent in-office visits while allowing the specialist to maintain control over their treatment progress.

Orthodontic treatment–monitoring software and other biometric devices are examples of technologies that can significantly improve the patient experience. Remote monitoring technology would be especially applicable to procedures in which treatment progress could be tracked with virtual checkups to supplement chairside appointments. For example, the number of office visits and travel time could be significantly reduced when evaluating initial leveling and alignment, correction with maxillary expanders, monitoring oral hygiene status, and checking cooperation with elastic wear. In addition, many emergencies such as broken brackets or power chains could be detected at the time of the event.

Telecommunication technology was found to be particularly useful in orthodontics because minor emergencies such as discomfort due to an appliance or elastomeric ligature displacement could usually be solved at home without an office visit.2 Another study3 that assessed the validity of a teledentistry system for initial orthodontic examinations reported that screening and accepting orthodontic referrals based on clinical photographs was comparable with in-office visits in the clinical decision-making process. Digital telecommunication systems were found to decrease the cost while improving the delivery and management of health care.3–5

With the routine use of intraoral scanners and digital imaging in dentistry, clinicians can take advantage of digital models and photographs to analyze and review cases and to communicate among doctors, patients, and laboratories. Recently, a monitoring technology that combines mobile phone applications with artificial intelligence has become available for orthodontists to follow their patients' treatment remotely.6 The orthodontic monitoring software allows tracking of tooth movement using the images of scanning videos taken by the patient using specific cheek retractors through a smartphone application (Android, iOS). The doctor's interface with the monitoring software makes it possible to monitor treatment progress in real time between office visits (Figure 1). It is also possible to request a prompt when a preset objective has been achieved, such as a specific amount of space opening or space closure. The doctor may receive an alert if a problem is detected, such as a broken appliance, poor hygiene, or gingival recession. In this way, many appointments requiring simple evaluation may be eliminated, and it is possible that the use of this technology could decrease cost and chairside time while improving the delivery of orthodontic care.6

Figure 1.

Treatment activity graph. Movement of the maxillary (blue) and mandibular (gray) teeth for 16 intervals over 7 months.

Although commercially available and perceived by many clinicians as a valuable tool for the treatment of orthodontic patients, to date there are no studies testing whether any monitoring software is accurate enough to provide quantitative data regarding orthodontic tooth movement. Therefore, the purpose of this study was to investigate the reliability and the accuracy of an orthodontic monitoring software application. Specifically, the intercanine and intermolar widths in orthodontic patients undergoing rapid maxillary expander (RME) treatment were calculated by a commercially available software application and then compared with the measurements made on plaster models produced during in-office visits. This study did not attempt to determine the amount of orthodontic expansion but rather to test the ability of the application to measure intercanine and intermolar widths accurately on digital models produced from the videos of teeth.

MATERIALS AND METHODS

Prior to the study, approval from the Institutional Review Board of Virginia Commonwealth University (VCU) and consent forms were obtained. Subjects (n = 12) between the ages of 10 and 17 years (four boys and eight girls) were recruited among VCU Department of Orthodontics patients seeking treatment that required maxillary expansion using the following selection criteria: (1) good overall health, (2) no craniofacial syndrome, (3) maxillary first molars and either both canines or both first premolars erupted, and (4) teeth with normal crown morphology.

At the initial appointment, study participants received the specific cheek retractors required by the monitoring software (Dental Monitoring, Paris, France) and were trained to use the smartphone application to take a scanning video of their teeth. To circumvent the possibility of patients forgetting to take their photos at home on the same day as their orthodontic appointment, the training session and all subsequent intraoral imaging procedures were conducted in the clinic using the provider's smartphone (iPhone 6 Plus, Apple Inc, Cupertino, Calif). To simulate the patient using the application outside of the clinic setting (ie, to better mimic a telemedicine setting), the patient performed his or her own video scanning procedure in a separate room, independent from the provider. When taking videos using the smartphone application, there was a target on the phone screen that the patient aligned with the cheek retractors by looking in a mirror. When the correct distance and angle were achieved by the patient, the target area changed from a red to green color, indicating a good position for image acquisition (Figure 2). Immediately after the patient finished the video scan of his or her teeth, the provider took a separate intraoral scan so that the quality of the two video scans could also be compared as part of the study.

Figure 2.

Patient taking the video scan using the smartphone application.

Subsequently, alginate impressions of the maxillary and mandibular arches were taken to produce plaster models (Fujirock, GC America, Alsip, Ill) to serve later as the gold standard to measure the intercanine and intermolar widths. To assess error in the impressions and plaster models, the intercanine width was measured intraorally and on the models. At this appointment, the scanning videos previously taken by both the patient and provider were also uploaded to the monitoring software platform using the smartphone application. The first set of scanned images was then used by the software to construct a stereolithographic (STL) digital model that served as a baseline. Using artificial intelligence technology, the software could then compare future images of the teeth to this baseline model to quantify and monitor tooth movements over time (Figure 3).

Figure 3.

Three-dimensional (3D) matching. Models of the previous tooth positions are superimposed on the current photos to illustrate the tooth movement in 3D.

At the second orthodontic appointment, the RME appliance (Hyrax, Innovatech, Richmond, Va) was delivered, and digital images of the maxillary and mandibular arches were taken with an intraoral scanner (iTero Element, Align Technology, San Jose, Calif). The maxillary arch, mandibular arch, and both arches in occlusion were uploaded to the monitoring software platform in STL format to serve as a baseline. At this time, the patient also completed and uploaded a set of video scans with the application software in the same manner as the first visit. After the video scans were completed, alginate impressions of the maxillary and mandibular arches were taken and casts were made (Fujirock, GC America). The plaster models served as the gold standard to measure movements of the maxillary molars and canines as assessed by the software. To assess error in the impressions and plaster models, the intercanine width was also measured intraorally and on the models by two operators.

The study was carried out during the active phase of maxillary expansion treatment until the amount of the expansion was deemed satisfactory by the treating orthodontist. Patients were scheduled for weekly clinical observations. During these observation visits, the alginate impressions, intraoral intercanine measurements, and a video scan with the smartphone application were repeated as described above. For most subjects, there were two follow-up appointments following RME delivery. No intraoral scans (with the iTero scanner) were taken at the subsequent appointments.

To evaluate the accuracy of the monitoring software in making linear measurements, the intercanine and intermolar distance measurements were made on the plaster models (in-office measurements) using the following method adapted from Yilmaz et al.7 (Figure 4):

Figure 4.

Intercanine (1) and intermolar (2) width measurements. Adapted from Yilmaz et al.7

1. Intercanine distance (millimeters): the straight distance between the cusp tips of the right and left canines or the center of the wear facets in case of attrition.

2. Intermolar distance (millimeters): the straight distance between the tips of the mesiobuccal cusps of the right and left maxillary first molars.7

Statistical Analysis

Two calibrated operators performed the in-office measurements on the plaster models with digital calipers. The repeatability was calculated from measurements of 10 randomly selected plaster models remeasured after an interval of 2 weeks. The intraclass and interclass correlation coefficients (ICC) for repeated measurements for all parameters were near perfect (ICC = .99).

Two one-sided tests for equivalence were performed between the in-office and software measurements using video scans taken both by the patient and provider. To compare the intercanine and intermolar width values between in-office and the software measurements, the equivalence bounds were set at ±0.5 mm. The rates of successful video scans between patients and the provider were compared using a chi-square test.

RESULTS

A total of 28 measurements were performed by two operators on plaster models and by the software using the images obtained by the patients. There were 27 measurements carried out by two operators and the software using images obtained by the provider. In terms of photo quality, 6% of the photos taken by patients were deemed by the software to be insufficient for processing (n = 2 out of 30) compared with 4% of the photos taken by the provider (n = 1 out of 28). There was no statistically significant difference in the number of the video scans rejected by the software when the images were taken by the patient or the provider (P = .61).

The intercanine and intermolar width values calculated by the software using both the video scans taken by the patients and by the provider were found to be equivalent within ±0.5 mm to those calculated by direct measurements on the plaster models (Table 1). The Bland-Altman plots to demonstrate the average differences between the sets of measurements between the in-office measurements on plaster models and the software calculations are given in Figures 5 and 6.

Table 1.

Average Difference and Equivalence Bounds for Intercanine and Intermolar Widths and Arch Depth Measurements Between In-Office and Software Measurements Using Patient or Provider Video Scans

	Average Difference	90% Confidence Limits	Equivalent (±0.5 mm)
Patient video scans
Intercanine width	0.17	0.00 to 0.34	Equivalent
Intermolar width	−0.02	−0.26 to 0.29	Equivalent
Provider video scans
Intercanine width	0.18	0.00 to 0.36	Equivalent
Intermolar width	0.10	−0.14 to 0.34	Equivalent

Figure 5.

(a) Bland-Altman plot showing differences in intercanine (IC) widths between in-office measurements and software calculations from patient video scans. (b) Bland-Altman plot showing differences in intercanine widths between in-office measurements and software calculations from provider video scans.

Figure 6.

(a) Bland-Altman plot showing differences in intermolar (IM) widths between in-office and software calculations from patient video scans. (b) Bland-Altman plot showing differences in intermolar widths between in-office and software calculations from provider video scans.

The average absolute differences between the manual measures and the application using images produced from patient and provider video scans are given in Table 2.

Table 2.

Average Absolute Difference Between Patient Video Scans and Provider Video Scans

	Average Absolute Difference	SD	Min	Max
Patient video scans
Intercanine width	0.48	0.30	0.02	1.17
Intermolar width	0.68	0.56	0.02	2.65
Provider video scans
Intercanine width	0.48	0.32	0.02	1.44
Intermolar width	0.62	0.41	0.02	1.17

DISCUSSION

In orthodontics, the use of a monitoring application, using a secure web server accredited for health data storage,8 may enable clinicians to closely follow their patients' treatment with fewer office visits, reflecting a more patient-centered approach. By setting notification preferences when a predetermined amount of space opening, space closure, or expansion has been achieved, unnecessary evaluation appointments may be eliminated. Some patients may even be able to receive treatment for an orthodontic emergency from home if the orthodontist can visualize the problem with photos through the application. Also, the monitoring software could analyze tooth movement changes in retention compared with a posttreatment baseline to determine the necessity for in-office retention appointments, thereby maintaining the doctor-patient relationship without having to take up time in the daily schedule.

If video scans uploaded by the patient are of insufficient quality, the software does not build a new digital model or take measurements. Instead, the patient receives an e-mail approximately 1 day later informing him or her of the inadequate photos and prompts a new video scan. At the start of the experiment, several of the video scans were of insufficient quality for processing. However, the quality of the patients' photographic exams appeared to improve over the study period, and this may be attributed to better patient training toward the end of the study as the investigator gained experience with the software. Therefore, if a clinician decides to implement monitoring software into his or her practice, it may be beneficial to assign the duty of teaching patients to use the application to one designated employee. Also, while study participants ranged in age from 10 to 17 years, it is possible that younger patients might not be able to take videos of sufficient quality.

According to estimates provided by the manufacturer of the software used in this study, the error margin is approximately 0.5° for tip, torque, and rotation. For linear measurements, the error is estimated to be about 0.05 mm in the anterior and 0.07 mm in the posterior regions. These values are less than those found in this study. Since the current study evaluated the parameters related to palatal expansion, the software measurements were made on the occlusal view of the palate. The difference between the in-office and the software measurements was slightly greater for the intermolar width than for the intercanine width, supporting the company's disclosure of greater error of tooth movement measures in the posterior segment.

Each video scan consisted of three sets of images taken by the patients with the cheek retractors in place. The first two were similar with the patient's head turning side to side to capture the anterior teeth and the buccal segments from the facial view, first with the teeth in occlusion and then with the teeth slightly apart. The third set of images captured the occlusal views of the arches while changing the angle of the smartphone camera with the patient's head moving up and down with the jaws wide open. Therefore, a likely reason for a higher measurement error with the software could be due to angle and limited view of the images taken of the posterior teeth.

Measurements of arch dimensions were made on plaster models because they are still considered to be the gold standard, although digital models may be reaching a tipping point in replacing plaster models as the gold standard. Multiple studies have confirmed the accuracy of measurements taken on digital models, particularly intra-arch measurements.9–11 However, some studies have found small but statistically significant differences between measurements taken on plaster and digital models.12,13 No studies have examined the accuracy of digital models with an appliance in place, such as an RME. Therefore, in this study, plaster models with the appliances were used to make the measurements.

While plaster models are considered to be the true reproduction of the dentition, there is inevitably some degree of inaccuracy.10 It has been well documented that alginate shrinkage is time dependent.14,15 To minimize distortion, impressions were made and poured immediately by one operator. However, no previous studies investigated the effect of impressing over an RME or other appliance on alginate distortion, introducing a variable of unknown significance. To eliminate the inaccuracies caused by model-making procedures, direct intraoral measurements could have been made and compared with the calculation made by the software. However, due to the visual limitations of working in the oral cavity, this method could also be problematic and would not have produced a hard record of the actual changes.

The intercanine and intermolar width measurements made on the models showed strong agreement between the two operators. The equivalence bounds of ±0.5 mm were chosen because, according to the cast-grading system of the American Board of Orthodontics, a discrepancy of this magnitude is not considered to be clinically relevant.16 However, the distances for overbite and overjet, for example, are far shorter than the cross-arch measurements used in the current study, so using equivalence bounds of ±0.5 mm may be stringent beyond what is clinically relevant.

It is likely that the differences between actual and software-reported distances found in this study were not clinically significant, but clinicians should interpret their relevance. In practice, orthodontists never estimate the amount of expansion within tenths of millimeters. Measuring devices such as a periodontal probe or ruler may occasionally be used to aid in clinical decision making, but they are typically accurate to a half-millimeter at best. More often, an orthodontist relies on visual inspection when evaluating the amount of tooth movement required and achieved. Thus, with average absolute differences of less than 1 mm for all parameters, the accuracy of the monitoring software's measurements was considered to be sufficient for use in clinical practice. In addition to using the software for tooth movement evaluation, the oral hygiene status and appliance breakage could be closely monitored to increase provider-patient feedback without the need for additional office visits.

This pilot study had several limitations. Only two linear measurements were considered, so future studies should consider additional parameters to determine intrusion, extrusion, and torque of the teeth. In addition, the sample size was small. A sample size calculation was not performed prior to the study because of the lack of pilot data. However, the difference observed was assessed for clinical relevance and deemed to be acceptable.

CONCLUSIONS

• In-office and software measurements of the intercanine and intermolar widths are equivalent within 0.5 mm.

• There is insufficient evidence of a difference in image quality when the intraoral video scans are taken by patients vs by the clinician.

• Provided the quality of the video scans are acceptable, the use of monitoring software can be reliable for making clinical decisions.