A comparison between stereophotogrammetry and smartphone structured light technology for three-dimensional face scanning

Abstract

Objectives

To compare three-dimensional facial scans obtained by stereophotogrammetry with two different applications for smartphone supporting the TrueDepth system, a structured light technology.

Materials and Methods

Facial scans of 40 different subjects were acquired with three different systems. The 3dMDtrio Stereophotogrammetry System (3dMD, Atlanta, Ga) was compared with a smartphone (iPhone Xs; Apple, Cupertino, Calif) equipped with the Bellus3D Face Application (version 1.6.11; Bellus3D Inc, Campbell, Calif) or Capture (version 1.2.5; Standard Cyborg Inc, San Francisco, Calif). Times of image acquisition and elaboration were recorded. The surface-to-surface deviation and the distance between 18 landmarks from 3dMD reference images to those acquired with Bellus3D or Capture were measured.

Results

Capturing and processing times with the smartphone applications were considerably longer than with the 3dMD system. The surface-to-surface deviation analysis between the Bellus3D and 3dMD showed an overlap percentage of 80.01% ± 5.92% and 56.62% ± 7.65% within the ranges of 1 mm and 0.5 mm discrepancy, respectively. Images from Capture showed an overlap percentage of 81.40% ± 9.59% and 56.45% ± 11.62% within the ranges of 1 mm and 0.5 mm, respectively.

Conclusions

The face image acquisition with the 3dMD device is fast and accurate, but bulky and expensive. The new smartphone applications combined with the TrueDepth sensors show promising results. They need more accuracy from the operator and more compliance from the patient because of the increased acquisition time. Their greatest advantages are related to cost and portability.

INTRODUCTION

Facial analysis is a fundamental aspect in orthodontic diagnosis.1,2 Today, one of the most commonly used technologies for three-dimensional (3D) facial surface imaging is digital stereophotogrammetry. This system offers a variety of advantages over other facial analysis methods; for example, it is minimal invasive, radiation free, fast to capture images (1.5 milliseconds), and easy to use. Thus, the system is considered the gold standard for 3D face analysis.3–5 Its high accuracy has been quantified by various authors, such as Ayoub et al.,6,7 Littlefield et al.,8 and Khambay et al.,9 who reported accuracy between 0.2 and 0.5 mm to identify landmarks.

Despite this, stereophotogrammetry systems show some limitations that diminish their popularity in clinical routine. The most important is the large footprint of the device, consisting of the entire system with multiple cameras. In addition, stereophotogrammetry equipment is expensive and requires frequent calibration.10 To overcome these disadvantages, recently published experimental studies by Elbashti et al.11 and Amornvit et al.12 proposed the use of smartphones as an alternative to photogrammetry for facial soft tissue imaging. Current smartphones easily provide 3D data, including facial scans. Benefits that could be leveraged by the use of smartphones in clinical practice are the possibility to obtain 3D scans easily and accurately in a few seconds, acceleration of patient acceptance of the treatment plan, easy processing and analysis of the face in virtual format, and simplified collaboration between laboratories and specialists.

The purpose of this study was to compare 3D facial scans obtained by gold standard stereophotogrammetry with two different applications for the smartphone supporting the TrueDepth system.

MATERIALS AND METHODS

A total of 40 adult volunteers (27 women and 13 men; average age, 25.5 ± 2.6 years) were recruited from among students and postgraduate students of Catholic University of the Sacred Heart in Rome between December 2019 and February 2020. Exclusion criteria were systemic diseases, facial deformations and excessive wrinkling, presence of a beard, and history of orthognathic surgical treatment. Each participant signed written informed consent. The study was approved by the ethics committee of the Catholic University of the Sacred Heart in Rome (protocol no. 47734/19; identification no. 1945).

Design of the Study

Three-dimensional face scans of each participant were captured with the following three different methods: stereophotogrammetry (3dMDtrio System; 3dMD, Atlanta, Ga) and two applications for the smartphone, taking advantage of the TrueDepth technology (iPhone Xs; Apple, Cupertino, Calif)—the Bellus3D Face Application (version 1.6.11; Bellus3D Inc, Campbell, Calif) or Capture (version 1.2.5; Standard Cyborg Inc, San Francisco, Calif).13 The 3dMD system was chosen among the stereophotogrammetry methods because it was the most validated and reproducible.3,14 For each patient, the 3dMD vs Bellus3D image and the 3dMD vs Capture image were compared. All scans were performed by the same operator, and time for scanning and image elaboration was recorded by a second operator. All images were acquired in the same room dedicated to clinical photography of orthodontic patients. Earrings, necklaces, hats, glasses, and other accessories that could interfere with the capture area were removed from the patients before photography. Volunteers were also asked to tie their hair and wear a hairband to fully expose the facial skin, including the forehead and ears, as hair complicates the analysis and evaluation of results.15 Patients were asked to wear no make-up.

Acquisitions

Participants were placed on a height-adjustable stool. They were asked to maintain a neutral facial expression and to keep their head still in a natural head position (NHP).16 The three images of the same patient were acquired on the same day in rapid succession. The acquisition methods were as follows:

• 3dMD system: The volunteer was positioned in front of the device, adjusting the height of the seat and centering the face on the screen with the aid of a webcam and grid-centering system on the computer monitor. A photograph was then acquired.

• Bellus3D app: The smartphone was fixed on a tripod in front of the patient at a distance of 30 cm. The participant was asked to rotate his or her head while maintaining NHP, with specific directions and speed as indicated by the graphic interface and voice instructions.

• Capture app: The volunteer was asked to remain motionless. Starting the scan with the internal camera facing the participant, the smartphone was moved by the operator to capture all aspects with a “cross” movement, from the right to left, from back to the center, and to the top and bottom of the face.

Mesh Comparison

All images were exported as a VRML (Virtual Reality Modeling Language) file with the .wrl extension files and elaborated using Geomagic Control 2014 reverse engineering software (Geomagic; 3D Systems, Rock Hill, S.C.). The first step was to cut the face out of the photographs, eliminating all external elements (Figure 1). After this step, the workflow was split into two similar paths. The first path analyzed the differences of two corresponding scans (of the same individual) obtained with the 3dMD and Capture systems, whereas the 3dMD and Bellus3D images were compared in the second path. The surfaces were previously oriented according to Verhoeven et al.14 The 3dMD scan was set as the reference the Capture scan as “Test,” and both surfaces were selected in full. The two images were then aligned by applying the “optimized alignment” tool using the “fine alignment” method7 to obtain the “3D comparison.”17 A superimposition of the two surfaces was obtained on which a color map, as depicted with shades ranging from red to blue, indicated the extremes of the deviation values (respectively, from the most positive to the most negative; Figure 2). The same procedure was applied to compare Bellus3D with the 3dMD images.

Figure 1.

3D scans of the face obtained with (A) 3dMD, (B) Bellus3D, and (C) Capture, following the “cleaning” phase. Only the face is kept, eliminating all confounding elements, such as hair, ears, neck, and shoulders.

Figure 2.

Superimposition of the scans obtained by 3dMD and Capture of the same subject. A color map with shades ranging from red to blue indicates deviation values.

Measurements

Two different times were recorded. The scan time represented the time it took for the device to capture the patient's facial surface, whereas the processing time was the time required by the mobile application or software to generate the 3D model until exported to the computer. Surface-to-surface deviation and point-to-point deviation between 3dMD and the two applications were measured. For analysis of the surface-to-surface distance between the two models, the following two tolerance ranges were initially selected: (1) 1-mm discrepancy, indicated in the literature as the maximum acceptable value for use in the clinical routine of a face scanner,18,19 and (2) 0.5-mm discrepancy to better define the accuracy of the images. With the “3D comparison” function, the percentage of the surface of the two application models included in each comparison was calculated within the two tolerance ranges from the surface of the 3dMD model. To calculate the point-to point distance, 18 landmarks for cephalometric measurements, according to Farkas et al.,20 Toma et al.,21 and Plooij et al.22 were placed on the surface of the superimposed images (Table 1). Of these points, the software analyzed the deviation values along the three orientation axes (x, y, and z) and the mean deviation value (Figure 3).

Table 1.

Landmark Definitions

Name	Abbreviation	Definition
Glabella	G	Most prominent midpoint between the eyebrows
Nasion	N	Deepest point of the nasal bridge
Exocanthiona	Ex	Outer commissure of the eye fissure
Endocanthiona	En	Inner commissure of the eye fissure
Pronasale	Prn	Most protruded point of the apex nasi
Alarea	Al	Most lateral point on nasal alar contour
Labiale superius	Ls	Midpoint of the upper vermillion line
Stomion	St	Contact point between upper and lower lips on the sagittal midline
Labiale inferius	Li	Midpoint of the lower vermilion line
Cheiliona	Ch	Most lateral point of the labial commissure
Pogonion	Pog	Most prominent point of the chin on the sagittal midline
Menton	Me	Most inferior point of the chin on the sagittal midline
Goniona	Go	Most lateral point of the gonial angle contour

^aIndicates bilateral landmarks.

Figure 3.

Landmarks placed on the surface after the overlap between the 3dMD scan and the Capture scan. For each landmark, the deviation values along the three axes (x, y, and z) of the space and the mean deviation value are shown.

Error Method

Two weeks after the first set of measurements, virtual models of ten patients were randomly chosen and a new set of geometric coordinates of cephalometric landmarks was developed by the same operator and, additionally, by a second operator to assess intraoperator and interoperator reliability. Therefore, the intraclass correlation coefficients (ICCs) for intraoperator and interoperator reproducibility were calculated.

Statistical Analysis

All data and measurements were analyzed using SPSS statistics software (IBM Company, Chicago, Ill). The Kolmogorov-Smirnov test was used to evaluate normality of the data distribution. Because all data were found to be distributed normally, parametric tests were used to compare the results. In detail, Student's t-test was used to compare point-to-point distances of the cephalometric points obtained by comparing each application with the 3dMD system. Student's t-test was also used to compare surface-to-surface distances, expressed by a matching percentage, obtained from the same comparison. A P value <.05 was established as statistically significant. The ICC was used to evaluate the intraoperator and interoperator reproducibility of the geometric coordinates x, y, and z in the identification of the cephalometric points selected for this study. For all measurements, the Dahlberg formula was used to calculate the systematic error.

RESULTS

Intraoperator reproducibility showed ICC values between 0.982 and 0.999. In addition, the reproducibility between operators showed ICC values between 0.944 and 0.999 (Table 2). The systematic error measured using the Dahlberg formula was remarkably low. Results for the mean and standard deviation (SD) of scan, processing, and total time for each system used in this study are shown in Table 3. The 3dMD system had a fixed scan time of 1.5 milliseconds. The Bellus3D and Capture applications required an average time of 20.28 seconds and 40.34 seconds, respectively. In addition, the 3dMD system usually took 20.58 seconds to process a 3D model of the patient's face and to export it to the PC, whereas Bellus3D and Capture required 55.47 seconds and 53.57 seconds, respectively. For the surface-to-surface distance analysis between Bellus3D and 3dMD, the average percentage of surface overlap within deviation range A was 80.01 ± 5.92. The percentage of surface overlap within range B was 56.62 ± 7.65. For the matching measurements between the Capture and 3dMD surfaces, the percentages of surface overlap within ranges A and B were 81.40 ± 9.59 and 56.45 ± 11.62, respectively. Subsequently, the average percentage values obtained from the comparisons of the two applications with 3dMD were compared with each other, showing no significant differences (P > .05) between Bellus3D and Capture (Table 4).

Table 2.

Results Related to the ICC That Highlight the Intraoperator and Interoperator Reproducibility in the Placement of Landmarks on the Scanned Surfaces

	ICC
	Intraoperator			Interoperator
Landmark	x	y	z	x	y	z
G	0.998	0.998	1.000	0.998	0.986	0.999
N	0.999	0.995	0.999	0.999	0.983	0.999
Ex R	0.999	0.999	1.000	0.997	0.996	0.998
Ex L	0.998	0.999	1.000	0.988	0.999	1.000
En R	0.998	0.999	1.000	0.990	0.997	0.999
En L	0.999	0.999	1.000	0.997	0.999	1.000
Prn	0.999	0.997	1.000	0.998	0.991	1.000
Al R	0.998	0.999	0.999	0.998	0.999	0.998
Al L	0.997	0.997	0.998	0.996	0.996	0.996
Ls	0.999	0.998	1.000	0.997	0.984	1.000
St	0.998	0.998	1.000	0.995	0.997	1.000
Li	0.999	0.998	1.000	0.995	0.996	1.000
Ch R	0.995	0.999	0.999	0.997	0.999	0.999
Ch L	0.999	0.999	1.000	0.991	0.998	1.000
Pog	0.994	0.999	1.000	0.980	0.988	1.000
Me	0.998	0.999	0.999	0.971	0.981	0.987
Go R	0.994	0.991	0.996	0.973	0.966	0.991
Go L	0.991	0.982	0.996	0.983	0.944	0.992

Table 3.

Scanning, Processing, and Total Time of the Three Different Software Applications Used in This Study

	Timing
	Scan		Elaboration		Total
Application	Average	SD	Average	SD	Average	SD
3dMD	0.0015a	0.00	20.58	1.30	20.59	1.30
Bellus3D	20.28	0.56	55.47	3.58	75.75	3.67
Capture	40.34	2.99	53.57	4.19	93.91	5.56

^a3dMD has a standard scanning time of 1.5 milliseconds.

Table 4.

Surface-to-Surface Distance Analysis Results Expressed as an Average Percentage of Match for the Tolerance Ranges A (1 mm) and B (0.5 mm)

Range	Bellus3D		Capture		P Value*
	Average	SD	Average	SD
A	80.01	5.92	81.40	9.59	.439
B	56.62	7.65	56.45	11.62	.938

^*Level of significance: P < .05.

Results for the point-to-point deviation analysis are reported in Table 5. In the comparison between Bellus3D and 3dMD, an average distance of less than 1 mm was found for all reference points except for Ex L, En R, En L, Ch R, and Ch L. Comparing Capture with 3dMD, an average distance of less than 1 mm for most landmarks was found. The reference points with distances exceeding 1 mm were En R, En L, Prn, St, Ch R, Ch L, Go R, and Go L. Comparing the results obtained from the two applications, a statistically significant difference (P < 0.05) was observed for the following 11 landmarks: G, N, En L, Prn, Al R, Al L, St, Ch R, Pog, Me, and Go R. The color map obtained from the 3D surface comparison showed that the most accurate areas were the forehead, chin, and cheek, whereas the least accurate were the mouth, lips, and eyes (Figures 2 and 3).

Table 5.

Results of the Analysis of the Point-to-Point Distance of the Facial Landmarks Selected for This Study

Landmark	Distance (mm)				P Value*
	Bellus3D		Capture
	Average	SD	Average	SD
G	0.2310	0.1407	0.4172	0.3721	.004
N	0.2202	0.1800	0.3247	0.2238	.024
Ex R	0.9709	0.6150	0.8089	0.6380	.464
Ex L	1.1517	0.5146	0.8578	0.6270	.251
En R	1.2487	0.6171	1.3567	0.6919	.162
En L	1.0626	0.6420	1.2742	0.6975	.025
Prn	0.6735	0.3122	1.0264	0.4639	.000
Al R	0.3934	0.4230	0.8729	0.7684	.001
Al L	0.2420	0.1592	0.8373	0.6459	.000
Ls	0.8732	0.7051	0.9216	1.0171	.805
St	0.9967	0.6488	1.6065	1.4271	.016
Li	0.8505	0.5404	0.9921	0.8557	.379
Ch R	1.2458	0.8645	1.7138	1.1282	.041
Ch L	1.2491	0.8040	1.6840	1.3661	.087
Pog	0.7399	0.3924	0.3862	0.2891	.000
Me	0.3860	0.3324	0.6032	0.5039	.026
Go R	0.7803	0.7818	1.4954	1.6233	.014
Go L	0.7656	0.6981	1.0652	1.0752	.143

^*Level of significance: P < .05.

DISCUSSION

This study compared two smartphone applications for face scanning based on TrueDepth technology with the 3dMD system. The results showed that the Bellus3D and Capture applications were comparable with 3dMD in reproducing 3D facial surfaces when considering the error range of 1 mm, as suggested acceptable in the literature for a valid facial scan. The accuracy of smartphones decreased if the error range of acceptability was set to 0.5 mm. The scan time was notably different among the three tools used; 3dMD showed a standard scan time of 1.5 milliseconds, which allowed immediate scanning of the facial surface without motion artifacts. In contrast, the two applications required a longer time (20.28 seconds and 40.34 seconds). This could represent a major source for errors and artifacts. Indeed, the accuracy of a 3D scanner is affected by the length of scanning, probably as a result of involuntary facial muscle movement and deviation from a purely rotational movement of the head or the manual device around the face.12,23

The processing time for the 3D images (time used by the software to elaborate and to merge the acquired meshes together) was remarkably different among the three methods. The 3dMD software required a processing time of 20.58 seconds, whereas Bellus3D and Capture required 55.47 seconds and 53.57 seconds, respectively. The difference was probably attributed to the higher number of images registered during the scanning process.

Analyzing the color map obtained from the 3D surface comparison, the most accurate areas were the flat areas of the face (forehead, chin, and cheek). In contrast, the least accurate areas were those rich in curvatures (mouth, lips, and eyes; Figures 2 and 3), suggesting the inferior ability of the smartphone to capture facial aspects of complex anatomy.

The resolution and the esthetic rendering of the images obtained through smartphone scans were significantly lower than the images resulting from stereophotogrammetry. In particular, the scans from Capture appeared less realistic and “grainy” (Figure 1). However, the appearance did not affect the accuracy of the 3D shape of the scanned surfaces.

In the surface-to-surface analysis, no statistically significant differences were found between the results of the two smartphone applications, although the Bellus3D SD values were much lower, showing greater precision and reproducibility. For point-to-point analysis, Bellus3D was significantly more accurate, and a statistically significant difference was found for 11 of 18 landmarks (P < .05). This could have been attributed to the higher scan quality of Bellus3D in terms of the triangular mesh that reproduced the surface. This allowed greater precision in the selection of a specific point on the facial surface (landmark) and led to a higher esthetic rendering of the image.

Limitations

One of the limitations of this study was that movement of the operator and patient during image capture could have caused inaccuracies and artifacts. The Bellus3D application allowed positioning of the smartphone on a tripod to avoid errors caused by operator movement, but required movement of the participant's head instead. However, Capture did not require movement of the patient's head but, rather, a skilled operator to capture the image.

CONCLUSIONS

• Through smartphone applications based on TrueDepth technology, it is possible to obtain 3D facial models with a good level of accuracy to be used for various purposes in orthodontic clinical practice, such as preevaluation, treatment simulation, anthropometry, and esthetic analysis. However, 3dMD remains the most accurate and fast system for detailed measurements and can be used in all clinical situations, even with children or uncooperative patients.

• No statistically significant differences were found when comparing 3D surfaces between the two smartphone applications. However, Bellus3D showed significantly more accurate results for the analysis of landmarks. In addition, Bellus3D features improved ease of use, reliability, and accuracy and shorter scan times, allowing the operator to reduce the movement of patients during the scan process.

• Stereophotogrammetric systems are expensive and nonportable devices. In contrast, the smartphone applications are portable and less expensive, offering every practitioner the ability to introduce the benefits of face scanning into his or her clinical routine.