Faces in 4 dimensions: Why do we care, and why the fourth dimension?

Abstract

The purpose of this article is to describe a dynamic approach for 3-dimensional analyses of facial soft-tissue movements. The method and analysis have numerous applications but, most specifically, are used to assess diagnostic and treatment outcomes of soft-tissue surgery in patients with repaired cleft lip and palate.

The recent focus on smile analysis in orthodontics has heightened the specialty’s desire for dynamic measures to assess facial soft-tissue movements. Currently, the technology exists to record 3-dimensional static facial images, and soon, clinicians will be able to automatically record and quantify animated faces during expressive behaviors in 3 dimensions and add that all-important fourth dimension of time. As part of past and current National Institute of Dental and Craniofacial Research funding (DE013814, DE016964, and DE019742), our research group has developed an approach to objectively quantify 3-dimensional facial movements and measure impairments in facial animations. The animations are mathematically generated based on the patient’s mean movements and are compared with mean, normal noncleft movements. In addition, these mean objective facial movements are independent of the patient’s actual face, where scarring of the upper lip can confound a viewer’s (subjective) assessment of facial movement.¹ A clinician can use the comparative movements to objectively determine the degree of impairment in circumoral movements and the changes with age. This information supplements the clinician’s subjective evaluation of the patient. For the most part, we have used this approach on patients with facial disabilities such as congenital malformations (eg, cleft lip)^2-4 and dentofacial deformities,^5,6 but it is applicable to patients with other neuromuscular and acquired facial disorders (eg, facial trauma).

The importance of the approach lies in the fact that facial expressions are an important form of nonverbal communication that influences our interactions with others. Any disorder that impairs facial soft-tissue movements distorts and produces unesthetic animated behaviors that affect nonverbal communications. For example, early in life, there are indications that, because of the facial disability in babies born with a cleft lip and palate, the quality of mother-infant interactions is affected, and parents of these patients report various psychological problems in their children because of their facial appearance.⁷ The good news is that for the vast majority of orthodontic patients, a dynamic quantification of facial movements might be excessive. However, for patients with these more severe facial disabilities (eg, cleft lip and palate), this technology and analysis will have a positive impact on the clinician’s diagnosis of facial soft-tissue impairment, surgical treatment planning, and assessment of surgical outcomes.

When a child is born with cleft lip and palate (Fig 1, A), the cleft lip is surgically repaired soon after birth (Fig 1, B). Depending on the esthetic outcome of this initial or primary repair, the surgeon might recommend additional lip (revision) surgery later in life (Fig 1, C). The decision for additional lip surgery is heavily influenced by the patient and the parents, and is based mainly on a subjective evaluation of the lips and nose; the dynamic functional capacity of the nasolabial region often is not considered. Studies have shown that subjective assessments are somewhat unreliable.^1,8 The use of a popular subjective scale, the Asher-McDade ordinal scale, demonstrates that, at best, there is moderate reliability among surgeons.^9,10 Our own research using subjective scales demonstrated that surgeons tend to agree with themselves when making the decision for additional lip surgery, but they tend to disagree with each other on this same decision.⁸

Fig 1.

Child with cleft lip and palate: A, at birth; B, after lip repair; C, age 7 years.

The method presented in this article is not based on a subjective scale but on an objective measure. The more important issue with this measure is whether surgeons can use the diagnostic information obtained to modify and tailor their surgeries to meet a patient’s individual needs. Thus, it is recommended that, when formulating treatment plans for, and assessing the outcomes of facial soft-tissue surgery, a true quantification of facial movements should be used to complement the clinician’s subjective assessments. A further important benefit is that the diagnostic findings—the mean animation comparisons—can be viewed by the parents or the patient during the normal consent process to highlight the impairments before surgery as well as the outcomes after the surgery.

MATERIAL AND METHODS

This approach consists of data acquisition instruments and statistical analyses.

Data acquisition instrument

Facial movement data are acquired by using a motion analysis system (Motion Analysis; Motion Analysis Corporation, Santa Rosa, Calif). The system tracks retro-reflective markers only secured to specific landmarks on a patient’s face. Patients are asked to perform a sequence of repeated (5 times) maximum facial animations (Fig 2): smile, lip purse, cheek puff, mouth opening, and grimace. A natural smile also is performed and repeated twice. During each animation, as many as 38 facial hemispherical landmarks (3 mm in size) are tracked in 3 dimensions at a rate of 60 Hz for 4 to 5 seconds.

Fig 2.

Motion analysis system: A, data acquisition (reprinted from Trotman et al. Functional outcomes of cleft lip surgery. Part II: Quantification of nasolabial movement. Cleft Palate Craniofac J 2007;44:607-16. With permission from Allen Press Publishing Services); B, the system recognizes and tracks landmarks secured to the face.

Statistical analyses

The movement data are analyzed as follows.

1. For each subject, the same distances between 2 landmarks—referred to as the interlandmark distances—are specified as the units of measurement.

2. The facial landmarks for each subject are scaled to the same size.

3. For each subject, changes over time for the interlandmark distances are calculated (Fig 3).

4. For each subject, mean changes over multiple repeated movements for each animation are calculated and statistically modeled.

5. Mean changes for the animations of an entire subject group can be calculated and statistically modeled. In this regard, our research group has compiled large longitudinal data bases of facial animations of patients with cleft lip and palate and noncleft control subjects. Figure 4, A, shows the nasolabial region of a patient with a repaired left unilateral cleft lip, and the statistical modeling in Figure 4, B, shows the mean movement of the facial landmarks for this patient (solid red circles) compared with the mean movement of 37 noncleft subjects (open black circles) during a smile animation. From the frontal perspective, the upper lip of the patient contracts vertically and tracks the mean upper lip movement of the noncleft subjects fairly closely. There is, however, less movement and some distortion on left side of the patient’s upper lip where the cleft lip has been repaired (evident from the left profile view).

Fig 3.

The unit of measurement is the “interlandmark” distance. Consider any 2 landmarks on the face (left upper lip circle in the left schematic). When the patient smiles, this distance increases (first vertical line on graph), then the distance is held (horizontal portion of line on graph), and finally the distance decreases as the patient relaxes (second vertical line on graph).

Fig 4.

Patient with a repaired left unilateral cleft lip. Animation of statistical modeling (available online) shows the mean movement of facial landmarks for this patient (solid red circles) compared with the mean movement of 37 noncleft subjects (open black circles) during a smile animation.

Conversely, the nasolabial region of the patient in Figure 5, A, shows a repaired bilateral cleft lip. The smile movement modeling shows that, compared with the mean noncleft smile movement (open black circles; Fig 5, B), the patient has limited contraction in the upper lip as well as limited overall movement of the upper lip region (solid red circles). Similar comparisons can be made between the mean movements for groups of subjects: eg, group mean for patients with cleft lip and group mean of noncleft subjects.

Fig 5.

Patient with a repaired bilateral cleft lip. Animation of smile movement modeling (available online) shows that, compared with the mean noncleft smile movement (open black circles), the patient has limited contraction in the upper lip and limited overall movement of the upper lip region (solid red circles).

For the statistical modeling and comparisons, calculated means and standard deviations also can be generated. Because the subject’s face is not visible during the animation statistical modeling, this allows the movement to be separated from the confounding effects of the scarring on the upper lip. Additional modeled movements can be accessed at: http://www.maths.bath.ac.uk/~jjf23/face/focls/. These models were generated by Dr J. J. Faraway, a collaborator and coinvestigator on this research.

Although the approach that we have developed to quantify facial movements is novel, we are exploring other more complete methods of data acquisition that are not limited to facial landmarks but will include the entire 3-dimensional facial surfaces captured during animated sequences. Currently, research in our laboratory (funded by National Institute of Dental and Craniofacial Research Grant #DE019742) is ongoing with a 3dMD 4D Capture System (3dMD, Atlanta, Ga; Fig 6) that allows such data to be collected over time.

Fig 6.

Three-dimensional images obtained from the 3dMD Motion Capture System.