Abstract
Background:
Crowdsourcing is increasingly being used in medical research to obtain the opinion of laypeople. The investigators hypothesized that a layperson’s perception of a primary maxillary deficiency (PMD) dentofacial deformity (DFD) patient is more favorable after orthognathic surgery with regard to perceived personality traits and emotional facial expressions.
Methods:
The investigators implemented a survey, distributed through Amazon.com’s Mechanical Turk crowdsourcing platform, to compare 6 perceived personality traits and 6 perceived emotional traits before and >6 months after orthognathic surgery in subjects through standardized facial photographs. The sample was composed of 20 subjects randomly selected from a PMD DFD database, treated by 1 surgeon all having undergone bimaxillary and chin orthognathic surgery. The outcome variable was change in each of 6 perceived personality and 6 emotional expression traits studied. Descriptive and bivariate statistics were computed. P-value of <0.05 was considered significant.
Results:
Five hundred respondents (raters) completed the survey. A majority of respondent raters were male (59%) and White (71%), ranging in age between 25 and 34 years (52%). After bimaxillary and chin orthognathic surgery, PMD subjects as a group were perceived to be significantly more dominant, more trustworthy, more friendly, more intelligent, more attractive, and less threatening. They were also perceived as happier and less angry, less surprised, less sad, less afraid, and less disgusted than before surgery (P < 0.05).
Conclusions:
We confirmed that laypeople consistently report positive changes in a PMD DFD subject’s perceived personality traits and perceived emotional expressions after bimaxillary and chin orthognathic surgery.
INTRODUCTION
Orthognathic surgery for individuals with developmental jaw deformities (DFDs) has demonstrated value in correcting head and neck functions related to breathing, speech articulation, chewing ability, and swallowing mechanism. In the process of correcting a dentofacial deformity, the patient’s facial esthetics are altered. The quality-of-life benefits of orthognathic surgery from the patient’s perspective have been documented in the literature.1–7 The broader impact of facial esthetic surgery, on the individual with a baseline facial dysmorphology, during their everyday social interactions with others remains in the early stages of study.8–11 First impressions are known to be directly connected to the appearance of the face and made instinctively in social interactions.12–14 Stereotypes of others are developed rapidly by humans as early as 6 months of age with attractive individuals described as having more favorable personality traits.15,16
Mazzaferro and colleagues used crowdsourcing to demonstrate the potential for favorable impressions of an individual’s personality traits and perceived emotions based on viewed facial photographs after orthognathic surgery.9,10 Our group has previously reported on positive changes in social perceptions achieved through bimaxillary and chin orthognathic surgery for individuals with long face DFD.11 However, the esthetic outcomes for the other common patterns of developmental jaw deformities after orthognathic surgery has been insufficiently documented.
Primary maxillary deficiency (PMD) is a pattern of developmental facial disharmony that presents with horizontal deficiency in the maxilla and a class III negative overjet malocclusion as essential components of the deformity.17 Direct visual examination of the patient’s face typically suggests a prominent appearing chin with a thick-appearing lower lip.17 The nose may appear prominent, whereas the cheekbones appear flat, with a sunken midface and a thin hypotonic upper lip.17
This study tests the hypothesis that a layperson’s perception of a PMD DFD patient is more favorable after orthognathic surgery than before with regard to implicit personality traits and perceived emotional facial expressions. The specific aim of this study was to use crowdsourcing to gather unbiased, large sample layperson data about perceived personality traits and perceived emotional facial expressions in PMD subjects by viewing standardized facial photographs before and after bimaxillary and chin orthognathic surgery.
METHODS
Study Sample
To address the research objectives, a cohort study group was identified. The sample was derived from patients treated by 1 surgeon (J.C.P.) in a private practice setting (Posnick Center) with surgery performed at a single hospital (MedStar Georgetown University Hospital, Washington, D.C.) between 2004 and 2013. During this study period, all subjects with a PMD bimaxillary DFD also involving the chin and with symptomatic chronic obstructive nasal breathing were identified.17 These subjects underwent surgery that, at a minimum, included Le Fort I osteotomy, bilateral sagittal ramus osteotomies of the mandible, osseous genioplasty, septoplasty (submucosal resection), and reduction of the inferior turbinates. Design of osteotomies (ie, Le Fort I osteotomy, sagittal ramus osteotomies, and anterior inferior border (chin osteotomy)), and the method of fixation used for each osteotomy, was consistent for all study subjects and previously described.18
Other simultaneous procedures performed in the study group included type of Le Fort I osteotomy [3-segment Le Fort I (30 of 66; 45%) and 2-segment Le Fort I (16 of 66; 24%)], removal of third molars (34 of 66; 51%), interpositional bone graft to the maxilla (13 of 66; 20%), and interpositional bone graft to the chin (2 of 66; 3%).17 No subjects underwent other facial cosmetic procedures during the study period (ie malar implants, fat grafting, rhinoplasty, or neck liposuction).
Maxillary deficiency subjects were excluded from study if they previously underwent an operation for their DFD or did not reside in North America, as long-term follow-up was geographically inconsistent. Exclusions for orthognathic surgery, by our surgical team, included those individuals who used nicotine products within 3 weeks before surgery, those currently taking bisphosphonate medications, immunosuppressed individuals, and insulin-requiring diabetic patients. All subjects in the study were confirmed to be cardiovascularly stable and free of renal disease, known coagulation disorders, and pulmonary disease. This study protocol obtained institutional review board approval from Georgetown University (No. 2014-0871) and is compliant with the Declaration of Helsinki.
Sixty-six consecutively treated subjects with a PMD DFD who underwent the stated procedures were identified for inclusion in this study. PMD developmental jaw deformities were confirmed to have a symmetric (even side-to-side) class III reverse overjet malocclusion with coincident maxillary and mandibular dental midlines (±1 mm). A PMD growth pattern is differentiated from other forms of developmental DFD that also can present with a class III reverse overjet malocclusion (ie, condylar hyperactivity, long face, and bimaxillary dental protrusion) by the history of presentation and other identifying or excluding clinical and radiographic findings.17 The PMD subjects’ age at the operation was 22 years (range, 15–55 years) and were 73% male. Twelve percent of the PMD study subjects were confirmed by polysomnography and a pulmonologist evaluation to have symptomatic obstructive sleep apnea (OSA).17
All patients were planned for maxillary advancement as measured at the incisors (mean, 6 mm; range, 2 to 12 mm). Most patients (65%) were planned for vertical change at the maxillary incisors to improve dental show and lip posture. Fifty-five percent were planned for vertical lengthening at the incisors (range, +1 to +5 mm), and 11% were planned for shortening (range, −1 to −5 mm). Most patients were planned to undergo alteration of the maxillary plane (pitch orientation), and all those corrections were completed in a clockwise direction. Most patients (68%) were planned for mandibular advancement (range, 1 to 8 mm). Only 26% of patients were planned for mandibular setback at the incisors (range, −1 to −3 mm).17
We have previously reported the study subjects’ preoperative occlusion and facial esthetic findings from the perspective of professionals.17 All patients exhibited an even (±1 mm) canine Angle class III malocclusion with reverse incisor overjet (range, 2 to 15 mm) before surgery.17 Facial esthetic analysis confirmed that before surgery, the majority of subjects were judged to have a prominent chin (55%), flat labiomental fold (61%), and a prominent appearing lower lip (88%).17 The preoperative study patients also were assessed to have the appearance of a prominent nose (77%), sunken midface projection (100%), flat cheekbones (82%), and a recessed upper lip (73%).17
Crowdsourcing Raters
Mechanical Turk (Amazon.com, Inc., Seattle, Wash.) and Survey Monkey (Palo Alto, Calif.) were used to determine how before and after orthognathic surgery facial images of PMD patients were perceived on 6 personality traits and 6 emotional expressions. We used a 7-point Likert scale response format to assess each of the 12 variables. This survey was limited to Mechanical Turk respondents (raters) who had a minimum 95% approval rating and were living in the United States. A series of demographic questions (about each rater) preceded each survey, which included their age, sex, race, education level, and the annual income. Dispersed within this set of questions were 3 control questions to ensure a thoughtful effort was made on the survey; we asked for the current year, the current month, and simple addition (7 + 4). Survey respondents (raters) were excluded if they did not answer the 3 control questions correctly. The preoperative and postoperative facial images for review by raters were delivered in a random order. Each respondent (rater) was also blinded to the purpose of the study and to the knowledge of subjects having undergone face-altering procedures. Each respondent (rater) was prevented from completing the survey more than one time and was compensated with US$3 to complete the survey. The number of Mechanical Turk respondents was limited to 500. This survey was assessable (by the raters) by means of a computer-based Internet browser or mobile-device Internet browser (smartphones/tablets). Photographs of Study subjects were included at the beginning and end of each page to allow for immediate reference by the rater while they answered each question.
PMD Study Subjects’ Sample Selection for Review by Raters
A subgroup of the 66 consecutive PMD study subjects was selected for layperson rater review. The subgroup (N = 20) was reduced from the total group to limit rater fatigue and improve rater reliability. A random number generator was used to avoid bias when picking the subgroup of 20 maxillary deficiency subjects.19
OUTCOME VARIABLES
Perceived Personality Traits and Perceived Emotions of PMD DFD Subjects before and after Orthognathic Surgery as Judged by Layperson Raters
A series of 6 Likert scale questions were asked regarding the respondent’s (rater’s) perception of 6 personality traits and 6 emotional expressions for each subject based on viewing standardized photographs (3 facial photographs). The raters’ perception of each specific personality trait and each specific emotional expression for each subject was based on viewing a standardized facial image set (Figs. 1, 2). The 7-point Likert response scale used to rate each of the 12 social perceptions for each subject is highlighted in Figure 3 and 4. The methodology used to rate each subject for each personality trait and emotional facial expression in this study was drawn from prior research of the effects of facial appearance on character impressions.9,10,20–23
Fig. 1.
Example of a PMD DFD study subject before orthognathic surgery. The standardized facial photographic image set formatted for review by each layperson rater is shown.
Fig. 3.
The 6 personality traits and Likert scale used in this study.
Fig. 4.
The 6 emotional expressions and Likert scale used in this study.
A single preoperative standardized facial image set was created for each PMD subject. This included a three-quarter (oblique facial) view, a lateral (profile facial) view, and a frontal view in repose (Fig. 1). A similar facial image set was replicated from each subject’s postorthognathic surgery photographs (Fig. 2). The photographic images were gathered from each subject’s electronic medical record. During the course of treatment, each subject underwent 6 standardized facial view photographs before surgery (T1), at 5 weeks postoperatively (T2), and at a minimum of 6 months after surgery (T3). The facial photographic views taken at each interval included frontal in repose, frontal with smile, left and right oblique views, and left and right profile views. The T1 and T3 photographs were used to create the standardized image sets described for each subject.
Fig. 2.
The PMD DFD study subject in Figure 1 is shown after bimaxillary and chin orthognathic surgery. The standardized facial photographic image set formatted for review by each layperson rater is shown.
COLLECTION, MANAGEMENT, AND ANALYSIS OF DATA
The data were abstracted and recorded on a standardized data collection form from the hospital and outpatient medical records. This also included review of facial photographs before and after surgery. The data were entered into a database created using Microsoft Excel (Microsoft Inc., Redmond, Wash.). The data were then transferred to a software packaged for statistical analysis (STATA 13.1; STATA Corp, LP, College Station, Tex.). Descriptive and bivariate analyses were performed. Bivariate analysis included the 2-tailed unpaired t test to compare perceived personality traits and perceived emotions before and after orthognathic surgery as a pooled cohort and then on an individual basis. P-values of <0.05 were considered significant.
During the time frame of the study, no PMD DFD patient undergoing simultaneous bimaxillary and chin orthognathic surgery by the primary investigator (J.C.P.) was excluded or lost to follow-up and no data points were missing for any of the study parameters for any of the subjects.
RESULTS
Demographic Analysis of Mechanical Turk Raters
Five hundred respondents (raters) completed the survey in <9 hours. Seventeen respondents were excluded due to incorrect answers of the control questions, with a final sample size of 483 respondents. Each respondent spent an average of 35 minutes 15 seconds to complete the survey. The majority of respondents within each demographic group were male (59%), 25–34 year of age (52%), White (71%), college graduates (55%) with an annual income between US$20,000 and US$50,000 (48%). The detailed demographics of the Mechanical Turk raters are summarized in Table 1.
TABLE 1.
Demographic Information of the Mechanical Turk Raters in PMD DFD Study
Perceived Personality Traits of PMD DFD Subjects before and after Orthognathic Surgery
After jaw reconstruction and completion of orthodontic treatment, PMD study subjects as a group were perceived to be significantly more trustworthy, more friendly, more intelligent, more attractive, more dominant, and less threatening (P < 0.05). Table 2 summarizes the results, and Figure 5 clarifies the differences in each perceived personality trait from before to after bimaxillary and chin orthognathic surgery. An analysis of each subject’s before and after orthognathic surgery facial photograph set was also made. On an individual basis, after bimaxillary and chin orthognathic surgery, 85% of PMD subjects were perceived as significantly more friendly, 85% as more trustworthy, 85% as more intelligent, 90% as more attractive, and 70% as less threatening (P < 0.05) (Table 3).
TABLE 2.
Comparison of Pre- and Postorthognathic Surgery Facial Photographs in PMD DFD Study Subjects, Relative to Personality Traits
Fig. 5.
Perceived differences in personality traits after bimaxillary and chin orthognathic surgery in PMD DFD study subjects. *Significant P < 0.05.
TABLE 3.
Perceived Changes in Personality Traits in PMD DFD Study Subjects after Orthognathic Surgery
Perceived Emotions of PMD DFD Subjects before and after Orthognathic Surgery
After jaw reconstruction and completion of orthodontic treatment, PMD subjects as a group were perceived to be significantly happier and less angry, less surprised, less sad, less afraid, and less disgusted than they were before surgery (P < 0.05). Table 4 summarizes the results, and Figure 6 clarifies the differences in each perceived emotional expression from before to after orthognathic surgery. An analysis of each PMD subject’s before and after orthognathic surgery facial photograph set was also performed. On an individual basis, after bimaxillary and chin orthognathic surgery, 80% of PMD subjects were perceived as significantly happier, 85% as less sad, 80% as less angry, 80% as less disgusted, 75% as less afraid, and 50% as less surprised (P < 0.05) (Table 5).
TABLE 4.
Comparison of Pre- and Postorthognathic Surgery Facial Photographs in PMD DFD Study Subjects, Relative to Emotional Expression
Fig. 6.
Perceived differences in emotional expressions after bimaxillary and chin orthognathic surgery in PMD DFD study subjects. *Significant P < 0.05.
TABLE 5.
Perceived Change in Emotional Traits of PMD DFD Study Subjects after Orthognathic Surgery
DISCUSSION
The purpose of this study was to assess a layperson’s social perceptions of a PMD DFD subject before and after orthognathic surgery. Our null hypothesis (HO) stated that: bimaxillary and chin orthognathic surgery would make no difference or no improvement in perceived personality traits or perceived emotional facial expressions in PMD subjects. The study results reject the null hypothesis and indicate that laypersons (Mechanical Turk respondents) perceived the maxillary deficiency subjects’ personality traits and emotional expressions to be significantly different after orthognathic surgery than before surgery. We confirmed that after bimaxillary and chin orthognathic surgery and completion of orthodontics, PMD DFD subjects were perceived to be more trustworthy, friendly, intelligent, attractive, dominant, and less threatening. The subjects were also perceived to be happier, less angry, less surprised, less sad, less afraid, and less disgusted.
Treatment objectives for the reconstruction of developmental dentofacial deformities frequently include correction of malocclusion, opening documented sites of upper airway obstruction and optimizing facial esthetics. Facial esthetics before and after orthognathic surgery have proven difficult to quantify and are typically reported from the perspective of either the surgeon or the patient.8,17 Layperson’s opinions are likely to be more representative of whom the subject may randomly interact with on a daily basis, therefore providing valuable, unbiased input. This study reports change in social perceptions that an individual with PMD DFD might expect to encounter by the general public after orthognathic surgery. We report statistically significant improvement in the 6 perceived personality traits and the 6 perceived emotional facial expressions studied.
When assessing changes in social perceptions as reported by laypersons after orthognathic surgery, PMD DFD subjects experienced a greater magnitude of positive change than long face DFD subjects for the following personality traits, being perceived as more trustworthy, less threatening, more friendly, more intelligent, and more attractive.4 The maxillary deficiency subjects, as compared with long face DFD subjects, were perceived to have a greater net change in being perceived as more happy, less angry, less surprised, less sad, less afraid, and less disgusted.4
Crowdsourcing via mTurk is a valuable tool to rapidly gather the unbiased opinions of large numbers of laypersons.24,25 The quality of participant (rater) responses collected through mTurk have been found similar to responses collected in person and produce a more diverse group of respondents.25 Mechanical Turk respondents have also been shown to produce results that rival the work of highly paid, domain-specific experts and due to higher number of respondents have less inter-rater variability.26–28 Mechanical Turk has also been used in other healthcare-related research such as using laypersons to assess surgical technical skills, outcomes of treatment for cosmetic procedures, and patient preferences when seeking esthetic surgery.9,10,26–33
Strengths of this study include large sample size, a focus on just 1 subtype of developmental DFD (ie, PMD), and consistency in the orthognathic procedures performed (ie bimaxillary and chin osteotomies). In an effort to decrease bias, we only used standardized facial photographs before and at a minimum of 6 months after orthognathic surgery and completion of orthodontics. Subjects were also randomly chosen for study inclusion from our larger PMD DFD database without patient dropout or surgeon bias. An additional strength of this study is that the raters were blinded to the fact the subjects had undergone surgery and that the subject’s before and after photographs were presented in a random nonsynchronous order.
Study weaknesses include our inability to control the effort of raters in completing the survey and the inherent limitations of our study design. In addition, the rating group in our study may not replicate the population that each specific subject interacts with on a daily basis.34–36 Also, photographs were taken during routine appointments and at standard time intervals before and after surgery. We acknowledge that random hairstyle and makeup changes that may have been carried out may be confounders when assessing social perceptions based on review of photographic images. Lastly, although the Likert scale responses to questions were generated from validated studies, the transference of the perception of these traits to real-world scenarios remains unknown.9,10
CONCLUSIONS
We confirmed that laypeople consistently report improved positive social perceptions of PMD DFD subjects after bimaxillary and chin orthognathic surgery compared with before surgery. The improved social perceptions achieved for PMD subjects after orthognathic surgery are for a broad spectrum of the individual’s personality traits and perceiving emotional facial expressions.
Footnotes
Published online 11 April 2019.
Disclosure: The authors have no commercial associations or financial conflicts to disclose.
REFERENCES
- 1.Lazaridou-Terzoudi T, Kiyak HA, Moore R, et al. Long-term assessment of psychologic outcomes of orthognathic surgery. J Oral Maxillofac Surg. 2003;61:545. [DOI] [PubMed] [Google Scholar]
- 2.Nicodemo D, Pereira MD, Ferreira LM. Effect of orthognathic surgery for class III correction on quality of life as measured by SF-36. Int J Oral Maxillofac Surg. 2008;37:131. [DOI] [PubMed] [Google Scholar]
- 3.Posnick JC, Wallace J. Complex orthognathic surgery: assessment of patient satisfaction. J Oral Maxillofac Surg. 2008;66:934. [DOI] [PubMed] [Google Scholar]
- 4.Kim SJ, Kim MR, Shin SW, et al. Evaluation on the psychosocial status of orthognathic surgery patients. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2009;108:828. [DOI] [PubMed] [Google Scholar]
- 5.Rustemeyer J, Gregersen J. Quality of Life in orthognathic surgery patients: post-surgical improvements in aesthetics and self-confidence. J Craniomaxillofac Surg. 2012;40:400. [DOI] [PubMed] [Google Scholar]
- 6.de Ávila ED, de Molon RS, Loffredo LC, et al. Health-related quality of life and depression in patients with dentofacial deformity. Oral Maxillofac Surg. 2013;17:187. [DOI] [PubMed] [Google Scholar]
- 7.Hunt OT, Johnston CD, Hepper PG, et al. The psychosocial impact of orthognathic surgery: a systematic review. Am J Orthod Dentofacial Orthop. 2001;120:490. [DOI] [PubMed] [Google Scholar]
- 8.Sinko K, Jagsch R, Benes B, et al. Facial aesthetics and the assignment of personality traits before and after orthognathic surgery. Int J Oral Maxillofac Surg. 2012;41:469. [DOI] [PubMed] [Google Scholar]
- 9.Mazzaferro DM, Wes AM, Naran S, et al. Orthognathic surgery has a significant effect on perceived personality traits and emotional expressions. Plast Reconstr Surg. 2017;140:971. [DOI] [PubMed] [Google Scholar]
- 10.Lin LO, Zhang RS, Mazzaferro DM, et al. Influence of repaired cleft lip and palate on layperson perception following orthognathic surgery. Plast Reconstr Surg. 2018;142:1012. [DOI] [PubMed] [Google Scholar]
- 11.Posnick JC, Kinard BE. Orthognathic surgery has a significant positive effect on perceived personality traits and perceived emotional expressions in long face subjects. J Oral Maxillofac Surg. 2019;77:408.e1. [DOI] [PubMed] [Google Scholar]
- 12.Bar M, Neta M, Linz H. Very first impressions. Emotion. 2006;6:269. [DOI] [PubMed] [Google Scholar]
- 13.Willis J, Todorov A. First impressions: making up your mind after a 100-ms exposure to a face. Psychol Sci. 2006;17:592. [DOI] [PubMed] [Google Scholar]
- 14.Todorov A, Olivola CY, Dotsch R, et al. Social attributions from faces: determinants, consequences, accuracy, and functional significance. Annu Rev Psychol. 2015;66:519. [DOI] [PubMed] [Google Scholar]
- 15.Ramsey JL, Langlois JH, Hoss RA, et al. Origins of stereotype: categorization of facial attractiveness by 6-month-old infants. Dev Sci. 2004;7:201. [DOI] [PubMed] [Google Scholar]
- 16.Langlois JH, Kalakanis L, Rubenstein AJ, et al. Maxims or myths of beauty? A meta-analytic and theoretical review. Psychol Bull. 2000;126:390. [DOI] [PubMed] [Google Scholar]
- 17.Posnick JC, Makan S, Bostock D, et al. Primary maxillary deficiency dentofacial deformities: occlusion and facial esthetic surgical outcomes. J Oral Maxillofac Surg. 2018;76:1966. [DOI] [PubMed] [Google Scholar]
- 18.Posnick JC. Posnick JC. Sequencing of orthognathic procedures: step-by-step approach Ch15. In: Orthognathic Surgery: Principles and Practice. 2014:St. Louis, MO: Elsevier; 441. [Google Scholar]
- 19.Urbaniak GC, Plous S. Research Randomizer (Version 4.0) [Computer Software]. 2013. Available at: http://www.randomizer.org/. Accessed December 16, 2017.
- 20.Sutherland CA, Young AW, Rhodes G. Facial first impressions from another angle: how social judgements are influenced by changeable and invariant facial properties. Br J Psychol. 2017;108:397. [DOI] [PubMed] [Google Scholar]
- 21.Oosterhof NN, Todorov A. Shared perceptual basis of emotional expressions and trustworthiness impressions from faces. Emotion. 2009;9:128. [DOI] [PubMed] [Google Scholar]
- 22.Todorov A. Evaluating faces on trustworthiness: an extension of systems for recognition of emotions signaling approach/avoidance behaviors. Ann N Y Acad Sci. 2008;1124:208. [DOI] [PubMed] [Google Scholar]
- 23.Hareli S, Shomrat N, Hess U. Emotional versus neutral expressions and perceptions of social dominance and submissiveness. Emotion. 2009;9:378. [DOI] [PubMed] [Google Scholar]
- 24.Ranard BL, Ha YP, Meisel ZF, et al. Crowdsourcing–harnessing the masses to advance health and medicine, a systematic review. J Gen Intern Med. 2014;29:187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Mason W, Suri S. Conducting behavioral research on Amazon’s Mechanical Turk. Behav Res Methods. 2012;44:1. [DOI] [PubMed] [Google Scholar]
- 26.Aghdasi N, Bly R, White LW, et al. Crowd-sourced assessment of surgical skills in cricothyrotomy procedure. J Surg Res. 2015;196:302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Holst D, Kowalewski TM, White LW, et al. Crowd-sourced assessment of technical skills: an adjunct to urology resident surgical simulation training. J Endourol. 2015;29:604. [DOI] [PubMed] [Google Scholar]
- 28.Hu Y, Jiang B, Kim H, et al. Vessel ligation fundamentals: a comparison of technical evaluations by crowdsourced nonclinical personnel and surgical faculty. J Surg Ed. 2018;75:664. [DOI] [PubMed] [Google Scholar]
- 29.Tse RW, Oh E, Gruss JS, et al. Crowdsourcing as a novel method to evaluate aesthetic outcomes of treatment for unilateral cleft lip. Plast Reconstr Surg. 2016;138:864. [DOI] [PubMed] [Google Scholar]
- 30.Wu C, Scott Hultman C, Diegidio P, et al. What do our patients truly want? Conjoint analysis of an aesthetic plastic surgery practice using internet crowdsourcing. Aesthet Surg J. 2017;37:105. [DOI] [PubMed] [Google Scholar]
- 31.Nellis JC, Ishii M, Papel ID, et al. Association of face-lift surgery with social perception, age, attractiveness, health, and success. JAMA Facial Plast Surg. 2017;19:311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Reilly MJ, Tomsic JA, Fernandez SJ, et al. Effect of facial rejuvenation surgery on perceived attractiveness, femininity, and personality. JAMA Facial Plast Surg. 2015;17:202. [DOI] [PubMed] [Google Scholar]
- 33.Lu SM, Hsu DT, Leipziger LS, et al. The public face of rhinoplasty: impact on perceived attractiveness and personality. Plast Reconstr Surg. 2018;142:881. [DOI] [PubMed] [Google Scholar]
- 34.Semega JL, Fontenot KR, Kollar MA. Income and Poverty in the United States: 2016 Current Population Reports. Available at: https://www.census.gov/content/dam/Census/library/publications/2017/demo/P60-259.pdf. Accessed October 1, 2018.
- 35.Schondelmyer E. Demographics and Living Arrangements: 2013 Current Population Reports. Available at: https://www.census.gov/content/dam/Census/library/publications/2017/demo/p70br-148.pdf. Accessed October 1, 2018.
- 36.Ryan CL, Bauman K. Educational Attainment in the United States: 2015 Population Characteristics. Available at: https://www.census.gov/content/dam/Census/library/publications/2016/demo/p20-578.pdf. Accessed October 1, 2018.