Systematic Review and Meta-Analysis in Facial Plastic Surgery

Abstract

Systematic reviews and meta-analyses hold a unique position in the pyramid of evidence. They can provide transparent and rigorous summaries to answer many clinical questions in Facial Plastic Surgery. They can also identify areas of research deficiency, create new knowledge, and support guidelines or policies. A well-conducted systematic review follows a structured process to minimize bias and ensure reproducibility. When appropriate, a meta-analysis is incorporated to provide a statistical synthesis that combines the results of individual studies. This powerful quantitative method is becoming more prevalent in our field.. This article provides a practical framework to understand and conduct this valuable type of research.

INTRODUCTION

Facial plastic and reconstructive surgery is a highly specialized but remarkably diverse specialty – ranging from cosmetic rhinoplasty and facial rejuvenation surgery to craniofacial trauma reconstruction, cleft lip and palate surgery, microvascular surgery, and facial reanimation. In an era of evidence-based medicine, this diversity presents unique challenges and opportunities for facial plastic surgeons. Patients, practitioners, policy-makers, and third party payers all increasingly seek evidence-based answers to specific clinical questions: How prevalent is this clinical problem? What are the risk factors for a particular complication? How effective is one surgical procedure as compared to another? Systematic reviews and meta-analyses provide transparent and rigorous summaries of the best available evidence. They are an important addition to the literature because the conclusions play a critical role in developing practice guidelines, identifying gaps in knowledge, defining surgical quality metrics, and allocating resources.

WHAT IS A SYSTEMATIC REVIEW

Early efforts to summarize evidence in clinical medicine took the form of narrative expert reviews. They lacked clear structure and were subject to the authors’ bias in both the selection of the literature and the synthesis of the findings. Conversely, the systematic review follows a structured and reproducible process for searching, selecting, and summarizing the available evidence. This process minimizes bias and provides transparent and reliable answers to clinical questions. The process starts with formulating a focused clinical question and is followed by a comprehensive review of the medical literature. Explicit criteria then determine which studies are used to formulate a clinical summary of the findings. Systematic reviews with meta-analyses can summarize the best available evidence to answer many clinical questions in Facial Plastic Surgery.

HOW TO CONDUCT A SYSTEMATIC REVIEW

The systematic review is analogous to primary research in that one reports methods, data collection, and analysis. First, one defines a focused review question and specifies a search strategy of the medical literature that captures most, if not all, of the relevant literature.¹ The review proceeds to identify the eligible studies and evaluate the quality of the available evidence. Frequently, a systematic review is then combined with a meta-analysis, although they are methodologically distinct.

1. Defining the Research Question

The first, and sometimes most difficult, step is to define the objective of the systematic review. This objective can usually be expressed as a specific clinical question. The acronym PICOT is sometimes used to describe key components of the research question: Population, Intervention, Comparison, Outcome, and Time. It is advisable to survey the available literature to guide the development of a feasible research question. This consideration is particularly relevant in Facial Plastic Surgery, where the small sample size, difficulty of randomizing surgical patients, and the inconsistent outcome measures limit the research data. It is important to determine whether the research question is dealing with etiology, diagnosis, intervention, prognosis, or cost. The type of the research question dictates the most suitable study design as well as the potential biases that may influence findings. For example, when we want to evaluate if perioperative steroids decrease perioperative edema and ecchymosis following rhinoplasty, the highest quality studies should be RCTs. On the other hand, if the review question is examining which facial nerve outcome scale has the best reliability and validity, the studies will be cohorts of patients with facial nerve deficit.

2. Developing a Search Strategy

Systematic reviews are distinguished from other reviews by the well-structured, explicit, and reproducible search strategy. The strategy is designed based on the PICOT components of the review question. Although the goal is to capture all the relevant studies, increasing the comprehensiveness (or sensitivity) of a search will reduce its precision and therefore yield many non-relevant studies. The search should strike a favorable balance between being comprehensive, yet relevant and manageable. Navigating though databases such as MEDLINE, EMBASE or CENTRAL can be technically demanding, and collaborating with a healthcare librarian is strongly recommended. Each database has developed specific “controlled vocabulary” and filters to retrieve the studies of interest from millions of publications. It is important that the search is performed in more than one database using both controlled vocabulary as well as regular text words. Filters and limit terms can be added to refine the search such as a language, publication date, study design, or population age. Although the majority of systematic reviews are limited to the published literature, some review questions call for searching though dissertations, trial registries, meeting abstracts, or even contacting agencies or health providers. This is important in areas were publication bias is thought to heavily influence the results, such as adverse events and complications. Finally, the retrieved articles from several databases as well as any unpublished articles are merged together in a master library and duplicates removed. Figure 1 illustrates the value of a comprehensive search strategy that uses more than one database and possibly includes unpublished results.

FIGURE 1.

Venn diagram showing the interrelationship of several databases that may be used in a systematic review. Note the spread of data, spanning the 3 databases and the unpublished data which is not usually retrievable in electronic searches.

3. Identifying the Evidence

Once the pool of candidate articles has been accumulated, the reviewers then determine which articles meet the defined criteria for inclusion. The inclusion and exclusion criteria should be clearly specified a priori. It is typical for the search to retrieve several hundreds or even thousands of articles that need to be distilled to reach a handful of eligible studies. This process is often done in two stages. First, the reviewers screen the titles and abstracts to identify any potential articles. Subsequently, two independent reviewers evaluate the screened publications using the inclusion and exclusion criteria. Disagreements between the two reviewers are resolved by consensus or by a third adjudicator (kappa statistic can be provided as a measure of disagreement). This provides validity to the selection process by minimizing bias or arbitrary selection. It is helpful when the reviewers perform piloting to ensure that the selection criteria are clear and reproducible. If the criteria are vague and are heavily influenced by subjective interpretation, there may be substantial disagreements that call into question the reliability (reproducibility) of the selection process. Once the eligible studies are finalized, the data is extracted from each study using standardized data extraction forms. Usually the methodological details, sample size, and numerical results are summarized into a review table. An example data extraction form is available from the Cochrane collaboration (http://www.cochranerenal.org/docs/data_extraction_form.doc).

4. Evaluating Quality of Evidence and Bias

Rigorous quality control is an important feature of the systematic review. The methodological quality of each included study is evaluated with particular focus on the risk of bias. Bias introduces systematic deviation from the truth and can corrupt the results of the study. Experimental and observational studies alike can be subject to the five classic types of bias: reporting bias, selection bias, performance bias, attrition bias, and detection bias.² Table 1 summarizes these biases and provides practical examples. Reporting (publication) bias refers to a deviation typically towards favorable results in published studies compared to unpublished studies. Selection bias occurs when each group is selected differently causing incomparable groups with regard to important baseline characteristics and predictors of outcome. In studies of etiology or prognosis, selection bias can lead to a confounding effect. This is a distortion of the association between the exposure and the outcome because the study groups differ with respect to other factors that influence the outcome. Performance bias is a result of major differences in care amongst groups that influence the outcome. Attrition bias occurs if the rate of withdrawal was unequal between the study groups. Withdrawals from the study lead to incomplete outcome data that influence the analysis. Detection bias is a systematic difference between groups in how the outcome is determined or assessed. There are several instruments developed to evaluate the risk of bias based on the methodological quality of the study. The QUADAS (quality assessment of diagnostic accuracy studies)³ and Jadad scale⁴ (a brief instrument that evaluates risk of bias in RCTs) are among the commonly used instruments. The risk of bias is dictated by the specifics of the review question. For example, the risk of detection bias (how outcome is evaluated) can range from substantial if the outcome is “soft” (such as surgeon rating of rhinoplasty outcome) to minimal if the outcome is “hard” (such as peumothorax after rib harvest).⁵ Accordingly, blinding of the individual assessing the outcome is very important in the case of soft outcome but less critical in the case of hard outcome.

Table 1.

Summary of five classic types of bias that are evaluated in primary studies

Type of Bias	Definition	Example	How to Minimize
Reporting bias (publication bias)	There is deviation, typically toward favorable results, in published studies compared with unpublished studies.	Several RCTs evaluate if a new laser delivery improves facial aging compared with an existing treatment. Only the trials with statistically significant improvement are published; studies that encountered complications might also be suppressed.	Inclusion of unpublished data in the systematic review and meta- analysis provides a more balanced review.
Selection bias	Each group is selected differently, causing incomparable groups with regard to important baseline characteristics and predictors of outcome.	An RCT compares the effect of perioperative steroid vs placebo on decreasing facial edema after septorhinoplasty. Patients with more extensive osteotomies and bone mobilization were allocated to the treatment group.	In observational studies, rigorous cohort enrollment and adjustment methods, if required. In RCTs, allocation concealment can prevent biased selection.
Performance bias	Substantial differences in care among groups influence the outcome.	A cohort study evaluates if antibiotic use after laser resurfacing decreases the risk of infection. The treating physicians were not masked and were more likely to add topical antibiotic treatment to the control group.	Masking the health care providers to be unaware of patient allocation and treatment.
Attrition bias	The rate of withdrawal is unequal between the study groups, leading to incomplete data that influence the analysis.	A study compares repeated filler injection with placebo. Patients receiving placebo injections were more likely to drop out or not comply with follow-up.	Masking of patients may decrease aspects related to patient perception.
Detection bias	A systematic difference exists between groups in how the outcome is determined or assessed.	A study compares two methods of rhinoplasty using surgeon-based outcome. Surgeons’ perception and beliefs influence the evaluation of outcome.	Masking the assessors of outcome to minimize the influence of their beliefs and perceptions.

WHAT IS A META-ANALYSIS?

Meta-analysis is a statistical method to pool data from two or more studies. This quantitative synthesis aims to answer the research question with greater precision (certainty) and generalizability (external validity) than is possible from individual studies. It can also provide new knowledge that explains the variability observed in the literature or highlight unrecognized aspects of the research question, such as an effect modifier. We highlight a landmark meta-epidemiology by Wood et al that examined 1346 RCTs to compare ‘double-blind’ trials to those without double-blinding.⁶ The review demonstrated that lack of blinding was associated with an overall 7% (95% CI: 0–17) exaggeration of treatment effect, which increased to 25% (95% CI: 18–39) in trials with ‘subjective’ outcomes. The findings supported the importance of blinding to minimize bias, particularly in trials with subjective outcomes.

Relatively few meta-analyses have been conducted in Facial Plastic Surgery, likely due to the demanding methodology and limitations imposed by the available literature. A meta-analysis in the Aesthetic Journal of Surgery evaluated if perioperative steroids after rhinoplasty minimize edema and ecchymosis.⁷ The study included seven RCTs without evaluation of bias or heterogeneity. The statistical pooling failed to use acceptable methods, but rather combined absolute means without forest plot presentation. The authors concluded that “perioperative steroid decreases postoperative edema and ecchymosis associated with rhinoplasty” and made a strong recommendation for “evidence-based guidelines” supporting their use. Recently, a rigorous meta-analysis was completed by the Cochrane group using high sensitivity literature search and detailed methodological evaluation of the trials.⁸ The review identified nine eligible RCTs examining rhinoplasty, with two trials suitable for meta-analysis. Their review presented full evaluation of bias and heterogeneity; the standardized mean difference was pooled using a fixed effect model. The authors concluded: “There is limited evidence that high doses of corticosteroids decrease both ecchymosis and edema. The clinical significance of this decrease is unknown, and there is little evidence available regarding the safety of this intervention…Therefore, the current evidence does not support use of corticosteroids as a routine treatment in Facial Plastic Surgery.”

HOW TO CONDUCT A META-ANALYSIS

We present a practical framework for understanding and conducting meta-analyses. A complete systematic review is required before starting the meta-analysis. If the systematic review has two or more studies that can be quantitatively combined, then a meta-analysis can be performed. The process requires standardizing the results, evaluating heterogeneity, synthesizing a summary, and finally evaluating robustness (sensitivity analysis). Table 2 highlights the systematic reviews and meta-analyses retrieved from important journals in Facial Plastic Surgery.

Table 2.

Summary of systematic reviews and meta-analysis retrieved from journals in facial plastic surgery

Author, Year	Scope	Review Question	Outcomes	Conclusions
Wee et al,13 2015	Rhinoplasty	Evaluate complications related to autologous rib cartilage rhinoplasty	Nasal complications, donor-site morbidity, and revision surgery	Long-term complications and donor-site morbidity rates associated with autologous rib cartilage use in rhinoplasty were low. Because of limitations future studies are needed.
Paleri et al,18 2014	Microvascular	Evaluated impact of vascularized tissue on fistula rate after laryngectomy reconstruction	Fistula rate	Flap reconstruction/reinforcement with vascularized tissue reduced their risk of pharyngocutaneous fistula by approximately one-third.
Rhee et al,19 2014	Rhinoplasty	Define symptomatic, normative, and postoperative values for nasal obstruction	Nasal obstruction assessed with VAS and NOSE	VAS and NOSE can be used as a clinically meaningful measure of successful surgical outcomes.
da Silva et al,8 2014	Perioperative	To determine the effects, including safety, of perioperative administration of corticosteroids for preventing complications following facial plastic surgery in adults	Ecchymosis and edema	Limited evidence that corticosteroids decrease ecchymosis and edema. There is little evidence regarding the safety of this intervention.
Cheung et al,20 2013	Trauma	Determine safety and efficacy of endoscopic management of isolated orbital floor fractures	Resolution of diplopia and enophthalmos, postoperative complications	Reconstruction of orbital floor fractures through an endoscopic approach seems to be safe and effective.
Morris & Kellman,21 2014	Trauma (antibiotics)	Studied role of prophylactic antibiotics in the management of facial fractures	Postoperative infection rate	Risk of infection in patients with mandibular fractures is reduced with use of prophylactic antibiotics from time of injury to completion of the perioperative course.
Picavet et al,22 2011	BDD	Assessment of screening tools for BDD in cosmetic surgery setting	BDD questionnaire-dermatology version, dysmorphic concern questionnaire	Despite high prevalence of BDD in cosmetic surgery, little is known about these tools in the cosmetic surgery setting. Further research is needed on the prevalence of BDD in cosmetic surgery and impact of BDD on treatment outcomes.
André et al,23 2009	Rhinoplasty	Studied the relationship between subjective and objective evaluation of the nasal airway	Rhinomanometry, acoustic rhinometry, patient-reported outcomes	Patients’ subjective nasal obstruction does not correlate well with rhinomanometry and acoustic rhinometry. There is thus little evidence for routine use of rhinomanometry or acoustic rhinometry for quantifying surgical results in clinical rhinologic practice.
Spielmann et al,24 2009	Rhinoplasty	Evaluate surgical treatment strategies for nasal valve collapse	Subjective symptom relief, cosmetic outcome, objective measurements of nasal airway patency	No randomized controlled trials on nasal valve surgery were identified. Reporting of long-term outcomes is limited. There is uncertainty regarding evidence base for choice of specific technique and duration of benefit.
Nash et al,25 2010	Trauma/facial nerve	Evaluate impact of early surgical intervention vs steroid administration/observation	House-Brackmann scale	The available evidence does not clearly indicate whether surgical vs nonsurgical intervention achieves most favorable outcome for facial paralysis after trauma.
Rhee & McMullin,26 2008	Measurement	Identify outcome instruments specific for facial plastic surgery interventions and conditions	Outcome instruments validation	Validated outcome measures are available for common facial plastic surgery conditions. Challenges remain in harmonizing and standardizing the different measures to reach clinically meaningful assessments of outcomes.
Rhee et al,27 2008	Rhinoplasty	Critical review of evidence supporting functional rhinoplasty or nasal valve repair	Validated patient-reported outcome measures	Level 4 evidence supports the efficacy of functional rhinoplasty for treatment of nasal obstruction arising from nasal valve collapse. Further studies with standardized objective outcome measures and comparison cohorts are needed.
Leventhal et al,28 2006	Wound healing	Determine treatments that can improve keloid and hypertrophic scars	Assessment of keloid and hypertrophic scars	Most treatments for keloidal and hypertrophic scarring offer minimal likelihood of improvement.
Koch & Perkins,29 2002	Rhytidectomy/laser	Evaluate the safety of combining carbon dioxide laser resurfacing with full-face rhytidectomy	Rate of postoperative complications	Simultaneous rhytidectomy and carbon dioxide laser resurfacing can safely provide a dual cosmetic benefit for aesthetic rejuvenation.

The journals JAMA Facial Plastic Surgery, Laryngoscope, JAMA Otolaryngology, and Otolaryngology – Head & Neck Surgery were searched using publication type limit to “review,” “systematic review,” or “meta-analysis.” Cochrane library was also searched for related reviews. The titles and abstracts were first screened, and then two reviewers independently identified eligible studies (systematic review or meta-analysis in facial plastic surgery). Subsequently, the data were extracted from each eligible study using standardized form and summarized.

Abbreviations: BDD, body dysmorphic disorder; NOSE, Nasal Obstruction Symptom Evaluation scale; VAS, visual analog scale.

Data from Refs.^8,13,18–29

1. Standardizing the Results

It is important to first understand the type of data under study. Some data are from validated continuous or ordinal scales, such as the Nasal Obstruction Symptom Evaluation (NOSE) scale and the Wrinkle Severity Rating Scale (WSRS).^9,10 Other outcomes are dichotomous, such as the presence of infection or extrusion of implant. Time-to-event and count variables are unusual in Facial Plastic Surgery but may be encountered in the reporting of rare adverse events or rates. To allow meaningful pooling of the results, the type of outcome needs to be similar across studies. For example, if one study is reporting satisfaction with rhinoplasty as a dichotomous outcome and another study is reporting the satisfaction on an ordinal scale, then these outcomes cannot be simply pooled together. In some circumstances, it might be appropriate to collapse the ordinal scale into a dichotomous scale.

Next, the outcome of interest is extracted from each study using consistent methodology. Relative estimates (odds ratio, relative risk, mean difference) are preferable to absolute estimates of effect size. For continuous data, the mean with standard deviation can be extracted directly. If standard deviation is not reported, it is calculated from the standard error, confidence interval, or p-value. Ordinal data can be handled in several ways, depending on the data distribution. Frequently, the ordinal scale is treated as a continuous variable when the scale is large enough and the data exhibit symmetrical distribution. Alternatively, the data can be dichotomized or (rarely) maintained as a median with interquartile range. For dichotomous outcomes, the number of positive/negative events can be extracted directly.

2. Evaluating Heterogeneity

The step of evaluating heterogeneity is critical because it dictates whether data from different studies can be combined. Statistical heterogeneity refers to the inter-study variability of data. Heterogeneity can be evaluated graphically with forest plot (based on the overlap in the confidence intervals) or statistically with a chi-squared statistic.¹¹ Some differences between studies are expected by chance (sampling variation); however, statistically significant heterogeneity should be explored and explained. The variability in clinical factors (population, intervention and outcome) as well as the variability in methodological factors (study design and risk of bias) need to be carefully examined.¹² Only sufficiently similar studies are grouped and meta-analyzed together because combining conflicting results provides misleading estimates and obscures important findings. When heterogeneity cannot be explained, one may opt to use a statistical approach that assumes a distribution of effect (random effect model). Nonetheless, if the results are widely different or conflicting, it might be more suitable to avoid the pooling of the results and instead provide a qualitative summary. Consider a meta-analysis of NOSE outcomes following functional rhinoplasty. Some studies may have male patients with a history of trauma while others might be predominantly females with aesthetic goals. Heterogeneity arising from such population differences is important to recognize. It is preferable to stratify the studies into separate groups rather than obscuring the difference with inappropriate pooling.

3. Synthesizing a Summary

Meta-analysis cannot simply “add up” the numerical results of the individual studies as if they all belonged to one big group. Rather, the quantitative synthesis requires appropriate statistical methods. There are two main approaches to combine individual estimates: The fixed-effect model assumes that the effect is similar between studies, whereas the random-effect model assumes a distribution of effect size across studies. When there is no heterogeneity among studies, both the fixed-effect and the random-effect models provide identical pooled estimation. Nonetheless, when there is significant heterogeneity, the confidence interval for the pooled estimation will be wider (thus statistical significance more conservative) with the random-effect compared to the fixed-effect model. The results are then summarized graphically in a forest plot that provides the individual and the pooled estimations with the corresponding confidence intervals. Figure 2 presents a conceptual schematic of a forest plot from a meta-analysis. Such presentations are beginning to find application in the Facial Plastic Surgery literature, as in a recent comprehensive report examining rates of warping, resorption, infection, and displacement associated with rib cartilage use in rhinoplasty.¹³

FIGURE 2.

Appearance of a generic Forest Plot, depicting distribution and weight of constituent studies included in meta-analysis. In the example shown, odds ratios from 6 (fictitious) studies are depicted, with the size of each square proportional to the weight of the study in the meta-analysis. The horizontal lines reflect the confidence intervals individual studies. The diamond represents a summary measure, and the width of the diamond reflects the confidence interval, and the solid vertical line corresponds to no effect.

Several statistical methods have been developed for meta-analysis.¹⁴ The fixed-effect model is most commonly performed with inverse variance method, in which the weight given to each study corresponds to the inverse of the variance (reciprocal of the standard error squared). Thus, larger studies have more weight as they have smaller standard errors. When the sample size is small or the event rate is low, the inverse variance method leads to a poor estimation, and the Mantel-Haenszel method is preferred. Peto’s method is another modification of inverse variance that uses observed and expected statistics to estimate log odds ratio. It can only be used with odds ratio but has the advantage of not requiring correction for zero cells (groups without events). A random-effect model can sometimes be used to accommodate the variability.¹⁵ The model assumes a lack of knowledge about why an effect varies and considers that the effect has a distribution. The centre of the distribution describes the average effect while the width represents the degree of variability.

Facial Plastic Surgery presents some challenges for meta-analysis. First, as in other surgical fields, RCTs are sparse and most of the evidence is formed with small sample size observational studies of variable methodology. Second, despite some recent contributions, patient-centered outcome measures are relatively underdeveloped without clear consensus in reporting. The grading of outcome and severity for facial nerve deficit illustrates the dilemma. A recent systematic review identified 19 different scales; almost all scales had severe limitations in several measurement aspects and unclear correlation with patient’s quality-of-life.¹⁶ In such cases with marked variability in the methodology and outcome assessment, individual level data metaanalysis can offer a suitable solution.¹⁷ For example, if individual studies of facial nerve outcome can provide sufficient descriptive details for each case, this may facilitate rescore in a standardized scale to allow individual-level data pooling of the results.

4. Evaluating Robustness (Sensitivity Analysis)

Completing a systematic review and meta-analysis requires a sequence of decisions. While the process is structured, some steps might be arbitrary, based on assumption, or subject to opinion. This can make the results of the meta-analysis vulnerable. A sensitivity analysis is a repeat of the meta-analysis with an alternative decision or approach; it aims to evaluate the robustness of the results in the face of specific vulnerabilities. For example, the eligibility of some studies might be subject to opinion due to unclear methodological reporting. Here the sensitivity analysis might repeat the meta-analysis with only the “certainly” eligible studies included. Similarly, the assumption of insignificant heterogeneity might not be clear, and the sensitivity analysis can repeat the statistical pooling with a random effect method. When the results of the sensitivity analysis align with the results of the primarily analysis, the findings are robust. Nonetheless, if the difference is substantial then the synthesis is sensitive to specific assumptions, and this should be considered when formulating conclusions and recommendations.

CONCLUSION

Systematic reviews and meta-analysis have a well-established role in summarizing the best available evidence to answer a specific clinical question. They are the supporting foundation for clinical guidelines and policy decisions. Nonetheless, this valuable research continues to be underutilized in Facial Plastic Surgery. This situation is likely due to the technical challenges imposed by the current state of the literature, but also to the demanding aspects of the methodology. A collaborative effort is required to make high quality systematic reviews and meta-analyses an achievable.

KEY POINTS.

• Systematic reviews of the literature involve rigorous methods analogous to primary research studies. Investigators collect, analyze, and interpret data in an explicit, reproducible manner to avoid bias.

• Meta-analysis involves statistical pooling of data derived from multiple studies. To avoid bias in data selection, meta-analyses should be based on an underlying systematic review.

• Systematic reviews and meta-analyses strengthen the evidence base in Facial Plastic Surgery. Functional rhinoplasty, facial reanimation, facial reconstruction, and wound healing are among several areas with potential for enhancing level of evidence.

• In Facial Plastic Surgery, accruing well-designed original studies will improve the data set available for systematic reviews and meta-analyses.

• Current challenges include limited numbers of studies, weaknesses of study design/methods, and inconsistency in outcomes and definitions.