Quality of Surgical Outcomes Reporting in Plastic Surgery: A 15 Year Analysis of Complication Data

Abstract

Background

Postoperative complication data are integral to assessing patient outcomes and identifying areas to improve quality in surgical care. Accurate appraisal of surgical techniques requires consistency and reliability in complication data reporting. The purpose of this study was to analyze the quality of complication reporting in plastic surgery.

Methods

The authors critically reviewed the literature from January 1st, 2000 to December 31st, 2014 to identify articles reporting surgical outcomes after 3 index procedures (autologous breast reconstruction, prosthetic breast reconstruction and reduction mammaplasty). Studies were extracted from the journals Plastic and Reconstructive Surgery and Annals of Plastic Surgery. Two authors independently analyzed data using a modification of established criteria for complication reporting that incorporates 10 critical elements.

Results

296 articles reporting outcomes for 299,819 procedures in 249,942 patients were analyzed. Of the 10 reporting criteria, no articles met all criteria, < 1% met 9, 16% met 7 to 8, 43% met 5 to 6, 35% met 3 to 4, and 6% met 1 to 2 (mean 5 criteria met). Commonly under-reported criteria included: complication definitions (37% of articles reported), aesthetic or patient reported outcome (28%), and complication severity (16%). Only 46 studies (16%) reported complication severity, with 15 different definitions of what constituted a “major” complication. Risk factors for complications were absent in 37% of articles.

Conclusions

Inconsistency in reporting complications in the plastic surgery literature confounds the comparison of surgical outcomes. The use of standard guidelines to accurately, efficiently, and reproducibly report complication data is essential for quality assurance and improvement.

INTRODUCTION

Improving patient safety and quality in healthcare is integral to the foundation of modern medicine. In 2001, the Institute of Medicine concluded the American system is in need of fundamental change to improve the quality of health care.¹ Since that time, there has been a concerted effort to redefine and implement a system that is safe, effective, patient-centered, timely, efficient and equitable. As a result, healthcare is transforming into a value-based system, defined as health outcomes achieved per dollar spent.² A critical component of a value-based system is the ability to accurately and efficiently measure, report, and compare outcomes.^2,3

In surgical care, the incidence and nature of complications are the most frequently measured and reported outcomes.^4–6 As an innovative, constantly evolving specialty, plastic surgery is subject to the frequent introduction of operative techniques and technologies. Postoperative complication data are integral, and often the primary means to accurately appraise these disparate surgical techniques, compare outcomes based on intervention performed, and compare outcomes between different institutions and surgeons. Furthermore, there is an increasing demand from payer organizations, hospital systems, policy makers, and patients for objective, comparative complication data to assess surgical outcomes and target areas to improve quality.³ Therefore, consistency and reliability in complication data reporting becomes an imperative in improving the quality and safety of surgical care.

To date, little is known about the consistency and reliability of complication data reporting in plastic surgery. Thus, the purpose of this study was to analyze the quality of surgical complication reporting in the plastic surgery literature.

MATERIALS AND METHODS

Study Identification

A review of the plastic surgery literature using the MEDLINE and EMBASE bibliographic databases was performed independently by two authors to identify all articles reporting surgical outcomes after three commonly performed procedures (autologous breast reconstruction, prosthetic breast reconstruction, and reduction mammaplasty). These operations were chosen as a convenience sample because they are well represented in the literature and commonly performed by surgeons at different institutions around the world. Controlled vocabulary terms and keywords were used in the search strategy and articles were extracted from the journals Plastic and Reconstructive Surgery (impact factor 3.784, 2016/2017) and Annals of Plastic Surgery (impact factor 1.596, 2016/2017).

Inclusion and Exclusion Criteria

Articles published from January 1^st, 2000 to December 31^st, 2014 with study populations of more than 75 patients were included. We chose to review articles published after 1999 to restrict our analysis to current practice and because it coincided with the Institute of Medicine’s report, To Err is Human: Building a Safer Health System, which brought the quality improvement movement to prominence.⁷ Retrospective and prospective clinical articles were identified. All duplicate articles were removed and database limits were utilized to exclude pediatric and anatomic/cadaver articles. All articles designed to report only a single outcome measure were excluded. Review articles, case-reports, abstracts without full text, and letters to the Editor were also excluded.

Data Extraction

All articles were screened and analyzed independently by two reviewers with any differences reconciled by discussion. If differences could not be resolved, the senior author (TMM) served as tiebreaker. Data were independently extracted utilizing a standardized abstraction form and data fields were predetermined to capture all descriptive and outcomes data. Descriptive data included: author, journal, topic, date of publication, institution of publication/study location, study design, level of evidence (if reported), number of patients, number of operations, duration of follow-up, and risk factors/comorbidities. Outcomes data included: complications, readmissions, return to work, quality of life and patient-reported outcomes, aesthetic results, and cost data.

Critical Appraisal & Data Analysis

Articles were reviewed and appraised using a modification of previously published criteria related to the completeness of surgical complication reporting.⁸ This criteria included 10 critical components for article evaluation (Table 1). Inclusion of these components in an article allows the reader to gain a comprehensive understanding of outcomes and complications following a surgical intervention. As the initial development of this criteria was in the surgical oncology literature, an exact translation to the plastic surgery literature could potentially include irrelevant categories and exclude relevant categories of interest for plastic surgeons; therefore, modifications were made, when necessary, to the original criteria. Specifically, the category of “mortality rate and causes of death” was eliminated as this outcome is rare in plastic surgery operations. In exchange, the category “patient-reported outcomes and/or aesthetic outcomes” was created as these are common outcomes of interest following plastic surgery operations.

Table 1.

Criteria to evaluate the quality of complication reporting in the plastic surgery literature (modified from Martin RCG, Brennan MF, and Jaques DP. Quality of complication reporting it the surgical literature. Ann Surg. 2002; 235:803-813.⁸)

Criteria	Requirement
Study design/method of accruing data	Prospective or retrospective accrual of data and study design are indicated
Duration of follow-up indicated	Report clarifies time period of postoperative accrual of complications
Outpatient information included	Study indicates that complications first identified following discharge are included in analysis
Definitions of complications provided	Article defines at least one half of complications listed with specific inclusion criteria
Patient-reported outcomes or aesthetic result included	Article includes any data for patient reported outcomes or aesthetic result (subjective or objective)
Morbidity rate and total complications indicated	The number of patients with any complication and the total number of complications are recorded
Procedure-specific complications included	Includes at least half of the following complications for each procedure reported: Breast Reduction: infection, skin necrosis, nipple necrosis/loss, hematoma, seroma, and dehiscence Breast Reconstruction (Autologous): infection, fat necrosis, skin necrosis, dehiscence, flap loss, anastomotic complication, hematoma, seroma, and donor-site complications Breast Reconstruction (Prosthetic): infection, contracture, necrosis, dehiscence, implant exposure, implant loss, implant deflation/rupture, hematoma and seroma
Severity grade utilized	Any grading system designed to clarify severity of complications is reported
Length-of-stay data reported	Median or mean length of stay indicated in the study
Risk-factors included in analysis	Evidence of risk stratification and method used indicated in study

Summary statistics were tabulated via established norms. Student’s t-test was used to compare sample distributions of quality metric by article type. Pearson product-moment correlation was used to analyze the linear association between article quality score over time. Alpha = 0.05 indicated significance in all tests. All analysis was performed via commercially-available STATA v14.0 (StataCorp LP, College Station, TX).

RESULTS

A total of 4,543 articles were identified on initial search of the MEDLINE and EMBASE bibliographic databases with the aforementioned limits applied (Figure 1). After excluding duplicates, titles and abstracts were assessed for relevance to the inclusion criteria. Articles that met selection criteria were extracted for full-text review. The kappa coefficient on the level of agreement between the two reviewers was 0.85 for title and abstract screening and 0.80 for full-text article screening. After full-text review, a total of 296 articles were included for analysis and data extraction.

Figure 1.

PRISMA flow diagram of articles screened and selected for evaluation.

Description of Included Studies

The 296 articles identified included outcomes for 299,819 procedures in 249,942 patients (See Appendix 1, Supplemental Digital Content 1, for a Reference list of all articles included for critical analysis regarding the completeness of surgical outcomes and complication data reporting in the plastic surgery literature from 2000 to 2014, INSERT HYPER LINK). There were 3 randomized control trials, 219 retrospective studies, 35 prospective studies, and 3 cross-sectional studies (36 articles did not specify method of accrual). There were 123 articles (42%) pertaining to autologous reconstruction, 74 articles (25%) pertaining to prosthetic (tissue-expander/implant based) reconstruction, 46 articles (15%) pertaining to both autologous and prosthetic reconstruction, 52 articles (18%) pertaining to breast reduction, and 1 article (<1.0%) pertaining to breast reconstruction and breast reduction. The journal was Plastic and Reconstructive Surgery for 203 articles (69%) and Annals of Plastic Surgery for 93 articles (31%). When reported (69 out of 296 studies, 23%), the mean level of evidence was 3.1. Patient follow-up duration, reported in 187 studies (63%), was a mean 15.2 months (range: 3 days to 6.6 years) and a median of 10.0 months post-operatively.

Critical Appraisal

Of the 10 criteria related to the completeness of surgical outcomes and complication data reporting, no article met all criteria (Figure 2). Only 1 article (<1.0%) met 9 of the criteria and 47 articles (16%) met either 7 or 8 of the criteria. Additionally, 128 (43%) articles met 5 to 6 criteria, 103 (35%) met 3 to 4 criteria, and 17 (6%) met 1 to 2 criteria. Overall, there was a mean of 5 criteria met and a median of 5 criteria met. There was no statistical difference in mean criteria met based on the operation performed (Table 2). Furthermore, there was no significant linear trend between the quality of articles, as reflected by the mean criteria met, over time (m=0.03, p>0.05) (Figure 3).

Figure 2.

Completeness of surgical outcomes and complication data reporting in a convenience sample of the plastic surgery literature from 2000-2014.

Table 2.

Mean quality criteria met compared by surgical operation reported in articles.

	Autologous Breast Reconstruction (n = 123)	Prosthetic Breast Reconstruction (n = 74)	Autologous and Prosthetic Breast Reconstruction (n = 46)	Breast Reduction (n = 52)	P-value
Quality Criteria Met (Mean ± SD)	4.92 ± 1.49	5.31 ± 1.52	4.78 ± 1.49	4.71 ± 1.66	0.12

Figure 3.

Analysis of mean quality criteria met over time from 2000 to 2014. Figure demonstrates no significant linear trend between the quality of articles, as reflected by the mean criteria met, over time (m=0.03, p>0.05).

Commonly underreported criteria included: complication definitions (only 37% of articles reported), aesthetic or patient-reported outcome (only 28% of articles reported), and complication severity (only 16% of articles reported). In the 46 studies (16%) reporting complication severity, there were 15 different definitions included as to what constituted a “major” complication. Definitions of a “major” complication varied substantially from “surgical site infection requiring antibiotics” to “surgical site infection requiring operating room.” Furthermore, other than using the “major” versus “minor” grading system, no articles used validated or frequently referenced grading systems.^9–11

In regards to definitions, there was substantial variability for when and how certain complications were defined. For example, surgical site infection was not recorded in 65 articles (22%). When infection was reported, it was listed, but not defined, in 133 articles (45%). Furthermore, across articles that did define surgical site infection, there were 33 different definitions used. Examples include: “cellulitis requiring observation,” “cellulitis requiring admission for intravenous antibiotics,” “erythema, fever, or drainage with documentation of positive cultures,” “cellulitis requiring implant or expander removal,” and “erythema, warmth, or induration requiring oral antibiotics.” Similarly, wound healing complications were not reported in 104 (35%) articles and not defined in 88 (30%) articles. When reported, they were defined in 22 distinct ways. Examples of disparate definitions include: “wound healing requiring prolonged dressing changes,” “wound requiring irrigation and debridement,” and “total breakdown of surgical closure compromising integrity of the procedure.”

Commonly, articles did not report the duration of review for complications (not reported in 233, or 79% of, articles). When reported, duration varied substantially. Some authors reported duration as “early” versus “late” complications with little to no consistency in what time periods constituted early versus late. Complications considered to be “early” were defined as occurring during the initial admission period, prior to the first post-operative visit, within 30 days of surgery, within 6 weeks of surgery, within 2 months of surgery, or within 90 days of surgery. Similarly, authors also defined a wide time range for “late” complications, including after initial inpatient admission, after first post-operative visit, after 30 days, after 6 weeks, after 2 months, or after 90 days.

Patient-reported outcomes were reported in 61 articles (21%); however, only 11 of these (18%) utilized an instrument that demonstrated adequate development and validation in the population of interest, such as the BREAST-Q.¹² Additionally, risk factors for complications were absent in 37% of articles. Though not a part of the quality criteria scoring system, we analyzed quality metric and cost data for all articles. Unplanned hospital readmission rates within 30 days were only provided by 19 articles (6.4%), with 17 of those specifying the reason for readmission. Cost data was rarely included in articles, with only 11 articles (3.7%) reporting any cost data or economic analysis.

DISCUSSION

Postoperative complication data are integral to assessing patient outcomes and identifying areas to improve quality in surgical care. Accurate appraisal of surgical techniques requires consistency and reliability in complication data reporting. This can only enhance the ability of plastic surgery to provide value based care. In this critical review of the plastic surgery literature over a 15 year period, we demonstrated there is considerable inconsistency in reporting complications across a variety of relevant outcomes criteria when using three commonly performed procedures. Nearly two-thirds of articles did not adequately define the complications reported in the study. Even when complications were defined, there was substantial variability in the published literature as to what constituted certain complications. For example, there were 33 different definitions for surgical site infection and 22 different definitions for wound healing complications. In addition to inconsistent definitions, there was also limited reporting of complication severity using grading criteria. Only 1 in 6 articles provided data on complication severity. Most articles that did report severity used the simple classification of major versus minor complications; however, authors defined “major” 15 different ways. Appropriate consideration of risk factors for complications was absent in 37% of publications. This inconsistency in complication reporting presents significant challenges to the synthesis and analysis of data for operative interventions and techniques and ultimately confounds the comparison of outcomes across studies.

In the last 15 years, the U.S. National Library of Medicine MEDLINE database has expanded by 9 million articles, with an estimated one paper being added to the medical literature per minute.¹³ This increase in the volume of available literature has included the body of plastic surgery research. While more publications create the opportunity to enhance our understanding on a variety of topics, it also increases the burden on clinicians to synthesize, analyze, and interpret data to provide optimal, evidence-based care to patients. Over the last several years, led by the editorial board of Plastic and Reconstructive Surgery journal and other major journals in our specialty, there has been a concerted effort to implement evidence-based medicine principles into the plastic surgery literature in an effort to provide the best evidence to guide clinical decision making and improve the ability of readers to translate research findings to clinical practice.^14–16 This has led to the inclusion of the level of evidence for an article, an important first step which reflects the study design and methodology quality. Unfortunately, even though an article may have a high level of evidence and appropriate study methodology, if the reporting of complication data is inadequate, it precludes surgeons from effectively comparing outcomes from the intervention performed, and diminishes the ability to draw meaningful conclusions about the applicability (a component of evidence-based medicine) of the results presented.¹⁷ Furthermore, variability in reporting outcomes data makes it difficult to perform meta-analyses studies. This is an important limitation because meta-analyses studies often have a significant impact in the literature, frequently receiving more citations than other study methodologies.¹⁸ As a result, authors have argued that an increase in meta-analyses studies in the plastic surgery literature would result in a greater impact on patient outcomes.¹⁹ Consistency in complication and outcomes reporting would facilitate the comparison of data across studies and enable authors to perform these studies. This in turn will improve the value of plastic surgery care as evidence-based medicine is implemented in practice.

While substantial progress has been made in improving outcomes reporting in plastic surgery, our study demonstrates that additional efforts are needed to ensure plastic surgery remains at the forefront in producing high quality, impactful publications that improve patient care. Our study builds on the work of others who have also demonstrated limitations in the way outcomes and complications are reported in the surgical literature. In 2002, Martin et al. reported there was a breadth and depth of limitations in the reporting of surgical complications and outcomes for surgical oncology procedures.⁸ In 2007, Donat demonstrated disparities in the quality of complication reporting after urologic oncology procedures. Potter et al., using similar quality grading criteria, expanded this to breast reconstruction studies published before 2009 and confirmed similar inconsistencies and lack of methodological rigor in complication reporting.²⁰ Furthermore, in 2007, Sears and Chung critically assessed outcomes studies over a 17 year time period in the journals Plastic and Reconstructive Surgery and Annals of Plastic Surgery using the Agency for Healthcare Research and Quality’s outcomes impact scale and concluded that “outcomes research can have a greater impact on…patient outcomes…by pursuing studies that demonstrate direct improvement in patient outcomes with one treatment compared with clinical standards.” This study confirms and expands on these previous publications. Despite the recent emphasis on improving outcomes reporting in plastic surgery, our analysis revealed the quality of complication data reporting did not improve significantly over time. In fact, in the 6 years subsequent to the publication by Potter et al., minimal improvements in the consistency of complication reporting have occurred. This presents a significant challenge, as well as an opportunity, in the implementation and practice of evidence-based medicine in plastic surgery.

This study, along with others, demonstrates the need for the creation and widespread use of standard guidelines to accurately, efficiently, and reproducibly report complication data for core plastic surgery procedures. Standardized reporting of complications is essential for quality assurance and quality improvement. In the surgical literature, two areas that have received substantial attention in regards to complication reporting are 1) measuring the severity of complications, and 2) developing standardized outcome measures and definitions for complications. Grading systems to characterize the severity of complications have been developed and gained widespread acceptance for reporting outcomes following procedures in general surgery, urology, and otolaryngology.^11,21–26 Commonly utilized grading systems are the Clavien-Dindo scale, the Accordion Classification, and the Comprehensive Complication Index.^11,25,27 Each of these severity grading systems is intended to differentiate complications, that may otherwise be considered identical if simply reported, based on the severity of therapeutic intervention required. In the Accordion and Clavien-Dindo scales, complications are ranked by levels or grades of severity and range from deviations from the normal postoperative course managed with pharmacological treatment to complications requiring intervention under general anesthesia and/or life-threatening complications requiring intensive care management. Expanding or adapting these classification systems and quantitative severity weighting of complications to plastic surgery procedures is logical and essential for accurate evaluation and comparison of outcomes. The value of complication severity grading can be seen in a simple example. If two separate studies report “surgical site infection” as a postoperative complication following prosthetic breast reconstruction procedures, readers will assume the authors are reporting a similar complication and potentially compare these data to other published reports. However, as all clinicians are aware, there is a wide spectrum of severity for postoperative infections. A “surgical site infection” that requires a phone call to clinic and oral antibiotics is obviously different than a “surgical site infection” that requires hospital admission for intravenous antibiotics and return to the operating room for prosthetic explantation. Without weighting and reporting the severity of complications, readers would not be able to make this differentiation and therefore, would not be able to accurately compare outcomes across studies.

Of equal importance to reporting the severity of complications is the development and utilization of standardized definitions for complications and the consistent reporting of standardized outcomes. An example of this is the development and widespread utilization of the BREAST-Q for standardized reporting of patient-reported outcomes in the literature.¹² Similar efforts are needed to standardize reporting of other outcomes and complications in plastic surgery. In the United Kingdom, Potter et al. previously developed a core outcome set for reconstructive breast surgery that includes 11 key outcomes that all studies should report on.²⁸ While the lack of specific definitions for the outcome domains is a key limitation, this work was an important first step towards standardized outcome reporting. Unfortunately, adoption of this core outcome set has been sparse in the United States. Additional attempts to standardize complication reporting have come from the use of administrative databases and quality improvement program databases such as the NSQIP. However, prior studies have demonstrated NSQIP is often inadequate for comprehensive complication reporting in plastic surgery.²⁹ One positive of the NSQIP, though, is that complication duration is quantitatively characterized (within 30-days) versus qualitatively characterized (“early” versus “late”). As our results indicated, the duration of complication surveillance and definitions of “early” versus “late” complications varied substantially. A move to replacing qualitative with temporal or quantitative terms is a positive step. Similar to the NSQIP, using ICD-9 codes for complication reporting, as is common with studies utilizing administrative databases, is also limited by potentially inaccurate and incomplete data.³⁰ Thus, it is imperative that future efforts focus on developing standardized outcome measures with consistent definitions for complications. This process can be led by our speciality societies, a collaborative group of researchers, and/or major journals in our specialty. In urology and neurosurgery, the efforts for developing consensus guidelines for reporting and defining complications were led by major societies within each specialty.^31,32 It is our opinion that the optimal process would involve buy-in from all stakeholders, including authors, journal editorial leadership, and societal leadership. Ultimately, it is to everyone’s benefit to improve the quality of complication reporting in plastic surgery. A model for this type of collaborative effort already exists in plastic surgery, as exemplified by the adaptation and implementation of evidence-based guidelines and the levels of evidence in our literature.^14–16 For publications, this information could be reported in the methods section of manuscripts. Once publicized, investigators performing clinical studies can also be more consistent in capturing data using standardized outcome measure sets.

This study is not without limitations. Our analysis was limited to only two journals and three operations, therefore, selection bias may limit the generalizability. Additionally, our analysis extended until the end of 2014. While we evaluated for changes in quality scores over time and found no significant differences, more recent articles may reflect improvements in complication reporting not captured in this study. A further limitation is in the evaluation criteria used. Though this criteria was previously published and utilized in several studies to evaluate complication data, we acknowledge it is not universally accepted. Unfortunately, while checklists such as the STROBE statement exist for evaluating what items should be reported in different types of studies, there are no universally accepted guidelines for evaluating the quality of complication reporting.³³ Thus, we chose to use this particular criteria and modify it because it represents a practical and easy to understand way to characterize the quality of complication reporting. We also acknowledge that certain criteria are not always relevant. For example, it is not always possible to account for risk factors for complications if the relationship between the covariate and the outcome has not previously been established. Also, patient-reported outcomes are incredibly valuable metrics in certain studies, but may not have relevance in other studies.

CONCLUSIONS

This study demonstrates the presence of inconsistencies in the reporting of complication outcomes in the plastic surgery literature. The variability in the three index operations explored in this study confounds the comparison of surgical outcomes across studies, making implementation of evidence-based medicine and improvements in value based care difficult. The development, adoption, and widespread use of standard guidelines to accurately, efficiently, and reproducibly report complication data is essential for quality assurance and improvement in plastic surgery.