Artificial Intelligence in Plastic Surgery: Current Applications, Future Directions, and Ethical Implications

Abstract

Background:

Artificial intelligence (AI) in healthcare delivery has become an important area of research due to the rapid progression of technology, which has allowed the growth of many processes historically reliant upon human input. AI has become particularly important in plastic surgery in a variety of settings. This article highlights current applications of AI in plastic surgery and discusses future implications. We further detail ethical issues that may arise in the implementation of AI in plastic surgery.

Methods:

We conducted a systematic literature review of all electronically available publications in the PubMed, Scopus, and Web of Science databases as of February 5, 2020. All returned publications regarding the application of AI in plastic surgery were considered for inclusion.

Results:

Of the 89 novel articles returned, 14 satisfied inclusion and exclusion criteria. Articles procured from the references of those of the database search and those pertaining to historical and ethical implications were summarized when relevant.

Conclusions:

Numerous applications of AI exist in plastic surgery. Big data, machine learning, deep learning, natural language processing, and facial recognition are examples of AI-based technology that plastic surgeons may utilize to advance their surgical practice. Like any evolving technology, however, the use of AI in healthcare raises important ethical issues, including patient autonomy and informed consent, confidentiality, and appropriate data use. Such considerations are significant, as high ethical standards are key to appropriate and longstanding implementation of AI.

INTRODUCTION

What is Artificial Intelligence?

For over 50 years, artificial intelligence (AI) has been an important topic of ongoing scholarship. In 1947, Alan Turing envisioned the concept of machines capable of human thought processes, stimulating groundbreaking research in AI and computer programming.^1,2 Later research in the 1960s and 1970s led to one of the most significant early applications of AI in medicine known as MYCIN.³ The MYCIN system was trained to provide a differential for bacterial infections, which would include ranked probabilities for each possible microorganism and the recommended treatment.³ In recent years, there have been strides in technologies designed for effectively diagnosing diseases and predicting outcomes using healthcare data.⁴ It is conceivable that AI will be employed for the majority of diagnostic and decision-making processes in the near future, thus transforming healthcare as we know it.

AI encompasses a broad discipline of computer science that focuses on computer learning using large data sets. Recent progress in AI with regard to computational ability and ever-expanding data sets has been immense. AI can be used to create applications to complete tasks previously requiring a significant human input, improving efficiency in a wide range of fields. Several clinically applicable subdisciplines of AI include machine learning, deep learning, natural language processing, and facial recognition (Table 1).

Table 1.

Major Subdisciplines of Artificial Intelligence

Subdiscipline	Description	Examples	References
Machine learning	Algorithms able to uncover associations in large data sets via pattern recognition among interacting variables. Subcategories include supervised and unsupervised learning.	• Supervised learning: An application tested with photographs to monitor postoperative free flap viability based on skin color.	Noorbakhsh-Sabet et al5; Bogle et al.6; Ebert and Golub7; Knoops et al.8
		• Unsupervised learning: The organization and interpretation of large amounts of unlabeled genetic data without a training set.
Deep learning	Machine learning models that use artificial neural networks to improve predictive performance and accuracy with continued training.	• A deep learning convolutional network to determine rhinoplasty status via photographs.	Borsting et al9; Phillips et al10,11
		• An application capable of identifying melanoma in images of biopsied lesions taken via a smart phone.
Natural language processing	Machine learning software capable of understanding, interpreting, and manipulating human language.	• An AI bot within a smartphone application capable of providing answers to frequently asked questions among preoperative patients.	Mehta and Devarakonda12; Savova et al13; Jokhio et al14; Chopan et al15; Dodds et al16
Facial recognition	AI software capable of recognizing human faces by using biometrics to map facial features and compare the data with a database of photographs.	• Facial recognition neural networks capable of gender-typing transgender women after facial feminization surgery.	Zuo et al17; Chen et al18

Subdisciplines of Artificial Intelligence

Machine learning (ML) describes applications that can take data and uncover associations via pattern recognition among interacting variables.⁵ Common applications of ML include classification and prediction models, which fall under the subcategory of supervised learning. Supervised learning models employ algorithms that are programmed to identify or predict an outcome using training data.¹⁹ One example is a model that has been trained with electronic health record data to predict mortality of sepsis patients.⁶ Facial recognition is subset of supervised ML. This technology has shown the potential to facilitate postoperative satisfaction in aesthetic surgery patients.¹⁹ In contrast, unsupervised learning models require the discovery of novel associations in unlabeled data.¹⁹ An example includes a model that organizes large amounts of genetic data by pattern recognition.⁷

Deep learning (DL) describes ML models that use artificial neural networks (ANNs) based on human brain function. These networks can improve their predictive performance and accuracy in classification with continued training using novel data.^5,9 The layered model structure supports the coupled extraction and analysis of the desired features within a data set.²⁰

Models using natural language processing (NLP) and pattern recognition in unstructured data fall under the category of cognitive computing.⁵ Unstructured data includes information that is generally more difficult to organize and analyze using traditional methods of statistical analysis, such as auditory and visual data.⁹ Cognitive computing aims to complete tasks requiring the integration and organization of unstructured data to make decisions.

The technology underlying AI is progressing rapidly. Support from the scientific community and the federal government has generated the potential for AI to change healthcare delivery in profound ways.²¹ Gradually, AI has been applied in plastic surgery, although limited literature exists. In this article, we summarize the application of AI in plastic surgery, highlighting current work and future directions. Additionally, we discuss important ethical implications concerning the use of AI in healthcare.

METHODS

Literature Review

We conducted a systematic search of all electronically available publications as of February 5, 2020. Two authors (TJ and CT) independently performed the review to ensure accuracy and completeness. Publications that focused on the application of AI in plastic surgery were evaluated for inclusion.

Search Strategy

A search was performed in 3 databases: PubMed, Scopus, and Web of Science. A review of citations in all procured articles from the database search was performed for publications from 2000 to 2020. Keywords used for the search included: artificial intelligence, machine learning, natural language processing, big data, nanotechnology, and plastic surgery. A diagrammatic representation of the search scheme is displayed in Figure 1. A secondary search was performed to identify articles that highlight ethical and historical considerations for AI in healthcare.

Fig. 1.

Diagrammatic representation of the literature search performed.

Inclusion Criteria

All studies returned from the database queries utilizing the keywords were reviewed for applicability. Based on an abstract screening, publications were included if the content centered on the application of AI in plastic surgery. Due to the limited number of articles currently available, no restrictions were made based on publication type, quality, study methods, or risk of bias. Only published articles were included in the review.

Exclusion Criteria

A primary abstract screen was performed to exclude any article not pertaining to the application of AI in plastic surgery. A secondary screen excluded articles unavailable in English or without full text access by our institution’s library.

RESULTS

The initial literature search identified 129 articles. Of those identified in the initial search, 89 were novel articles. Upon review of titles and abstracts, 65 did not address the application of AI in plastic surgery and were therefore excluded. Of the remaining 24 articles, a full text review excluded 10 articles that were either non-English or unavailable for full-text review. The remaining 14 articles that satisfied the inclusion and exclusion criteria were considered and are addressed below (Table 2). Relevant articles yielded from review of citations in all procured articles were also included for discussion.

Table 2.

Article Characteristics

Reference	Year of Publication	Journal of Publication	Aspects of AI Addressed
Kanevsky et al19	2016	Plastic and Reconstructive Surgery	• Big data
			• Machine learning
Borsting et al9	2020	Journal of Craniofacial Surgery	• Deep learning
Cardoso et al20	2020	The Breast	• Machine learning
			• Deep learning
Zhu et al22	2016	Annals of Plastic Surgery	• Big data
Kim et al23	2019	Plastic and Reconstructive Surgery – Global Open	• Big data
Phillips et al10	2019	JAMA Network Open	• Deep learning
Jokhio et al14	2015	Mehran University Research Journal of Engineering and Technology	• Machine learning
			• Natural language processing
Chopan et al15	2019	Plastic and Reconstructive Surgery	• Natural language processing
Levites et al24	2019	Plastic and Reconstructive Surgery	• Machine learning
			• Natural language processing
Boczar et al25	2020	Annals of Plastic Surgery	• Natural language processing
Zuo et al17	2019	Plastic and Reconstructive Surgery	• Facial recognition
Liang et al30	2020	Aesthetic Plastic Surgery	• Big data
			• Machine learning
			• Deep learning
			• Natural language processing
Koimizu et al31	2019	Plastic and Reconstructive Surgery – Global Open	• Machine learning

DISCUSSION

Big Data

Recent advances in technology combined with evidence-based medicine has brought ever-increasing amounts of data, which have become the focus of a great deal of AI research. Big data research is centered on large databases produced by investigators and clinicians as well as large aggregates of smaller data sets within electronic health records.²² Such databases include the National Surgical Quality Improvement Project,³² the Tracking Operations and Outcomes for Plastic Surgeons Program, and CosmetAssure, which collect outcomes data for adult surgical cases, plastic surgery procedures, and cosmetic procedures, respectively.

Big data analytics using AI models such as ANNs can assist investigators in efficiently analyzing the massive amount of information contained within these databases to answer clinically significant questions.²³ Esbroeck et al demonstrated the use of ML in determining relationships between current procedural terminology and postoperative complications to produce a procedural risk score.³³ The authors trained a support vector machine to determine procedural complexity and risk using data from the National Surgical Quality Improvement Project that were comparable to other known measures of procedural complexity.³³

Machine Learning

Applications of ML in plastic surgery include predictive models that use the pattern-recognition abilities of ANNs to help surgeons make preoperative and postoperative decisions. In the early 2000s, Yeong et al developed a model that used data obtained from a portable reflective spectrophotometer to determine burn depth and healing time, with an average accuracy of 86%.²⁶ The ANN was able to differentiate between burns that would heal before or after 14 days, with an accuracy of 96% and 75%, respectively.²⁶ More recently, an application that monitored postoperative free flap viability based on skin color assessed via the photographs of a Samsung Galaxy S2.²⁷ Photographs of subjects’ hands under different degrees of venous and arterial occlusion were used to train the application which was able to accurately assess the vascular status of new subjects with a sensitivity and specificity of 94% and 98%, respectively.²⁷

Another predictive ML model was developed to predict the outcome of various nerve grafts with an accuracy exceeding 90%, demonstrating the potential utility of ML models in peripheral nerve surgery.²⁸ The researchers trained an ANN using over 30 variables identified from experimental records of nerve graft studies on rats.²⁸ Categories of variables included the biomaterials used for the wall and filling, the extracellular matrix proteins, growth factors, scaffold type, and surface.²⁸

In the near future, ML systems are likely to facilitate an early diagnosis of a wide range of conditions by streamlining the extraction and analysis of clinical data. An ML algorithm was trained to diagnose distinct types of craniosynostosis based on CT images at a level comparable to a trained radiologist with a sensitivity of 92.7% and a specificity of 98.9%.²⁹ In recognition of the subjectivity of the mental visualizations typically relied on by surgeons to correct such malformations, the authors hope their model will assist surgical planning by characterizing dysmorphology in a more precise manner.²⁹

Another supervised ML model was demonstrated to aid surgical planning via automated diagnosis and simulation.⁸ Investigators trained a 3D morphable model, a statistical model of face shape and quality, using databases containing 10,000 three-dimensional face scans of both healthy volunteers and orthognathic surgical patients.⁸ Three separate models were produced: a global model containing the faces of healthy volunteers and preoperative patients, a preoperative patient model, and a postoperative patient model.⁸ The models were able to differentiate orthognathic patients from healthy volunteers, as well as predict patient-specific postoperative face shape using regression analysis.⁸ The authors propose that their model can help surgeons in the objective assessment of preoperative and postoperative aesthetics, improving patient education and surgical planning.⁸

Deep Learning

Plastic surgery research is well suited for DL applications due to the abundance of unstructured, visual data collected via widely available technologies.⁹ Aesthetic surgeons routinely collect before and after images for procedures, which are subsequently made publicly available, creating a large source of data. Using a publicly available repository containing over 18,000 before and after rhinoplasty photographs, a neural network could correctly classify rhinoplasty status in 85% of the tested images, performing at a level of sensitivity and specificity equivalent to ENT and plastic surgery residents and attendings.⁹

Another DL application developed by Phillips et al was used to identify melanoma in images of biopsied lesions taken on smartphones with dermoscopic lens attachments.¹⁰ The Deep Ensemble for Recognition of Melanoma ANN is capable of learning features of malignant melanoma directly from the data given, rather than from features preset by the investigators.¹¹ When the application had been tested using published dermoscopic images, it assessed the likelihood of melanoma in the images at a level of accuracy similar to that of the clinicians.¹⁰ The study highlights the potential for a DL application to help clinicians in secondary prevention of skin cancer and to serve as a diagnostic and decision-making tool.¹⁰

Natural Language Processing

Over the last 20 years, we have seen a boom in the regular use of NLP algorithms with applications such as spelling and grammar check in word processers, autocorrect, and predictive text messaging.¹² Just 10 years ago, Mayo Clinic researchers developed the Text Analysis and Knowledge Extraction system, an NLP framework that could root out data from text within electronic health records.¹³ Recent applications of NLP in plastic surgery stem from a trending interest in the analysis of the public opinions regarding the field and an abundance of data within the Twitter microblogging system.¹⁴

Plastic surgeons in aesthetics have increasingly taken to using social media platforms to market their practices and provide patient education. To gauge the public perception of plastic surgery, Mustafa and colleagues used an NLP technique called hedonometrics to analyze tweets regarding plastic surgery occurring between 2012 and 2016.¹⁵ Hedonometrics is a procedure that uses an algorithm to quantify happiness based on text.¹⁶ The data set contains over 10,000 words pulled from Amazon Mechanical Turk.^15,16 They developed word-shift graphs to summarize the major shifts in overall word happiness and the words that caused them.¹⁵ After analyzing 1,037,146 relevant tweets, they found that the term “plastic” was the most popular and had the lowest positivity score.¹⁶ The terms “aesthetic,” “cosmetic,” and “reconstructive” were less popular, though more positively regarded.¹⁶ A similar study by Cognovi Labs and Duke University used supervised ML technology to examine commonly hashtagged words associated with plastic surgery.²⁴ The authors suggest their results might inform decisions to use the title of aesthetic or cosmetic surgeon and emphasized the potential for such applications to influence marketing strategies and the public perception of plastic surgery.^16,24

Recognizing the potential utility of a smartphone in addressing frequently asked questions in plastic surgery clinics, Boczar et al developed a smartphone application trained to provide answers to questions within 10 chosen topics of frequent concern to preoperative patients.²⁵ Participants asked the application questions from each of the chosen topics in their own words.²⁵ After a training period, the application provided an appropriate answer 92.3% of the time, with participants determining the answer to be accurate 83.3% of the time.²⁵ The authors anticipate the integration of the technology into clinical practice to improve patient support and free up time for surgeons.²⁵

Facial Recognition Technology

The commercial use of facial recognition technology is becoming commonplace with an increasing number of people using smartphones employing the technology. Facial recognition operates using pattern recognition models combined with image analysis and deep neural networks to take unique biometric measurements that are used to interpret facial characteristics.¹⁷ One model was able to classify facial beauty in patients relative to postoperative target features, which may be beneficial in estimating patient satisfaction and setting appropriate expectations before surgery.¹⁹ Further applications demonstrate promise in the diagnosis of developmental disorders that express characteristic facies and assessing the success of facial feminization surgery.^17,18

Acknowledging the desire for male-to-female transgender patients to be seen and treated as their identified gender, Chen et al trained several neural networks to identify gender based on facial features.¹⁸ Images of patients who underwent male-to-female facial feminization surgery were taken along with cisgender male and cisgender female control images.¹⁸ Four convolutional neural networks from Amazon, IBM, Microsoft, and Face++ were trained using these images to classify gender.¹⁸ The control male images were correctly classified 100% of the time, while control females were correctly identified 98% of the time.¹⁸ Preoperative images of the transgender patients were correctly classified as female 53% of the time, while postoperative images were correctly classified 98% of the time.¹⁸ The authors point out that the software successfully provided an objective measure of the efficacy of the procedures, fulfilling a recognized gap in the outcome measures of gender confirmation surgery.¹⁸

Future Directions

In early 2019, the American Artificial Intelligence Initiative was enacted, facilitating a national strategy to make the United States a leader in AI. This led to a $142.2 billion investment in research and development for AI for the 2021 fiscal year.²¹ Later that year, the FDA offered a regulatory framework for the use of AI-based software as a medical device, citing the potential for such technology to transform healthcare by taking advantage of the big data generated via the everyday delivery of modern healthcare.³⁴ With the increasing national awareness of the potential AI offers, it is becoming more important to see that its applications in healthcare are demonstrated.²³

Plastic surgery is one of the many specialties that have the capacity to make use of AI in its full potential. The plastic surgeon’s usual cognitive tasks of diagnosis, case planning, and perioperative assessment could be streamlined by thinking machines, allowing for increased productivity and patient satisfaction.^23,25 The deductive reasoning that drives surgical decision-making depends on the ability of the surgeon to produce an adequate list of differential diagnoses, to consider the best tests to establish the diagnosis, and to develop a plan to address the diagnosis using educated judgment and heuristic techniques.³⁵ AI-powered decision-making tools show tremendous potential in improving surgical outcomes by augmenting these processes via automated data acquisition, predictive analytics, and appropriate integration with human surgical intuition.³⁴

Ethical Considerations

Kohli and Geis named 3 domains of ethical issues regarding the use of AI: algorithms, data, and practices (Table 3).³⁷ They raise the issue of informed consent and the need for data use agreements on behalf of data providers and third-party data aggregators. Quality assurance of data used in AI algorithms, particularly when the intent is to augment patient treatment decisions, is also of concern. The data sets used to train the ML systems must be representative of the population for which it is intended. Providers must make efforts to optimize the data and algorithms to best benefit their patients.³⁰

Table 3.

Ethical Considerations of AI in Plastic Surgery

Ethical Issue	Description	Examples	Reference
Informed consent regarding the use of data	The need for data-use agreements on behalf of data providers and aggregators	• Consent for the use of patient data within an HER	Kohli and Geis37
		• Consent for the use of patient photographs in a training data set
Quality assurance of data	The need for high quality data that represent the patient population for which the AI system is intended	• Inclusion of people of different ethnicities in facial recognition and other AI systems that are dependent on visual data	Liang et al30
		• Provider awareness of biases that may stem from the data set
Integrity of the patient–physician relationship and the human dimension of health care	The need to assure that AI does not compromise the patient–physician relationship centered on trust, empathy, and shared decision-making process	• The integration of AI systems into patient-centered clinical practice	Koimizu et al31–; Aminololama Shakeri and López 38; Krittanawong39; Ahuja40
		• The automation of clinical tasks

The use of AI should not serve as a replacement for the shared decision-making process that is essential to ideal patient care. Providers taking advantage of this technology must be careful not to allow a biased view to disturb shared decision-making, considering the limitations imposed on such technology by its data sets.³¹ An example of such bias may include the use of facial recognition systems in aesthetic practices. The use of data sets developed in Western or wealthy Eastern countries may lead to biased suggestions that can marginalize values and perceptions of beauty in other cultures.³¹

An intuitive concern when considering the implementation of AI is that the patient–physician relationship will become compromised. Some speculate that AI may completely replace certain specialties, though many physicians anticipate the opposite effect. In breast-imaging radiology practices, physicians are expected to accurately diagnose lesions, relate imaging features to prognostic data, and relay available treatments to patients all while considering their personal preferences.³⁸ These conversations between doctors and patients are both sensitive and complex, yet there is very little time for them.³⁸ With AI to automate repetitive tasks such as screenings, examination reporting, and breast density assessments, there is more time for rapport-building and the facilitation of these complex interactions.³⁸

Krittanawong points out that AI is unable to engage in such high-level conversation with patients, nor is it able to build the trust and sense of empathy required to gain the therapeutic alliance that is integral to the patient–physician relationship and positive outcomes.³⁹ Among the primary goals of the implementation of AI in healthcare is the reduction of physician burnout via the automation of routine clinical tasks, therefore enabling physicians to spend more time on the more sophisticated and humane aspects of their practices. Many experts would agree that the purpose of AI in healthcare is to optimize a physician’s practice, rather than to replace it.⁴⁰

CONCLUSIONS

It is evident that AI is making its way through various aspects of the public and private sectors. The emergent federal support and the growing body of published literature demonstrating its potential to increase the efficacy and precision of healthcare make it imperative for researchers to further uncover the benefits of AI. The field of plastic surgery must take advantage of the widely available technologies, big data, and the policymakers’ recent interests in supporting the development of AI to see that it continues to be a leader of innovation in medicine and surgery.