Evolution, current challenges, and future possibilities in the objective assessment of aesthetic outcome of breast cancer locoregional treatment

Abstract

The Breast Cancer overall survival rate has raised impressively in the last 20 years mainly due to improved screening and effectiveness of treatments. This increase in survival paralleled the awareness over the long-lasting impact of the side effects of treatments on patient quality of life, emphasizing the motto “a longer but better life for breast cancer patients”. In breast cancer more strikingly than in other cancers, besides the side effects of systemic treatments, there is the visible impact of surgery and radiotherapy on patients’ body image. This has sparked interest on the development of tools for the aesthetic evaluation of Breast Cancer locoregional treatments, which evolved from manual, subjective approaches to computerized, automated solutions. However, although studied for almost four decades, past solutions were not mature enough to become a standard.

Recent advancements in machine learning have inspired trends toward deep-learning-based medical image analysis, also bringing new promises to the field of aesthetic assessment of locoregional treatments. In this paper, a review and discussion of the previous state-of-the-art methods in the field is conducted and the extracted knowledge is used to understand the evolution and current challenges. The aim of this paper is to delve into the current opportunities as well as motivate and guide future research in the aesthetic assessment of Breast Cancer locoregional treatments.

Highlights

• •Past, present and future of objective assessment of aesthetic outcome of breast cancer treatments.
• •Critical discussion of the role and impact of deep-learning-based methodologies in the field.
• •Recommendation for future efforts in the objective assessment of aesthetic outcome of breast cancer treatments.

1. Introduction

According to Globocan, 2018 witnessed about 2.1 million new breast cancers (BC), accounting for almost 1 in 4 cancer cases among women [1]. BC is the most frequently diagnosed cancer in the majority of the countries worldwide and is also the leading cause of cancer death in over 100 countries. However, BC is an increasingly treatable disease, and 10-year survival now exceeds 80% in most high-income countries. Given this high rate, survivorship issues have become a critical concern, especially the ones with impact in long lasting patient Quality of Life (QoL). The locoregional (LR) treatments for BC (surgery and radiation therapy) are undertaken by the majority of BC patients and usually have a significant impact on body image. In case of a poor aesthetic result, women will have to live with the potential disfiguring aesthetic consequences of their LR intervention. Both treatments, surgery and radiotherapy, can individually impact the aesthetic outcome and, when combined, there is usually an added effect that will eventually worsen the final aesthetic outcome. In general, it is estimated that 30% of all women submitted to LR treatment have a fair/poor aesthetic outcome with the consequent negative impact in psychosocial recovery and QoL [2].

Concerning LR treatments - besides the oncological criteria (re-interventions, recurrences), there are not standard available tools to evaluate the quality/impact regarding aesthetic outcome. Nowadays, due to better screening and optimized treatments, locoregional recurrences and reinterventions have almost universally attained the optimal goals, but aesthetic results need also to be improved as a way to guarantee a better QoL and also as a standard measure to allow Breast Units to audit outcomes and improve LR approaches whenever needed.

As an example, classic breast conservation surgery, according to EUSOMA, should represent 70–80% of surgeries for small breast cancers until 3 cm and reinterventions in these cases should not exceed 10% [3,4]. To be granted certification, all European accredited Breast Units must fulfill these mandatory criteria. However, it is easily understood that these values are more frequently attained if larger resections are undertaken, with a subsequent higher negative impact on aesthetic outcome, as asymmetry will be a detrimental feature in these cases. On the other hand, a more limited approach can be at risk of having a higher reintervention rate due to closer margins but will most probably result in better aesthetic outcomes. This fundamental balance between a cleaner resection and a better aesthetic result is a key issue and still very difficult to evaluate due to the lack of a reliable evaluation method (Fig. 1).

Fig. 1.

The balance between optimal oncological and aesthetic outcomes.

Many methodologies have been proposed and studied for the purpose of aesthetic evaluation, which are mainly patient-based [5,6], expert-based [7], and the so-called objective protocols [8,9]. However, none of the methods is recognised as a gold standard.

This research topic has now reached a turning point that deserves to be addressed and discussed. Researchers have recently started to explore diverse deep learning methodologies, which bring significant improvements in robustness, but also raise new challenges regarding data availability. At this turning point, this paper aims to showcase the evolution and current landscape of methods for aesthetic quantification of the LR treatment for BC. After presenting the most significant advances in aesthetic assessment, that deep perspective is used to discuss the most relevant challenges and the most promising future opportunities regarding research.

2. The path already taken

Intuitively, self-assessment through Patient Reported Outcome Measures (PROMs) should be the most valued form of evaluation of aesthetic outcome; unfortunately, in spite of being a valuable measure of patient satisfaction, and hence very important, it has very low reproducibility values when compared to other evaluation methods due to the lack of knowledge patients have about how they are expected to look by the end of treatment and also due to personal factors that can have an impact on this evaluation [10]. Although there is an undeniable truth residing in the fact that the most important outcome should be evaluated by the patient herself, to use this type of evaluation that is inevitably biased would never allow a true evaluation of results, making any analysis and eventual quality control virtually impossible (Fig. 2).

Fig. 2.

The variability of patient self-evaluation.

Trying to overcome the self-assessment problems, the introduction of aesthetic evaluation by experts, through patients’ photographs, has also been frequently used [11,12], especially when a new technique of LR treatment needs to be evaluated. Although a step forward, and frequently used as the gold-standard evaluation, in the absence of a better one, it is still very time consuming, expensive and also presents low to medium reproducibility values [13].

The introduction of objective methods was started by Pezner et al. [14]; in 1985, with the first objective measure to evaluate asymmetry, one of the most important aspects of aesthetics: Breast Retraction Assessment (BRA). The reinforcement of the importance of objective measures was also a significant contribution from this author, with the demonstration that observer consensus of aesthetic outcome is difficult to obtain. This line of thought was followed by other authors, who contributed with new measurements to value mainly asymmetry:Limbergen et al. [15] proposed two new asymmetry measurements, the Lower Breast Contour (LBC) and the Upward Nipple Retraction (UNR); Tsouskas & Fentiman [16] described the Breast Compliance Evaluation (BCE), which is the difference between the distance from the nipple to the Infra-Mammary Fold (IMF). Although all these works defined measures that could be “objectively” computed from the photograph of the patient, the measures were manually extracted (and therefore not completely independent from the user). Moreover, the measures were related to the asymmetry impact on the aesthetic result, leaving all other factors outside the analysis.

The last decade of the twentieth century also witnessed the introduction of attempts to enrich the collected data with multiple views of the patient [17], special cameras - telecameras [18], 3d scanners [19] and Moiré topographic [20] in order to facilitate the aesthetic evaluation. However, the benefits of using more information were offset by the cost and the complexity of the data acquisition process, leaving the single frontal image as the preferred acquisition protocol for the aesthetic assessment [21,22]. Nevertheless, 3D imaging can provide a stronger starting point than conventional photography for physics-based models that may be beneficial for surgical planning and supporting physician-patient communication [23].

The first decade of the twenty-first century witnessed the softwarization of the solutions [24,25]. In particular, the first two computer programs developed specifically for the LR aesthetic assessment were presented almost simultaneously, BCCT.core and BAT (Fig. 3).

Fig. 3.

Screenshots of the first two computer programs for the aesthetic assessment of LR.

The development of BCCT.core also brought with it the adoption of machine learning methodologies to integrate disparate measures into a global assessment of the aesthetic result [26]. In such (which are now thought of as shallow) machine learning approaches, engineers do not need to be concerned with constructing precise and exact rules to combine the multiple measures. Instead, they focus on statistical models or simple neural networks as an underlying engine and then automatically learn or ‘tune’ the parameters of the engine using the past data to make them handle uncertainty and generalize well for yet to be seen patients. These approaches require historical data, a rich set of photographs of patients, each individually evaluated by (a panel of) expert(s). These influential works adopted a Delphi Panel procedure to reach a consensus and guide the machine learning process [13].

Still, in the first decade of the 2000s, another line of efforts tackled the automation of the detection of fiducial points in the photograph to support the computation of the measures, alleviating the dependency on the user to achieve the overall assessment. The most relevant anatomical landmarks include the nipples, breast contours (with particular emphasis in the endpoints) and incisura jugularis [[27], [28], [29], [30]]. Although full automation was not achieved, the process was much less dependent on the user, often requiring only minor corrections. Once these marks were in place, the process flows transparently, with the computation of several measures and their combination in the overall assessment.

It is also worth to emphasize a change that simplified the acquisition process. Initial measures, like BRA and LBC, are quantities to which a physical dimension is assigned with a corresponding unit of measurement. To be properly recorded, they require a known scale to be present, enabling the conversion from pixels in the digital photograph (or units in the analogue photograph) to the true physical dimension. Thus, dimensionless quantities, which are based on ratios, were introduced to dismiss the need for the scale and therefore simplify the evaluation process [26].

An additional bonus related to these dimensionless quantities was that they were defined to dismiss the need to know which was the treated breast, further facilitating the full automation of the process, see Table 1.

Table 1.

Examples of the dimensionless asymmetry measures [26]. $\left(X_1,Y_1\right)$ and $\left(X_2,Y_2\right)$ are the coordinates of both nipples (using the sternal notch as the centre of coordinates); $N{I}_1$ and $N{I}_2$ are the nipple to infra-mammary fold distances.

Original measure	Dimensionless measure
BRA	$\text{pBRA}=\frac{BRA}{0.5\left(\sqrt{X_1^2+Y_1^2}+\sqrt{X_2^2+Y_2^2}\right)}$
LBC	$\text{pLBC}=\frac{LBC}{0.5\left(Y_1+N{I}_1+Y_2+N{I}_2\right)}$
BCE	$\text{pBCE}=\frac{BCE}{0.5\left(N{I}_1+N{I}_2\right)}$
UNR	$\text{pUNR}=\frac{UNR}{0.5\left(Y_1+Y_2\right)}$

3. The steps being taken: from handcrafted to deep-learning-based methodologies

The recent artificial intelligence (AI) breakthroughs achieved with deep learning have also reached this field. The machine learning traditional workflow is typically based on extracting predesigned features (also referred to as handcrafted or engineered features) from the patient photographs. For instance, BCCT.core algorithm relied on handcrafted measures like BRA to estimate the overall aesthetics. The algorithm did not learn that BRA was indeed useful for the aesthetics’ evaluation, it was predesigned by an expert. This feature engineering is a bottleneck requiring significant human expertise. By limiting the model to the use of handcraft features, one may be missing the integration of information not captured by the handcrafted features but still relevant for the aesthetic outcome.

Without any preconception about how to construct features relevant to the aesthetic evaluation, deep learning breaks away the aforementioned difficulties by the use of a deep, layered model structure, often in the form of neural networks, and the associated end-to-end learning algorithms. With deep learning, the feature extraction and analysis parts are fully coupled. The algorithm learns, directly from the image, to compute features and to use those features in the analysis of the aesthetic result. Feature construction and prediction are now unified in a single process.

Deep keypoint detection [31] relies on a cascaded refinement of the keypoint position to achieve high precision position estimation, as shown in Fig. 4. The architecture of the proposed model comprises two main modules: regression and refinement of heatmaps, and regression of keypoints. The goal of the first module is to generate an intermediate representation consisting on a fuzzy localization (yellow highlighted in Fig. 4) for the keypoints we want to detect. The second module receives and refines this fuzzy localization, outputting the x and y coordinates of the keypoints.

Fig. 4.

Deep keypoint detection. CNN stands for convolutional neural network. VGG is a specific CNN developed by the Visual Geometry Group of Oxford, which achieved very good performance on the ImageNet dataset. The authors use the VGG as a building block in the model.

The proposed method was designed and compared with baseline algorithms on more than 200 photographs, properly splitting the data in independent sets for training and testing. Significantly, the method with the best performance was a hybrid model consisting on the detection of the endpoints, nipples and supra-sternal notch using the deep model and finding the breast contour using a canonical approach based on the computation of the shortest path in a graph.

The next logical step is to improve the overall aesthetic evaluation also resorting to deep learning. A simple deep learning approach will try to learn automatically everything from the data/photographs, including the best set of attributes (distances, textural differences, etc.) that best capture the factors relevant to the aesthetic outcome. Since the datasets in the field are still strongly limited, one can facilitate the learning by using some prior knowledge about the problem. So, instead of having the model freely learning the attributes from the photographs, one can give preference for intermediate representations that approximate classical quantities like BRA, known to be strongly correlated with the aesthetic outcome (Fig. 5). The final stage of the model is trying to reliably evaluate the aesthetic outcome (and that is the main goal of the model), the intermediate layer is biased to approximate one/several of the classical attributes.

Fig. 5.

Potential deep architecture to predict the overall aesthetic result.

One of the most significant limitations of deep learning algorithms is the lack of transparency. This means that these models’ internal logic and inner workings are hidden to the user, which is a serious disadvantage, as it prevents a human, expert or non-expert, from being able to verify, interpret, and understand the reasoning of the system and how particular decisions are made. The first incursion in explaining the automatic aesthetic assessment has already been made [32,33], promising accountability for future certified Breast Units (and also with applications in teaching, etc.). Fig. 6 illustrates a case under study, the automatic evaluation by the AI algorithm, and some tentative, automatically generated, explanations of the assigned class.

Fig. 6.

Illustration of the explanations. The authors Silva et al. [32,33] considered only a binary decision and the corresponding explanations.

4. An outlook for upcoming technology

There is a definite conviction that objective methods will play a central role in assessing breast surgery procedures and in future Breast Units. The recent recommendations are pointing in that direction [34] and the traction that BCCT.core is gaining in the research community [35,36] support this view. Although several challenges impede bringing aesthetic evaluation into daily clinical practice now, it is expected to be a critical component in future BC management workflows.

Some needs are likely to be satisfied in the near future as they are the corollary of current efforts. The full automation of the process with high accuracy will bring efficiency and user independence to the process. The current deep based approaches, supported in large sets of past data, are likely to attain this goal. Although the full integration in current Hospital Information Systems is not foreseen in the near future, the deployment of the tools of aesthetic assessment as web applications or cloud services may facilitate the widespread adoption. The softwarization in the 2000s was an important landmark, with software developed specifically for the task, but it provided only desktop applications. While a desktop application must be installed on the computer before it can run, web applications allow us to access it on demand by using a web browser, not requiring installed software. Furthermore, they can be accessed on any device with internet connection, providing maximum accessibility with minimal system requirements. Web development tools are nowadays much more mature than 10 years ago, making the engineering task a lesser effort. Although concerns with security and privacy in e-health cloud-based systems are justified, solutions exist to properly secure health data in the cloud [37,38].

Despite all the progress in the field, there is still a lack of big data and extensive international studies. For example, Silva et al. [39] have developed a deep neural network for skin lesions classification using a dataset of 129450 clinical images. Efforts of similar dimension can be found in other medical domains. It is fundamental to set up and populate a sizeable interoperable repository of photographs of breast cancer LR treated patients, enabling the development, testing, and validation of AI-based solutions to improve aesthetic evaluation and overall quality of life follow-up. Following current trends, the platform should promote access to anonymised image data sets to be made more openly reusable across the globe for developing AI solutions. The Dialogue on Reverse Engineering Assessment and Methods (DREAM) initiative recently hosted an open crowd-sourced Digital Mammography (DM) DREAM challenge1 to foster the development of algorithms for the detection of cancer in screening mammography, and to objectively determine by blind evaluation whether machine learning methods applied to data from mammography exams can improve screening accuracy. Similar endeavours in the aesthetic evaluation could have a massive impact in the field.

While the focus has been in the aesthetic outcome evaluation for breast conserving surgery, similar concerns about aesthetic assessment exist for related populations, such as the minority of women who require total mastectomy and may desire reconstruction. Additionally, new surgical techniques, as well as new forms of delivering loco-regional radiation therapy, remain unexplored. Due to the introduction of plastic surgery techniques into the BC surgery arena, a large number of new surgeries have been generalized without a proper evaluation tool. There is an almost absolute lack of knowledge of the aesthetic outcome of these surgeries as well as its impact on patients’ QoL. At almost the same time, new radiation therapy techniques were also introduced, and again, there is a heterogeneous evaluation of their impact, either per se or associated with the different surgical operations previously referred. The correlation of this aesthetic outcome with patients QoL is also not standardized and very difficult to evaluate due to the vast diversity of available tools. Although the different surgical procedures and radiation therapy techniques require specific models to evaluate aesthetics, they all share properties that can be explored to improve the model design. In the machine learning community, transfer and multitask learning focuses on building better predictive models by exploiting knowledge gained in previous/related tasks, which allows for the softening of the traditional supervised learning assumption of having identical train-test distributions [40]. These methodologies can help us to build on top of the present efforts for LR surgeries, adapting the models for the aesthetic evaluation to the vast offer of BC treatments.

Finally, this journey towards more objective methods excluded patient self-assessment from the evaluation process, leading to a division in the community. It is still not totally understood why patients’ evaluation is different from expert evaluation and objective evaluation of results. A first reason is related to the fact that aesthetic evaluation is dependent on many individual (psychological, physical, social, and cultural) factors that can impact on how the patients see the results. The other reason, less explored, has to do with the patients’ usual absence of knowledge on the resulting outcome. Confronted with the fear of cancer and the fear of losing the breast, they usually tend to be more benevolent with worse aesthetic results and evaluate themselves better. It seems fundamental to unite these two perspectives of QoL, researching methods for the evaluation of the aesthetic outcome of BC treatments integrating objective methods and significant factors derived from patient input.

5. Conclusion

While the EU is fighting the battle of Breast Units certification, creating quality indicators mainly related to overall survival (OS) and breast cancer specific survival (BCSS), it seems incongruent not to have a proper evaluation of these important parameters of outcome that will allow comparison of results between Breast Units and correction of factors responsible for worst results and will also allow patients to fight for better care by choosing units with more consistent and favourable results. In the European Society of Mastology (EUSOMA) quality indicators review of 2017, there is still no reference to any form of aesthetic evaluation of results. This is possibly the consequence of none of the discussed methods being recognised as a gold standard, and the European Organisation for Research and Treatment of Cancer (EORTC) still advises a combination of methods for this evaluation (self-evaluation + subjective panel assessment + objective measurements). Unfortunately, with the current numbers of incident BC cases, this is neither practical nor feasible. The current rise of AI, propelled by the new paradigm of deep-structured machine learning or deep learning, seems to offer the tools that were missing to achieve the accuracy and automation to convince the community. The striking successes in speech recognition, computer vision, and machine translation – completely taken over by the deep-learning paradigm – give us a set of reasons to approach the task with confidence.

Declaration of competing interest

There are no identified conflicts of interest.