Diagnosis of Melanoma by Imaging Mass Spectrometry: Development and Validation of a Melanoma Prediction Model

Abstract

Background

The definitive diagnosis of melanocytic neoplasia using solely histopathologic evaluation can be challenging. Novel techniques that objectively confirm diagnoses are needed. This study details the development and validation of a melanoma prediction model from spatially resolved multivariate protein expression profiles generated by Imaging Mass Spectrometry (IMS).

Methods

Three board-certified dermatopathologists blindly evaluated 333 samples. Samples with triply concordant diagnoses were included in this study, divided into a training set (n=241) and a test set (n=92). Both the training and test sets included various representative subclasses of unambiguous nevi and melanomas. A prediction model was developed from the training set using a linear support vector machine (SVM) classification model.

Results

We validated the prediction model on the independent test set of 92 specimens (75 classified correctly, two misclassified, and 15 indeterminate). IMS detects melanoma with a sensitivity of 97.6% and a specificity of 96.4% when evaluating each unique spot. IMS predicts melanoma at the sample level with a sensitivity of 97.3% and a specificity of 97.5%. Indeterminate results were excluded from sensitivity and specificity calculations.

Conclusion

This study provides evidence that IMS-based proteomics results are highly concordant to diagnostic results obtained by careful histopathologic evaluation from a panel of expert dermatopathologists.

Introduction

The paradigm for understanding and analyzing melanocytic neoplasia has shifted in the past two decades. However, despite advancements, most dermatopathologists rely solely on histopathologic evaluation to assess melanocytic lesions, dictating clinical treatment and outcome.¹ During this process, the classification of a particular tumor (for example, a lesion with features in between a severely dysplastic nevus or an early superficial spreading melanoma) is dependent on the relative weight subjectively assigned to each diagnostic criterion, leading to discordant diagnoses.^{2, 3} Furthermore, discordance rates increase for all melanoma types when evaluating in situ and thin invasive melanoma.^4–8 In one study, diagnostic discrepancies ranged from 15% to 25% of cases.⁹ In another, an expert panel review of 1069 melanocytic lesions considered 14% (22/158) of cases initially classified as invasive melanoma to be benign, and 16.6% (85/513) of cases initially classified as benign to be malignant.¹⁰

An essential focus in pathology practice is developing strategies and tools that provide diagnostic clarity to such lesions. One strategy uses a second opinion, but they are still subjective and potentially prone to the same biases as the initial interpretation.¹¹ Clinicians widely use immunohistochemistry to measure molecular information for specific epitopes.¹² The revolutionary use of molecular information to characterize the mutational and transcriptional landscape of melanocytic neoplasms to aid classification has emerged and continues to evolve. Assays based on comparative genomic hybridization (CGH), gene expression profiles, and fluorescence in situ hybridization (FISH) are among the recently developed laboratory tests for this specific purpose.^13–17

Matrix-assisted laser desorption ionization imaging mass spectrometry (MALDI IMS) is a technology ideal for providing proteomics information supporting the pathologists’ diagnosis. In a MALDI IMS experiment, thin tissue sections are prepared similarly to traditional histological experiments and analyzed using a MALDI mass spectrometer. A laser ionizes biomolecules (protein-derived peptides in this study) from discrete (in this work, 50 μm) spatial locations annotated by the dermatopathologist. A profile of detected ions, indicative of the healthy or diseased state of the cells in that region, is recorded by the mass spectrometer.^{18, 19} Mass spectra of many tissues with known pathology are the basis to train a machine learning classification model used to classify spectra taken from tissue with unknown pathology. Previous work demonstrates that this approach can accurately classify various cancers, including head and neck cancers,²⁰ ovarian cancer,²¹ lung cancer, pancreatic cancer, and skin cancer.^22–25 Previous studies on the skin have shown proof-of-principle to distinguish spitzoid melanoma,²⁶ malignant melanoma,^27–29 and melanoma metastasis.^{27, 30–32} Original studies used a reagent spotter to perform digestions. These experiments used larger tissue regions (300μm) to obtain each mass spectrum and had a low throughput.^{23, 24, 27} Current technology uses homogeneous reagent coatings, helping collect data faster and from smaller regions of interest.²⁸ Despite these studies, there remains a need for more well-controlled studies and advanced bioinformatics approaches to translate the technology into clinical use.

This report establishes MALDI IMS as a technology that accurately confirms the diagnosis of melanocytic neoplasms compared to triconcordant histopathological evaluation. We developed a melanoma prediction model from spatially resolved multivariate protein expression profiles using support vector machine models with strict per spot thresholding, including only confident spectra for patient classification. Support vector machines (SVMs) are a robust machine learning algorithm that is used to classify mass spectra as benign or malignant. SVM maps labeled mass spectra as points in mathematical space and does so in a way that maximizes the width of the gap between the labeled classes, i.e., maps them to maximally separate benign and malignant spectra. Margin-maximization is a good strategy for classification tasks, as a larger margin intuitively implies a more robust classification model. SVMs owe their popularity to the aforementioned statistical robustness combined with several advantageous computational properties.³³

The patient cohort included various subclasses of unambiguous nevi and melanomas with triply concordant diagnoses by three dermatopathologists. We validated the melanoma prediction model on an independent test set of 92 specimens. This study provides clear evidence that the results obtained from MALDI IMS are highly concordant to those obtained by careful histopathologic evaluation from a panel of expert dermatopathologists. This testing approach will be useful for objectively validating the diagnosis of melanocytic neoplasia.

Materials and Methods:

Institutional Review Board

Sterling Institutional Review Board (Atlanta, GA) and the institutional review boards (IRB) of the participating institutions (Indiana University IRB number: 1910325060, Vanderbilt IRB 030220, Duke University IRB Pro00102363) determined this study exempt from IRB review under the terms of the US Department of Health and Human Services Policy for Protection of Human Research Subjects at 45 CFR §46.104(d).

Selection of cases

Cooperating academic institutions and private practices provided deidentified formalin-fixed, paraffin-embedded skin biopsies. Patients diagnosed with nevi and invasive melanoma were identified from the electronic medical record and enrolled with best efforts to match benign nevi to melanoma based on age and gender and represent less common subtypes of melanoma. Enrolled samples were excised between 2010–2016. Archival slides were screened to ensure adequate diagnostic material.

Pathology Analysis

Three board-certified dermatopathologists independently evaluated 375 deidentified H&E slides in a blinded fashion. The study director provided patient age, sex, and lesion location for each patient along with the H&E slides, and the dermatopathologists classified the sample as melanoma or benign nevus. We analyzed the result for concordance and included only triconcordant samples in this study (n=333). The final number of samples enrolled in this study is consistent with similar studies using this technology to classify melanoma.^{23, 24, 28} The panel of dermatopathologists reviewed sixteen biconcordant specimens and determined that two samples were unambiguous (included in the 333 triconcordant samples) and 14 were diagnostically challenging and excluded due to lack of consensus. We excluded 22 samples due to assay errors, and six samples did not meet study criteria upon pathological analysis. Figure 1 details the study design and samples in each category.

Figure 1: Study Design.

Three dermatopathologists independently provided a sample diagnosis. The study included only triconcordant samples in the training and test set.

H&E slides were scanned at 20× magnification using an SCN-400 digital slide scanner (Leica) or a Huron TissueScope LE120 (Huron Digital Pathologies.) Dermatopathologists annotated the melanocytic cell populations from whole slide images to guide mass spectrometry data acquisition using an online portal.³⁴ Dermatopathologists annotated images using a spot tool with a 50 μm diameter and placed annotations in the following categories: melanoma in situ, invasive melanoma, junctional nevus, intradermal nevus, uninvolved epidermis, and dermal stroma. Table 1 details specific patient demographics included in the training and test sets.

Table 1:

Patient Demographics for Training and Text Sets

Mass Spectrometry Analysis & Tissue Classification

This study used serial sections of FFPE tissue blocks. The IMS process used annotated stained sections to guide the mass spectrometer to the same region on an unstained 6 μm serial section. The unstained section was subjected to antigen retrieval, followed by in situ tryptic digestion to liberate peptides from the tissue. Next, a MALDI matrix was applied to the tissue surface to aid in the desorption and ionization of molecules during the analysis. Each 50 μm annotation generated a unique mass spectrum. Spectral data were removed from the analysis if they were obtained from damaged tissue areas or did not meet minimum mass spectral quality standards. An optimized melanoma prediction model was constructed and optimized from the training set data (n=241 patients) using a linear support vector machine classifier. The classifier’s performance was validated using data obtained from a separate set of samples (n=92). A complete description of the analytical and computational methodologies is available as supplementary materials.

Results

This study includes a total of 333 unambiguous melanocytic neoplasms divided into a training (n=241) and test set (n=92) (Table 1). Each sample had an average of 21 annotation spots placed on each sample, and each annotation resulted in a mass spectrum. Table 2 details the assay performance. We constructed and applied quality metrics to ensure the inclusion of only high-confidence spectra in the classification. This analysis required a prediction probability greater than or equal to 95% (see supplemental methods for details). We exclude spectra with a prediction probability below this threshold from the sample diagnosis. The algorithm correctly classified 507 of 526 benign nevi spectra in the test set for a specificity of 96.4% (95% CI 94.4% to 97.8%) and 489 of 501 melanoma spectra for a sensitivity of 97.6% (95% CI 95.9% to 98.8%). Indeterminate results were excluded from sensitivity and specificity calculations.

Table 2:

MALDI IMS assay prediction accuracy of the test set relative to clinical diagnosis

Note: A total of 1958 spots were evaluated in the test set (melanoma, n=1023; benign, n=935). A total of 92 sample diagnoses confirmed in the test set (melanoma, n=43; benign, n=49).

The MALDI IMS process independently evaluates a mass spectrum from each annotation for each sample. We constructed a sample-level scoring algorithm to provide an objective metric (scale from 0 to 1) that scores the likelihood that a sample is a melanoma. This scoring algorithm considers the spots for each sample with a prediction probability greater than or equal to 95%, then computes the simple ratio of spots that predict melanoma relative to that sample’s total number of spots. The scoring algorithm assigns samples with a score ≥0.85 as “melanoma,” samples with a score ≤0.15 as “benign,” and samples with scores >0.15 and <0.85 as “indeterminate.” In a final step, additional samples are classified as “indeterminate” if they have fewer than two spots with a prediction probability greater than or equal to 95%. The sample-level scoring algorithm of the test set classified 39 of 40 benign nevi samples for a specificity of 97.5% (95% CI 86.8% to 99.9%), 36 of 37 melanoma samples for a sensitivity of 97.3% (95% CI 85.8% to 99.9%), and 15 samples as indeterminate (n=9 benign nevi and n=6 melanoma). Indeterminate results were excluded from sensitivity and specificity calculations. Overall, the classification performance of the melanoma prediction model in the test set and self-recognition of the training set are not statistically different (eTable 1 in Supplementary Material).

Representative Case Studies of Samples

Figure 2 shows an example of the analysis results. The top panel shows a low magnification view of a melanoma (A) and a nevus (C) enrolled in the study (test set). The different colored annotations indicate different pathologic components, including invasive melanoma (red), melanoma in situ (orange), dermis stroma (black), and intradermal nevus (teal). Panels B and D show a higher magnification micrography of select areas indicated in panels A and C. Panels E and F show averaged mass spectra from these biopsies. The asterisks indicate several ion abundance differences between the melanoma and the nevus sample (F).

Figure 2: Example analysis of melanoma and benign nevus analyzed by IMS.

The samples were H&E stained and scanned using a Huron Tissuescope digital slide scanner with a 20× objective and a 1.5× Barlow lens. Panel A and C show a low magnification image of the entire lesion and a higher magnification of annotated areas of interest for melanoma (B) and nevus (D). Panels E and F show average mass spectra obtained from each sample. The red trace indicates the averaged mass spectrum from all melanoma spots in (A), while the blue trace is the averaged mass spectrum from all benign spots in (C). Panel (F) shows the averaged mass spectra of these regions from m/z 1050–1250. The asterisks denote peaks with relative intensities that differ between nevus and melanoma.

Discussion

Melanoma Diagnostic Testing Technologies

Melanoma is the third most common form of skin cancer. Although melanoma accounts for only 1% of skin cancers, it causes most skin cancer deaths.³⁵ There is a great need for objective novel analytical technologies to assist with melanoma diagnoses. Immunohistochemical markers are commonly used alongside histopathologic examination but remain subjective, especially for proliferation markers.^{36–39,12, 39, 40} CGH leverages the genome instability typical of melanomas and surveys the entire genome for copy number abnormalities that correlate with clinical behavior.^41,42 Extraction of adequate DNA from small biopsies or shave biopsies can be challenging. However, microarray technologies have significantly decreased specimen requirements, needing only 37 ng DNA.^43–46 Researchers developed fluorescence in situ hybridization (FISH) as an alternative way to monitor genomic instability using fluorescently-labeled probes to tag DNA. Although FISH is performed directly on tissue sections, preserving spatial localization and allowing for detection of small subpopulations, this approach provides copy number data on a relatively small number of chromosomal loci frequently altered in melanoma.^{47, 48} Gene expression profiling is another approach; for example, MyPath Melanoma (Myriad Genetics) offers a 23-gene expression signature to differentiate melanoma from nevi.^{49, 50} Macro-dissection of samples containing adequate diagnostic tissue, where the pathologist circles region of interest on an H&E slide guiding RNA extraction from the corresponding region of unstained tissue, precedes data collection.⁵¹

MALDI IMS is a technology for spatial proteomics

MALDI IMS provides an alternative technology with multiple benefits for the diagnosis of skin cancers. First, mass spectrometry requires minimal diagnostic material (a single 6 μm section) for in situ analysis. This study utilized only two sections for most samples: one for histopathological annotation and a serial section for mass spectrometry data acquisition. This approach eliminates dilution of target cell populations by sample homogenization. Mass spectrometry analysis of each annotated spot results in individual measurements and classification scores mapped to the pathological origin, making it possible to visualize spots confidently predicted to be melanoma in perfect registration with the reference H&E for added diagnostic confidence (Examples, Figures 1 & 2).

Alongside the benefits of minimal sample requirements and spatial information, there are caveats as well. As MALDI IMS ionizes molecules directly from tissue, there can be a decrease in assay performance for cases in which the histology is either highly heterogeneous or complex.⁵² Tumors that grow as small clusters (smaller than annotations spots) are subject to spectral contamination by non-melanocytic cells. We perform IMS analysis on an adjacent section, serial to the stained and annotated section, in the process described. Therefore, indeterminate or inaccurate readings can result from samples with minimal melanocytes or small regions of interest.

MALDI Mass Spectrometry Successfully Differentiates Melanoma from Nevi Samples on a Per Sample and Per Spot Level

As shown in Table 2, machine learning classification is robust for the classification of melanoma. Despite the high level of accuracy achieved in this study, there are opportunities for further improvement. Spectral data from each spot do not always meet data-driven, empirical performance metrics for quality predictions; the targeted cells might be contaminated with adjacent cells or out of the plane of analysis. As IMS technologies advance for histology-directed analysis, future studies may include smaller annotated regions or performing annotations on the same section analyzed by IMS as a strategy to decrease targeting errors.^{53, 54}

We developed a sample-level scoring algorithm that classifies lesions as benign nevus or malignant melanoma. The algorithm only utilizes spots that passed quality control to score lesions. The algorithm only assigns sample-level diagnostic predictions (melanoma or nevus) when 85% or more confident spot scores indicate the same diagnosis. In this analysis, the minimum number of spots required to meet the quality threshold was two. This approach allowed for the inclusion of smaller biopsies with smaller melanocytic foci. Of the 15 samples that were ultimately indeterminate, six samples had no spectra that met the inclusion criteria. In future studies, we hypothesize that annotating a higher number of melanocytic regions for each specimen will significantly decrease the rate of indeterminacy.

Despite these restrictions, the melanoma prediction algorithm returned definitive results for 83.6% (77 of 92 samples) in the test set with 97.8% overall accuracy (90 of 92 samples). The indeterminate rate was 16%, similar to other ancillary tests for melanoma.^{13, 16, 43, 55} Although this study utilized unambiguous nevi and melanomas, future studies will examine histologically ambiguous lesions.

Conclusion

Clinical analysis of patient biopsies using mass spectrometry represents a promising ancillary tool to diagnose melanocytic neoplasms. This study utilized spatially targeted IMS of unambiguous nevi and melanoma, determined using triple concordance by board-certified dermatopathologists, to build a melanoma prediction model. Dermatopathologists annotate a tissue section with regions of interest, guiding the collection of MALDI IMS data from serial sections. The workflow is analogous to traditional pathology workflows, and in optimum cases, only requires a single serial section for IMS. This resulting classifier detects melanoma with a sensitivity of 97.6% and a specificity of 96.4% on a per spot level. The melanoma prediction model’s performance on the sample level is high, with a sensitivity of 97.3% and a specificity of 97.5%. These results confirm that IMS results are highly concordant to results obtained by expert dermatopathologists. Furthermore, these data demonstrate that IMS is an effective tool for objectively confirming melanocytic neoplasia diagnoses.