Pigmented Lesion Assay for Suspected Melanoma Lesions: A Health Technology Assessment

Abstract

Background

Early detection of melanoma is key, as survival rates are substantially better when the cancer is detected in its early stages. Currently, the standard of care is to biopsy any lesion suspected of melanoma for diagnostic confirmation by histopathology. As a result, most people who undergo biopsy receive negative melanoma results. If effective, a non-invasive alternative, such as pigmented lesion assay, could minimize the number of unnecessary biopsies performed. We conducted a health technology assessment of pigmented lesion assay for people with suspected melanoma lesions, which included an evaluation of diagnostic accuracy, clinical utility, the budget impact of publicly funding pigmented lesion assay, and the preferences and values of people who have undergone biopsy for suspected melanoma.

Methods

We performed a systematic literature search of the clinical evidence. We assessed the risk of bias of each included study using the Quality Assessment of Diagnostic Accuracy Studies–2 (QUADAS-2) and the Risk of Bias Assessment Tool for Non-randomized Studies (RoBANS). We assessed the quality of the body of evidence according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group criteria. We performed a systematic literature search of the economic evidence. We also analyzed the budget impact of publicly funding pigmented lesion assay in adults with suspected melanoma in Ontario. To contextualize the potential value of pigmented lesion assay, we spoke with people who had undergone skin biopsy for melanoma. We also used the qualitative research synthesis from a report by the Canadian Agency for Drugs and Technologies in Health to provide context for the preferences and values of those with suspected melanoma.

Results

We included seven studies in the clinical evidence review. Pigmented lesion assay has a sensitivity of 79% (95% confidence interval [CI] 58%–93%) and a specificity of 80% (95% CI 73%–85%; GRADE: Low). We found one published cost-effectiveness study with potentially serious limitations. Therefore, the cost-effectiveness of pigmented lesion assay compared with the standard care pathway is currently uncertain. Assuming a very low uptake, we estimated that the budget impact of publicly funding pigmented lesion assay in Ontario over the next 5 years is about $3.44 million if the test is used exclusively by primary care providers, or about $2.56 million if it is used exclusively by specialists. The people with whom we spoke who had experienced biopsy for suspected melanoma responded positively to the potential benefits of pigmented lesion assay, emphasizing its ease-of-use, potential increase in early detection of melanoma, and reduction in physical and emotional burden of unnecessary biopsies. Participants also felt that the accuracy of this tool was essential to ensure minimal false negatives.

Conclusions

There is uncertainty because of the low-quality evidence for the diagnostic accuracy of pigmented lesion assay. The cost-effectiveness of pigmented lesion assay compared with standard care is also uncertain. We estimated that publicly funding pigmented lesion assay in Ontario over the next 5 years would result in additional costs of $3.44 million (if used exclusively by primary care providers) or $2.56 million (if used exclusively by specialists). For people who had experienced biopsy for suspected melanoma, it was felt that pigmented lesion assay could represent an effective tool to increase early detection and avoid unnecessary biopsies, if the tool was accurate.

Objective

This health technology assessment evaluates the diagnostic accuracy, clinical utility, and budget impact of publicly funding pigmented lesion assay for people with suspected melanoma skin lesions. It also evaluates the experiences, preferences, and values of people who have experience with skin biopsy for suspected melanoma.

Background

Health Condition

Melanoma is a type of skin cancer that starts in the melanocyte cells of the skin. Sun exposure can increase one's risk for skin cancer, in general; however, melanoma can occur on any part of a person's body and not necessarily on skin that has been exposed to the sun.¹

Clinical Need and Target Population

Melanoma accounts for approximately 3% of all new cancer cases in Canada and is the cause of about 1.4% of all cancer deaths.² Furthermore, the number of melanoma cases is expected to increase over time as the population ages and grows.³ In 2017, approximately 7,250 Canadians were diagnosed with melanoma, of whom 1,250 died.² Early detection is key for survival, with a 5-year survival rate of 97% for melanomas diagnosed as stage 1A (the earliest detection stage of invasive melanoma). However, there is only a 15 to 20% survival rate when diagnosed at stage 4 (the most advanced stage of cancer).² Melanoma is more prevalent among Caucasian populations, and is correlated with fairness of skin and the intensity of sun exposure in the geographic region.⁴

There are several different types of melanoma. Superficial spreading melanomas initially spreads across the surface layers of the skin, whereas nodular melanomas initially grows as a vertical bump.^5,6 Lentigo maligna melanoma is a type of in situ melanoma (melanoma that is contained and has not spread to other tissues of the body) and forms on areas of the skin that get a lot of sun exposure, such as the face.^5,6 A rarer form, called acral lentiginous melanoma, is found on the palms of the hands and soles of feet, and make up less than 5% of melanomas. Acral lentiginous melanoma is more common among darker skinned individuals.⁶

Once diagnosed, melanoma is staged based on depth of skin invasion, spread to lymph nodes, and metastatic spread to distant skin sites or organs (e.g., liver or lungs). This staging involves excision of the melanoma lesion and in some cases, a blood test, imaging tests, and a biopsy of the sentinel lymph node (the first lymph node in which the cancer is likely to spread.

Current Diagnostic Approach

The majority (70%) of melanomas are first discovered by the patient or their family members as a mole with an unusual or changed appearance.⁷ The physician assessment begins with a medical history and visual assessment of the lesion and surrounding skin. The Canadian Dermatology Association supports the use of the common “ABCDE criteria” for identifying potential melanomas for biopsy through visual inspection, which is an acronym for: Asymmetry; Borders that are irregular or ragged; Colour variation within the lesion, including possible shades of brown, black, red, grey, or white; Diameter growth that can be larger than 6 mm; and Evolution of the lesion, including possible changes in shape and/or colour, or new symptoms, such as itchiness or tenderness.⁷ However, strict adherence to these criteria may result in missed cases. For example, some experienced clinicians may be able to correctly identify suspicious lesions when they are only 2 mm to 4 mm wide.⁵ In addition, dermoscopy (an examination of the skin using a specialized magnifying tool) may be used to support physician visual assessment and improve the diagnostic accuracy. Lesions that remain suspicious for melanoma are biopsied and samples are then sent to a histopathologist for further investigation. Histopathology is the study of microscopic changes in tissues caused by disease, and is used to diagnose melanoma.

Biopsy and histopathology are considered the gold-standard for diagnosing melanoma; however, it is an imperfect reference standard. Diagnosis is reliant on the histopathologist's interpretation of the cells, and while there are guiding principles, there is not always agreement among pathologists.⁸ In addition, the size of the lesion may affect the diagnosis, as histopathology tends to demonstrate more reliable accuracy in lesions larger than 6 mm in diameter.⁹ One reason for this inconsistency may be that when a suspected melanocytic lesion is small (< 4 mm), it can be difficult to distinguish melanoma from nevi (benign, non-cancerous lesions), as there are not enough definitive clinical features present. ⁹ One study found that only 4% of biopsied lesions smaller than 6 mm in diameter were diagnosed as melanoma, compared with 13% of biopsied lesions larger than 6 mm.¹⁰

In Ontario, patients typically present to their primary care provider with a suspicious lesion. It is then the responsibility of the primary care provider (e.g., family physician or nurse practitioner) to determine the need for a biopsy or a referral to a dermatologist to confirm a melanoma diagnosis (Figure 1). Biopsies can be conducted by family physicians; however, not all family physicians perform them. Depending on the size, location, and nature of the suspected lesion, biopsies can be conducted in different ways, including shave biopsy, punch biopsy, or complete excision.⁷ One survey of Canadian primary care physicians found that only 19% (18 of 93) would always do excisional biopsy, while 38% (35 of 93) would consider incisional biopsy (including punch biopsy).¹¹ The survey found that when primary care providers were faced with a suspicious lesion and did not want to biopsy themselves, they would refer the patient to a specialist—typically a dermatologist or surgeon.¹¹ Reasons provided for not performing a biopsy themselves included the location of the suspected lesion on a sensitive area (such as the face), or the lack of experience, skill, time, or equipment in their office.¹¹

Figure 1: Clinical Management From Initial Presentation of Suspicious Lesion to Histopathology.

It is not feasible or reasonable to biopsy every lesion. However, there is an overall concern that more biopsies are being performed than necessary. In 2016, the United States (US) Preventative Services Task Force recommended against routine screening for skin cancer partly due to the high rate of unnecessary biopsies.¹² Therefore, one of the challenges of diagnosing melanoma is selecting which lesions to biopsy. A Cochrane special collection of reviews compiled evidence on four interventions used to assist with selecting lesions to biopsy and found that the use of visual support tools, such as dermoscopy and reflectance confocal microscopy, improved the diagnostic accuracy compared with visual inspection alone, and that in-person inspections were more effective than the inspection of images (results summarized in Table 1).¹³ In Ontario, the use of dermoscopy is not wide spread among primary care providers, but is used more frequently among dermatologists (Wade Mitchell, MD, phone communication, November 2019; Sakina Walji, MD, phone communication, November 2019; Frances Wright, MD, phone communication, November 2019).

Table 1:

Summary of Results from Cochrane Reviews on Tests for Identifying Lesions Suspicious for Melanoma

Test	Description of Subgroup	Results Summary
Visual inspection16	Limited prior testing (e.g., primary care)	In-person inspections are more accurate than image inspections. Relative DOR 8.54 (95% CI 2.89–25.3)
	Referred to a specialist (e.g., dermatologist)	Specialists have higher specificity in selecting lesions for excision than primary care practitioners. Relative DOR 1.51 (95% CI 0.32–7.09)
Dermoscopy17	In-person visual inspection and dermoscopya	Dermoscopy is more effective than visual inspection alone. Relative DOR 4.7 (95% CI 3.0–7.5)
	Image-based dermoscopya	In-person inspections are more accurate than image inspections. Relative DOR: 4.6 (95% CI 2.4–9.0)
Reflectance confocal microscopy (RCM) 18	In-person RCM ≥ 3	Reflectance confocal microscopy (Vivascope) is more accurate than dermoscopy. Relative DOR 4.82 (95% CI 2.16–10.8)
Smartphone applications for triaging suspicious skin lesions19	High risk for melanoma vs. not melanoma	Smartphone applications have not yet demonstrated sufficient accuracy.

Abbreviations: CI, confidence interval; DOR, diagnostic odds ratio; RCM, reflectance confocal microscopy.

^aIn a subgroup analysis, the review did not find any difference in accuracy by experience or training of the assessor; however, study authors acknowledge that the majority (95%) of their included studies were in specialist assessors, and not first-contact primary care providers.

When a biopsy is found positive for melanoma, the lesion should be fully excised with a safety margin of 5 mm to 2 cm to ensure all potential cancerous cells have been removed.¹⁴ It may also be warranted to conduct a sentinel lymph node biopsy at this time to determine the potential spread of melanoma.¹⁴ Patients diagnosed with earlier stages of melanoma (in situ and stages 1A to 2A) do not require an oncologist visit, and it is recommended they see a dermatologist every 6 to 12 months for follow-up.¹⁵ Patients with later stages of melanoma should be treated and monitored by an oncologist for 5 years.¹⁵

Health Technology Under Review

Pigmented lesion assay is a non-invasive, adhesive patch gene expression test intended to guide biopsy decisions and rule out melanoma. This adhesive patch collects a tissue sample that is then sent to the DermTech laboratory in California, USA.²⁰ With proper storage, samples can be analyzed up to 11 days after collection (DermTech Canada Inc., email communication, January 2020). The patch can be applied to nearly the entire body, with the exceptions of the palms of the hands, the soles of the feet, and areas with hair (e.g., scalp). Results of the samples are analyzed for the presence of two genes: long intergenic non-coding ribonucleic acid (RNA) 518 (LINC00518) and preferentially expressed antigen in melanoma (PRAME). It is recommended by the manufacturer that samples that test positive for at least one of the two genes be confirmed with biopsy.²¹ Therefore, in practice, pigmented lesion assay is considered a test to rule out the potential need for biopsy. There are publications describing the gene discovery and analytical validity of the adhesive patch, supporting the genes selected, and method for RNA extraction.^22–24

There are other gene expression profiling tests to support the diagnosis of melanoma, including myPath and DecisionDx. However, unlike the DermTech pigmented lesion assay, both of these tests are conducted on tissue samples obtained by skin biopsies. The myPath²⁵ test can support confirmatory diagnosis of melanoma, and DecisionDx²⁶ is intended to support the prognostication of melanoma and treatment path decisions.

The 2019 American Academy of Dermatology melanoma guideline stated that there is insufficient evidence to support the widespread adoption of gene expression profiling for diagnosis or prognosis, and reported that technologies such as the pigmented lesion assay adhesive patch (referred to as “bladeless biopsy”) were “emerging” and required more evidence.²⁷

Regulatory Information

DermTech Canada Inc. holds an active medical device establishment license to import a Class I medical device (licence number 7680; https://health-products.canada.ca/mdel-leim/index-eng.jsp, accessed August 14, 2019).

Ontario, Canadian, and International Context

The DermTech pigmented lesion assay is licensed to sell in Ontario; however, it is not currently publicly funded by the Ontario Health Insurance Plan or any other insurance plan in Ontario at this time (DermTech Canada Inc., in-person and phone communications, October 2019). The technology is being sold and used in the United States, where the device was developed and where the DermTech laboratory is based. The technology has Medicare coverage in the United States for use as a decision tool for atypical melanocytic lesion prior to the decision to biopsy, but not as an adjunctive test for lesions already warranting biopsy.²⁸ As of October 2020, it is also covered by Geisinger Health System.²⁹ We have identified reviews by select Blue Cross/Blue Shield agencies, which concluded that there is a lack of robust clinical validity evidence for pigmented lesion assay³⁰ and that the technology is considered investigational.³¹ However, the California BlueCross/BlueShield policy was archived in July 2020.³²

Expert Consultation

We engaged with experts in the specialty areas of skin cancer, dermatology, pathology, and primary care to help inform our understanding of aspects of the health technology and our methodologies and to contextualize the evidence.

PROSPERO Registration

This health technology assessment has been registered in PROSPERO, the international prospective register of systematic reviews (CRD42020153122), available at https://www.crd.york.ac.uk/PROSPERO.

Clinical Evidence

Research Question

What are the diagnostic accuracy and clinical utility of pigmented lesion assay for people with suspected melanoma skin lesions?

Methods

Clinical Literature Search

We performed a clinical literature search on September 10, 2019, to retrieve studies published from database inception until the search date. We used the Ovid interface in the following databases: MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, the Health Technology Assessment database, and the National Health Service Economic Evaluation Database (NHS EED).

A medical librarian developed the search strategies using controlled vocabulary (e.g., Medical Subject Headings) and relevant keywords. The final search strategy was peer-reviewed using the PRESS Checklist.³³

We created database auto-alerts in MEDLINE, Embase, and monitored them for the duration of the assessment period. We also performed a targeted grey literature search of health technology assessment agency websites as well as clinical trial and systematic review registries. The grey literature search was updated on December 18, 2019. See Appendix 1 for our literature search strategies, including all search terms.

Eligibility Criteria

STUDIES

Inclusion Criteria

• English-language full-text publications

• Studies published from database inception until September 10, 2019

• Randomized controlled trials, diagnostic case-controlled studies, observational studies, provider surveys

Exclusion Criteria

• Animal and in vitro studies

• Editorials, letters, commentaries, conference abstracts, conference posters

PARTICIPANTS

• Adults with suspected melanoma lesions

INTERVENTIONS

Inclusion Criteria

• Non-invasive, adhesive patch genetic-based test to support the identification of suspected lesions to biopsy for melanoma diagnosis (i.e., pigmented lesion assay)

Exclusion Criteria

• Genetic tests for prognosis (e.g., BRAF) or risk of developing melanoma (e.g., CDKN2A)

• Non-genetic based interventions to support the identification of suspected lesions to biopsy for melanoma diagnosis (e.g., dermoscopy)

REFERENCE STANDARD

• Skin biopsy and histopathology

COMPARATOR

• Visual inspection with or without non-invasive decision support tools (e.g., dermoscopy)

OUTCOME MEASURES

• Diagnostic accuracy

  • Diagnostic odds ratio (the odds of a test being positive among those who have a disease relative to the odds of it being positive among those who do not have the disease)

  • Sensitivity and specificity

  • Positive predictive value and negative predictive value

  • Number needed to biopsy

• Clinical utility

  • Impact on clinical management (including number of biopsies avoided, and treatment decisions)

  • Patient clinical outcomes (e.g., overall survival, disease specific survival)

Literature Screening

A single reviewer conducted an initial screening of titles and abstracts using Covidence³⁴ and then obtained the full texts of studies that appeared eligible for review according to the inclusion criteria. A single reviewer then examined the full-text articles and selected studies eligible for inclusion. A single reviewer also examined reference lists for any additional relevant studies not identified through the search and reviewed a list of publications submitted by the industry representatives to identify relevant studies.

Data Extraction

We extracted relevant data on study characteristics and risk-of-bias items using a data form to collect information on the following:

• Source (e.g., citation information, study type)

• Methods (e.g., study design, study duration and years, participant allocation, allocation sequence concealment, blinding, reporting of missing data, reporting of outcomes, whether the study compared two or more groups)

• Outcomes (e.g., outcomes measured, number of participants for each outcome, number of participants missing for each outcome, outcome definition and source of information, unit of measurement, upper and lower limits [for scales], time points at which the outcomes were assessed)

We contacted study sponsors to provide clarification as needed.

Evidence Synthesis

We performed calculations using Review Manager and RStudio.^35,36 Where evidence synthesis was considered inappropriate, results are reported narratively.

Critical Appraisal of Evidence

We assessed risk of bias using the Quality Assessment of Diagnostic Accuracy Studies–2 (QUADAS-2) tool for diagnostic accuracy studies and the Risk of Bias Assessment Tool for Non-randomized Studies (RoBANS) for non-randomized clinical utility studies (Appendix 2).^37,38

We evaluated the quality of the body of evidence for each outcome according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Handbook.³⁹ The body of evidence was assessed based on the following considerations: risk of bias, inconsistency, indirectness, imprecision, and publication bias. The overall rating reflects our certainty in the evidence.

Results

Clinical Literature Search

The database search of the clinical literature yielded 1,938 citations published between database inception and September 10, 2019. We identified 16 additional studies from other sources. In total, we identified seven studies that met our inclusion criteria. See Appendix 3 for a list of selected studies excluded after full-text review. Figure 2 presents the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram for the clinical literature search.

Figure 2: PRISMA Flow Diagram—Clinical Search Strategy.

Source: Adapted from Moher et al.⁴⁰

Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses.

Characteristics of Included Studies

Characteristics of the included studies are summarized in Table 2. All included studies were conducted in dermatology settings, examining adults (> 18 years old) with suspected melanoma lesions using the DermTech pigmented lesion assay. Five of the studies had some connection to industry funding, either by receiving partial funding to conduct the study or through coauthors serving as employees or consultants of DermTech.

Table 2:

Characteristics of Included Studies

Author, Year	Study Design and Methods	Participants	Outcomes
Brouha et al, 202041	Retrospective cohort study	Lesions that had pigmented lesion assay testing conducted from 53 US dermatology offices	Biopsies conducted or not conducted as a result of a positive or negative PLA test
Ferris et al, 201742	Retrospective cohort study - chart abstraction	People with images of their lesions and completed PLA results for assessment available in their charts	Clinicians were surveyed to determine: (1) the concordance rate between their diagnostic assessment after visual inspection (image-based) vs. with PLA results; and (2) diagnostic accuracy of PLA with the reference standard of histopathology
Ferris et al, 201843	Retrospective cohort study - chart abstraction	People who received pigmented lesion assay from 4 US dermatology centres	Accuracy of PLA assessed with the reference standard of biopsy
Ferris et al, 201944	Prospective cohort study	People with negative PLA results from 5 US dermatology centres	Follow-up and biopsies conducted for individuals with negative PLA results
Gerami et al, 201721	Mixed method– Retrospective cohort (chart abstraction) and prospective cohort study	People with pigmented lesions suspicious for melanoma from 28 sites from the US, Europe, and Australia	Accuracy of PLA assessed with the reference standard of biopsy
Hornberger et al, 201845	Cohorta study	People who received PLA from 2 US dermatology centres	Accuracy of PLA assessed with the reference standard of biopsy
Varedi et al, 201946	Survey	Clinician specialists in managing pigmented lesions	Familiarity and preferences around PLA

Abbreviations: PLA, pigmented lesion assay; US, United States.

^aThe primary purpose of the study was to conduct an economic model; however, they conducted a cohort analysis for diagnostic accuracy.

Risk of Bias in the Included Studies

The risk of bias in the included studies was generally moderate to high (Appendix 2). The largest source of bias was patient selection, with inadequate details regarding study design in most of the studies (additional details in Appendix 2).^42,43,45

The Gerami et al²¹ study enrolled patients using two methods (chart abstraction [N= 195] and consecutive enrollment [N = 203]), with a reported melanoma prevalence of 32% among patients enrolled through chart abstraction and 12% for patients enrolled through consecutive enrollment. However, given the observed difference in melanoma prevalence between the two groups, and recognizing that consecutive enrollment is the preferred method for diagnostic accuracy studies, it is possible that patient selection bias may have been introduced in the chart abstraction group. Therefore, we only focused on the results reported for the consecutively enrolled subgroup of patients in our overall conclusions.

Diagnostic Accuracy

Four studies reported on diagnostic accuracy.^21,42,43,45 We extracted the true positive/false positive/false negative/true negative data from the studies and present sensitivity and specificity calculations in Figure 3 and diagnostic odds ratio calculations in Figure 4. Table 3 lists the positive and negative predictive values that were calculated for this report at various prevalence rates. Our approach was to align with the prevalence rates reported by Cochrane reviews on melanoma. The Cochrane reviews applied a range of prevalence rates based on the actual observed melanoma prevalence rates from the primary studies included in their reviews.^16,17

Figure 3: Sensitivity and Specificity of Pigmented Lesion Assay.

Abbreviations: CI, confidence interval; FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Figure 4: Diagnostic Odds Ratios of Pigmented Lesion Assay.

Abbreviations: CI, confidence interval; OR, odds ratio.

Table 3:

Positive and Negative Predictive Values at Various Prevalence Rates

Author, Year	Prevalence Ratea
	5%		12%		21%
	PPV	NPV	PPV	NPV	PPV	NPV
Ferris et al, 201742	10.7	99.9	23.8	99.7	37.8	99.6
Ferris et al, 201843	36.1	99.7	59.4	99.4	74.0	99.1
Gerami et al, 201721	17.2	98.8	34.9	97.3	51.1	95.9
Hornberger et al, 201845	37.4	99.7	60.7	99.2	75.1	98.8

Abbreviations: NPV negative predictive value; PPV, positive predictive value.

^aAll calculations were conducted by the authors of this report based on data reported in the original publications.

We rated the quality of the evidence as very low, downgrading for risk of bias, inconsistency, and publication bias (Appendix 2). Three of the four studies in the body of evidence have concerns with the risk of bias—specifically patient selection—which contribute to the very low GRADE assessment of the overall body of evidence. This uncertainty can be further demonstrated by examining the diagnostic odds ratios where the three studies with risk of bias^42,43,45 have higher results than Gerami et al²¹ (Figure 4).

In order to consider the best quality of evidence available, we examined a subgroup of consecutively enrolled patients from the Gerami et al study²¹ separately. This study design was the only group to have not introduced a possible source of bias (Appendix 2). The best-quality evidence available was from the Gerami et al study,²¹ which suggests that pigmented lesion assay has a sensitivity of 79% (95% confidence interval [CI] 58%–93%) and specificity of 80% (95% CI 73%–85%; GRADE: Low; Appendix 2). Based on these results, if there was a 12% prevalence rate of melanoma in a population, and assuming all people who received a pigmented lesion assay would have otherwise had a skin biopsy, then for every 1,000 pigmented lesion assays conducted, 704 biopsies would be appropriately avoided (true negative), 94 biopsies would be conducted in individuals who ultimately had melanoma (true positive), 177 biopsies would be conducted on individuals who did not have melanoma (false positive), and 25 people with melanoma would be missed (false negative).

Using the diagnostic accuracy estimates from Gerami et al,²¹ we calculated positive and negative predictive values at three prevalence rates: 5%, 12%, and 21% (Table 3). These values were taken from a relevant Cochrane review¹⁷ and the middle value (12%) is representative of the Ontario prevalence estimates calculating using Ontario population based health administrative data (details in the Economic Evaluation section).

COMPARATIVE DIAGNOSTIC ACCURACY

Figure 5 shows the respective sensitivity and specificity data from one study comparing the diagnostic accuracy of visual inspection alone versus with pigmented lesion assay, with both methods using biopsy as the reference standard.⁴² We also calculated the number needed to biopsy. In the visual inspection group, it took 5.6 biopsies to identify 1 melanoma, and 0.7% (18) of the cohort had melanomas that would have been missed. In the pigmented lesion assay group, 3.8 biopsies were performed for every 1 melanoma identified and 0.2% (5) of the cohort had melanomas that would have been missed outside of research conditions.

Figure 5: Comparative Sensitivity and Specificity of Pigmented Lesion Assay Versus Visual Inspection in Ferris et al, 2017⁴².

Abbreviations: CI, confidence interval; FN, false negative; FP, false positive; PLA, pigmented lesion assay; TN, true negative; TP, true positive.

We rated the quality of the evidence as very low for the diagnostic accuracy of pigmented lesion assay when directly compared to visual inspection alone, downgrading for risk of bias, inconsistency, and publication bias (Appendix 2).

Clinical Utility

No study was found that reported on the impact of pigmented lesion assay on patient-important health outcomes, such as survival or melanoma progression.

Three studies reported on the use of pigmented lesion assay in clinical practice.^41,44,46 Varedi et al⁴⁶ found 21% (9 of 42) of the clinicians surveyed had ordered a pigmented lesion assay adhesive patch test. Among those who had ordered a test, 89% (8 of 9) found the results affected their clinical management decisions. Specifically, eight respondents said the use of pigmented lesion assay resulted in the avoidance of biopsy for negative test results, and six respondents said a positive test led to conducting a biopsy that otherwise would not have been performed. The most common reasons cited for not ordering a pigmented lesion assay test were the need for further validation studies for the test, and the lack of usefulness in their clinical practice.⁴⁶

Ferris et al⁴⁴ conducted a follow-up chart review of patients who participated in a previous study and received a negative pigmented lesion assay result.²¹ Of the 734 patients included, 1.8% (13) had a surgical biopsy in the 12 months following their negative pigmented lesion assay result, and none of those biopsied were diagnosed as having melanoma after histopathology. Study authors implied that all included patients would have received a skin biopsy if the pigmented lesion assay was not conducted, and therefore the absence of biopsy among the majority of those with a negative result demonstrated the impact of the assay on clinical decision making.

Brouha et al⁴¹ conducted a review of a registry of 3,418 pigmented lesion assay tests conducted by 90 clinicians in the United States. Of the 324 lesions returned with a positive pigmented lesion assay test result, 97.5% were biopsied, while eight lesions did not receive a biopsy. Of the 3,094 lesions that had a negative test result, 99.9% were not biopsied while two were biopsied despite the negative pigmented lesion assay result. The majority (93%) of the pigmented lesion assay–negative cases were scheduled for a follow-up at 3, 6, or 12 months, with 7% having follow-up visits at a different time points.⁴¹ There was no difference observed between dermatologists, primary care physicians, nurse practitioners, or physician assistants in the rate of conducting biopsy based on test results received.⁴¹

We rated the quality of evidence as very low for the impact of pigmented lesion assay on clinical decision making, downgrading for risk of bias and publication bias (Appendix 2). There is no evidence of the impact of pigmented lesion assay on patient health outcomes.

Ongoing Studies

We are aware of one ongoing study, not yet published, registered at clinicaltrials.gov (NCT04563949).

Discussion

The best available evidence suggests that pigmented lesion assay has a sensitivity of 79% (95% CI 58%–93%) and a specificity of 80% (95% CI 73%–85%). When we assumed a melanoma prevalence of 12%, the negative predictive value (the probability that an observed negative result is a true negative) was calculated to be 97.3%, with a 2.5% probability of the test missing melanomas with a false negative result. The evidence is uncertain if pigmented lesion assay has an impact on clinical decision making. There was high risk of bias assessed in the body of evidence and the majority of the publications included in this review had some declared conflict of interest because they were industry sponsored, or because authors were employees or consultants for the manufacturer.

Publications identified by industry representatives reported a negative predictive value of over 99%.^47,48 This is a calculation reported in the Gerami et al²¹ study, which used a melanoma prevalence of 7%. There was no reference provided for this prevalence value estimate. As such, we calculated negative predictive value calculations using three prevalence values (5%, 12%, and 21%) modeled on the approach taken by relevant Cochrane reviews.^16,17 These Cochrane reviews used a range of prevalence value estimates for melanoma, as reported in the primary studies included in their reviews.¹⁷ The Gerami et al study had an actual melanoma prevalence of 22% in its primary data set (the “validation group” set), with a 32% prevalence of melanoma among the individuals enrolled from the archival chart group, and a prevalence of 12% from the consecutively enrolled patients.²¹ We acknowledge that not all of the studies included in the Cochrane review may be relevant to the Ontario context, which presents certain limitations. However, in the context of this review, the selected Cochrane review is the best quality of evidence regarding standard care in Ontario, which is visual inspection and dermoscopy by dermatologists.¹⁷

The original publication by Gerami et al reported pigmented lesion assay to have a sensitivity of 91% and specificity of 69% as their primary finding.²¹ This calculation was based on a cohort of 398 people enrolled in the study in two different ways: chart abstraction and consecutive enrollment. However, we had concerns with patient selection bias for the overall cohort in the Gerami et al²¹ study, and so we chose to base our accuracy calculations on only the data reported for the subgroup of patients who were consecutively enrolled. As such, the sensitivity and specificity presented in the paper's abstract as their primary findings are different from our results for the selected subgroup of patients. Our calculated sensitivity and specificity for this subgroup of patients align with that presented in the body of the original publication for this subgroup.²¹

The accuracy studies of pigmented lesion assay were all undertaken in dermatology clinics. Based on a Cochrane review, primary care providers tend to identify melanoma lesions less accurately compared to specialists when using on visual inspection alone,¹⁶ and it is therefore possible that pigmented lesion assay may have a greater impact for primary care providers. However, both primary care and specialist clinicians have expressed a very low threshold for false negatives, meaning it is preferred to biopsy healthy lesions than to miss a melanoma (Wade Mitchell, MD, phone communication, November 2019; Sakina Walji, MD, phone communication, November 2019; Frances Wright, MD, phone communication, November 2019). There may also be harms related to false results due to an inaccurate test. A false negative would miss potential melanomas and this missed opportunity for early diagnosis could increase mortality risk.² Additionally, a false positive may result in unnecessary treatment, such as excision of an otherwise healthy lesion, and stress for the individual and their families.

Implementation Considerations

In Ontario, there may be a wait to receive a biopsy when referred to a dermatologist or surgeon, which could potentially delay diagnosis and any subsequent treatment. Since earlier identification of melanoma can dramatically impact patient outcomes, delays can be concerning. One study found wait times to be less than 3 months,¹¹ which was confirmed by clinical stakeholders to be representative of the current Ontario practice (Wade Mitchell, MD, phone communication, November 2019; Sakina Walji, MD, phone communication, November 2019). This falls within the global clinical guidelines in dermatology, which recommend follow-up within 3 to 6 months for suspicious lesions assessed with visual inspection alone to determine need for biopsy.⁴⁹

There may be potential inequity in the implementation of pigmented lesion assay if the test was found to be effective and then was only sold to certain medical professionals (e.g., dermatologists) who may not be easily accessible to all Ontarians. It is generally believed that rural and remote areas of Ontario have a more difficult time accessing specialists, including dermatologists, and therefore may have more limited access to biopsy. To support improved patient care, Ontario physicians are using both formal strategies (such as the increased use of virtual care networks) and informal strategies (for example, a network of primary care physicians who refer to their most skilled colleague as an intermediate step to alleviate some need for specialist care [Dr. Wade Mitchell, phone communication, November 8, 2019]). There is also the potential for pigmented lesion assay to be a beneficial alternative for individuals otherwise unable or unwilling to undergo traditional biopsy. The Preferences and Values Evidence section explores this in more detail. Delays in diagnosis can also be due to patient factors, with one study finding that a third of patients had delayed seeking medical attention for more than 6 months.⁵⁰ The reasons for this delay were unclear; however, lack of melanoma recognition has been hypothesized to contribute to the delay.⁵⁰

Pigmented lesion assay is designed as a “rule out test,” meaning it is intended to help identify lesions that may not require a biopsy. Current practice relies on visual inspection, and the use of tools such as dermoscopy in Ontario is not widespread in primary care. When considering the potential adoption of pigmented lesion assay in Ontario, it may be necessary to examine best practices for the overall management of suspected melanoma lesions and compare all available strategies for determining which lesions require biopsy.

Conclusions

Evidence suggests that pigmented lesion assay has a sensitivity of 79% (95% CI 58%-93%) and a specificity of 80% (95% CI 73%-85%), which correspond to a negative predictive value of 97.3% when calculated using a 12% prevalence rate. The evidence is uncertain about the effect of pigmented lesion assay when directly compared to visual inspection alone. We did not identify any evidence of the impact of pigmented lesion assay on patient health outcomes. The evidence is uncertain about whether pigmented lesion assay has an impact on clinical decision making.

Economic Evidence

Research Question

What is the cost-effectiveness of pigmented lesion assay compared with standard care for people with suspected melanoma skin lesions?

Methods

Economic Literature Search

We performed an economic literature search on September 11, 2019, to retrieve studies published from database inception until the search date. To retrieve relevant studies, we developed a search using the clinical search strategy with an economic and costing filter applied. In addition to the databases used for the clinical search, we also used the Ovid interface in the Cochrane Central Register of Controlled Trials.

We created database auto-alerts in MEDLINE and Embase, and monitored them for the duration of the assessment period. We also performed a targeted grey literature search of health technology assessment agency sites, systematic review registries, and the Tufts Cost-Effectiveness Analysis Registry. The grey literature search was updated on December 18, 2019. See the Clinical Literature Search section, above, for further details on methods used. See Appendix 1 for literature search strategies, including all search terms.

Eligibility Criteria

STUDIES

Inclusion criteria:

• English-language full-text publications

• Studies published from database inception until September 11, 2019

• Cost–benefit analyses, cost-effectiveness analyses, cost-minimization analyses, cost–consequence analyses, or cost–utility analyses

Exclusion criteria:

• Narrative reviews, letters/editorials, case reports, commentaries, conference abstracts, posters, unpublished studies

PARTICIPANTS

• Adults with suspected melanoma skin lesions

INTERVENTIONS

Inclusion Criteria

• Non-invasive, adhesive patch genetic-based test to support the identification of suspected lesions to biopsy for melanoma diagnosis (i.e., pigmented lesion assay)

Exclusion Criteria

• Genetic tests for prognosis (e.g., BRAF) or risk of developing melanoma (e.g., CDKN2A)

• Non-genetic based interventions to support the identification of suspected lesions to biopsy for melanoma diagnosis (e.g., dermoscopy)

REFERENCE STANDARD

• Skin biopsy and histopathology

COMPARATOR

• Visual inspection with or without non-invasive decision support tools (e.g., dermoscopy)

OUTCOME MEASURES

• Costs

• Health outcomes (e.g., quality-adjusted life years [QALYs], life years, rate or number of biopsies, number-needed-to-biopsy, rate or number of excisions)

• Incremental costs

• Incremental effectiveness

• Incremental cost-effectiveness ratio (ICER)

Literature Screening

A single reviewer conducted an initial screening of titles and abstracts using Covidence³⁴ and then obtained the full texts of studies that appeared eligible for review according to the inclusion criteria. A single reviewer then examined the full-text articles and selected studies eligible for inclusion.

Data Extraction

We extracted relevant data on study characteristics and outcomes to collect information about the following:

• Source (e.g., citation information, study type)

• Methods (e.g., study design, analytic technique, perspective, time horizon, population, intervention[s], comparator[s])

• Outcomes (e.g., health outcomes, costs, incremental cost-effectiveness ratios)

We contacted study authors to provide clarification as needed.

Study Applicability and Limitations

We determined the usefulness of each identified study for decision-making by applying a modified quality appraisal checklist for economic evaluations originally developed by the National Institute for Health and Care Excellence (NICE) in the United Kingdom to inform the development of NICE's clinical guidelines.⁵¹ We modified the wording of the questions to remove references to guidelines and to make it specific to Ontario. Next, we separated the checklist into two sections. In the first section, we assessed the applicability of each study to the research question (directly, partially, or not applicable). In the second section, we assessed the limitations (minor, potentially serious, or very serious) of the studies that we found to be directly or partially applicable.

Results

Economic Literature Search

The database search of the economic literature yielded 16 citations published between database inception and September 11, 2019, after removing duplicates. We excluded a total of 14 articles based on information in the title and abstract. We then obtained the full text of the two potentially relevant articles for further assessment. One study met the inclusion criteria. Figure 6 presents the flow diagram for the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram for the economic literature search.

Figure 6: PRISMA Flow Diagram—Economic Search Strategy.

Source: Adapted from Moher et al, 2009.⁴⁰

Overview of Included Economic Studies

Our review identified only one study that met the inclusion criteria (Hornberger et al⁴⁵). We summarized the characteristics and results of this study in Table 4.

Table 4:

Results of Economic Literature Review—Summary

Author, Year, Country	Analytic Technique, Study Design, Perspective, Time Horizon	Population	Intervention(s) and Comparator(s)	Results
				Health Outcomes	Costs	Cost-Effectiveness
Hornberger et al, 2018,45 USA	Cost-effectiveness analysis and cost-utility analysis Decision-analytic model US health care payer perspective Time horizon: not reported Discount rate: not reported	Patients with pigmented skin lesions suggestive of melanoma	Pigmented lesion assay strategy (PLA): Positive PLA result is followed up with a biopsy and histopathology, and negative PLA result is followed up with surveillance. Biopsied lesions diagnosed as malignant or atypical are excised, and benign lesions are followed up with surveillance. Standard care strategy (VAH): Lesions suggestive of melanoma after visual inspection are biopsied and examined for histopathology. Biopsied lesions diagnosed as malignant or atypical are excised, and benign lesions are followed up with surveillance.	QALYs: PLA: 16.743 VAH: 16.707 Rate of initial biopsy: PLA: 13.3% VAH: 69.0% Rate of excision: PLA: 7.5% VAH: 18.8% Number-needed-to-biopsy: PLA: 2.7 VAH: 15.7 Excisions per melanoma: PLA: 1.18 VAH: 2.49 Proportion of patients with immediate diagnosis: PLA: 75.5% VAH: 68.0%	Total direct medical cost per patient (2017 USD): PLA (at a theoretical cost of $0): $1,928 ($126 for initial biopsy and subsequent excisions; $167 for office visits for diagnosis; $19 for surveillance; $1,616 for treatment of melanoma) VAH: $2,875 ($521 for initial biopsy and subsequent excisions; $167 for office visits for diagnosis; $138 for surveillance; $2,049 for treatment of melanoma) Difference (PLA at a theoretical cost of $0): -$947 Total indirect cost per patient (2017 USD): PLA: $54 VAH: $101 Difference: -$47	PLA strategy is dominant (cost saving and more effective)

Abbreviations: PLA, pigmented lesion assay; QALY, quality-adjusted life years; US, United States; USA, United States of America; USD, US dollars; VAH, visual assessment followed by surgical biopsy and histopathologic assessment (standard care).

Hornberger et al⁴⁵ is an industry-sponsored study, which evaluated the cost-effectiveness of pigmented lesion assay in people with pigmented skin lesions suggestive of melanoma. The study compared two diagnostic strategies using a decision-analytic model: pigmented lesion assay versus the standard care pathway (visual assessment followed by a biopsy and histopathologic assessment). For people who receive pigmented lesion assay, positive pigmented lesion assay results are followed up with a biopsy and histopathology, and negative pigmented lesion assay results are followed up with surveillance. For people who receive standard care, all lesions suggestive of melanoma after visual assessment are biopsied and followed up with histopathology. Management of the biopsied lesions depends on the histopathologic diagnosis: malignant or atypical lesions would be excised, and benign lesions would be followed up with surveillance. The analysis was conducted from a US health care payer perspective and included both direct medical costs and indirect costs. Direct medical costs included short-term costs such as physician visits, biopsy and excisions, histopathology, and management of complications, as well as long-term costs, such as surveillance and stage-related melanoma treatment.

The study found that the proportion of patients undergoing initial surgical biopsy was 69% with standard care, and 13.3% with pigmented lesion assay. The proportion of patients undergoing subsequent excisions was also estimated to be higher with standard care compared with pigmented lesion assay (18.8% vs. 7.5%). The number-needed-to-biopsy was estimated to decrease from 15.7 with standard care to 2.7 with pigmented lesion assay. The study also found pigmented lesion assay to be associated with higher QALYs than standard care (16.743 vs. 16.707; difference 0.036). Assuming a theoretical cost of $0 for pigmented lesion assay, the assay was found to save $947 USD in direct medical costs (of which $395 USD was due to reduction in biopsy and subsequent excision, $433 USD due to reduction in stage-related melanoma treatment costs, and $119 USD due to reduction in surveillance costs). Therefore, the authors concluded that at a selling price of $500 USD, pigmented lesion assay would lead to a cost reduction of $447 USD.

Applicability and Limitations of the Included Study

Appendix 4 (Table A4) provides the results of the quality appraisal checklist for economic evaluations applied to the included study. Although the Hornberger et al study⁴⁵ included the population of interest and demonstrated the economic implications of pigmented lesion assay, the study was considered partially applicable because it was conducted outside of the Canadian context. We then assessed the limitations of the study and presented the results in Appendix 4, Table A5.

The study has several limitations. Firstly, diagnostic accuracy of pigmented lesion assay may have been overestimated. The study used a sensitivity of 93% and a specificity of 92% based on data from a cohort of people who received pigmented lesion assay from two US dermatology sites (Hornberger et al, 2018).⁴⁵ This evidence was assessed as having very low certainty based on Grading of Recommendations Assessment, Development, and Evaluation (GRADE; see Clinical Evidence section). Our clinical review suggested that pigmented lesion assay has a sensitivity of 79% and a specificity of 80%, based on the best quality of evidence (a subgroup analysis from Gerami et al²¹).

Secondly, the diagnostic accuracy of visual assessment may have been underestimated when compared with published literature. For visual assessment, the study used a sensitivity of 84% (referencing 128 cases in the study itself, but this calculation could not be verified) and a specificity of 32% based on Ferris et al⁴² (in which 45 dermatologists evaluated 60 images of clinically atypical pigmented lesions). However, the Ferris et al study reported a sensitivity of 95%.⁴² We also found a recently published Cochrane systematic review on the diagnostic accuracy of visual assessment for suspected melanoma.¹⁷ For in-person visual inspection by specialists (usually conducted using a dermoscopy tool), the summary sensitivity and specificity were estimated to be 78.4% and 95.5%, respectively, based on six studies of 778 suspicious skin lesions.¹⁷

Lastly, many parameters and assumptions used by the economic analysis were not reported in the publication, making it difficult to assess the model and results (e.g., time horizon, discount rate, baseline patient characteristics, unit cost of excision, rates and costs of complications, proportion of patients in different disease stages after being diagnosed with melanoma, and model structure used to calculate long-term cost and QALYs). For example, the cost of initial surgical biopsy may have been overestimated compared to other published sources. The study assumed the unit cost of the initial surgical biopsy to be $373.00 (2017 USD; data source not provided). However, according to the US Medicare and Medicaid physician fee schedule, if performed in a physician's office, the cost of a shave biopsy ranges from $99.83 to $162.90 (2019 USD; more expensive for larger lesions), and the cost of an excisional biopsy ranges from $127.22 to $172.27 (2019 USD).^52,53

Overall, the study was deemed to have potentially serious limitations.

Discussion

Our literature review showed that economic evidence for pigmented lesion assay is sparse, as this is a relatively new technology. We identified one cost-effectiveness study that was only partially applicable to our research question because while the study had a similar study population, intervention, and comparator, its model was built from a US health care payer perspective. Due to variations in clinical practice patterns and health care costs, the results of this economic analysis may not be applicable to the Canadian setting. Biopsy rates are known to vary widely across different countries,⁵⁴ and health care resource use and costs differ between the US and Canada. For example, the cost of biopsy is much lower in Canada ($29.60 CAD for an incisional biopsy, and between $43.60 CAD and $92.15 CAD for an excisional biopsy) compared to the US.⁵⁵

Conclusions

Our economic literature review identified one study that evaluated the cost-effectiveness of pigmented lesion assay. While the study suggested that pigmented lesion assay reduced cost and may improve health outcomes in the US setting, we found it to have potentially serious limitations affecting the certainty of the study results. Therefore, the cost-effectiveness of pigmented lesion assay is currently uncertain based on the results of our economic literature review.

Primary Economic Evaluation

Pigmented lesion assay is proposed as a test to rule out melanoma and the need for skin biopsy.⁴⁵ As such, its clinical utility is in supporting the selection of patients who truly require biopsy. Pigmented lesion assay is not used as a diagnostic test because people with positive pigmented lesion assay results still need to receive diagnostic assessment as per standard care (biopsy and histopathology). We consider the main benefit of pigmented lesion assay as a means to improve the diagnostic process in the short-term (e.g., reduce unnecessary biopsies and excisions, reduce referrals to specialists). We decided not to conduct a primary economic evaluation for the following reasons: (1) there is low to very low quality clinical evidence about test characteristics and relevant patient outcomes (e.g., survival, quality of life); and (2) the potential avoidance of skin biopsies and specialist referrals with the use of pigmented lesion assay can be captured in the budget impact analysis.

Budget Impact Analysis

To consider the potential adoption of pigmented lesion assay in Ontario, we assessed the budget impact of the test being used by either only primary care providers or only specialists (e.g., dermatologists). We included the specialist setting because the accuracy studies for pigmented lesion assay were all undertaken in dermatology clinics. We also included the primary care setting because primary care providers tend to have less experience diagnosing melanoma compared to specialists, and therefore the impact of pigmented lesion assay may be greater in primary care.

Research Question

What is the 5-year budget impact for the Ontario Ministry of Health of publicly funding pigmented lesion assay for people with suspected melanoma skin lesions in Ontario in either (a) only primary care (used by primary care providers), or (b) only specialist care (used by specialists such as dermatologists)?

Methods

Analytic Framework

We estimated the budget impact of publicly funding pigmented lesion assay using the cost difference between two scenarios:

• 1.Current Scenario: current clinical practice without pigmented lesion assay
• 2.New Scenario: anticipated clinical practice with pigmented lesion assay for either (a) only primary care providers, or (b) only specialists

Figure 7 presents the budget impact model schematic.

Figure 7: Schematic Model of Budget Impact.

Key Assumptions

• We only considered short-term costs related to pigmented lesion assay because the main benefit of the test is to reduce unnecessary biopsies and referrals to specialists

• We assumed the unit cost of pigmented lesion assay to stay constant over the next 5 years

• For simplicity, we assumed the sensitivity and specificity of histopathology to be close to 100% since it is the current gold standard for diagnosing melanoma. In reality, it is an imperfect reference standard, since some biopsy approaches (i.e., incisional) can yield false positives and false negatives. As a result, we used a lower sensitivity and specificity in the sensitivity analyses

Target Population

The target population are people with suspected melanoma (i.e., those who would traditionally require a biopsy). Therefore, the number of people with suspected melanoma can be approximated from the number of skin biopsies performed for suspected melanoma.

First, we estimated the number of skin biopsies using administrative data from the Ontario Health Insurance Plan (OHIP) claims database. Based on consultation with clinical experts, we used the following OHIP fee codes to capture skin biopsy procedures (Wade Mitchell, MD, phone communication, November 2019; Sakina Walji, MD, phone communication, November 2019):

• Biopsies (any method, with or without suture): Z113, Z116

• Excision of pre-malignant lesions including biopsy of each lesion:

  • Face or neck: R160 (1 lesion), R161 (2 lesions), R162 (3+ lesions)

  • Other areas: R163 (1 lesion), R164 (2 lesions), R165 (3+ lesions)

• Excision of malignant lesions including biopsy of each lesion:

  • Face or neck: R048 (1 lesion), R049 (2 lesions), R050 (3+ lesions)

  • Other areas: R094 (1 lesion), R040 (2 lesions), R041 (3+ lesions)

Since skin biopsies can also be performed for other diseases (e.g., other non-melanoma skin cancers, warts, eczema, psoriasis), we only included skin biopsies performed for suspected melanoma. The following International Classification of Diseases (ICD9) diagnosis codes were used to capture patients presenting with a suspected melanoma lesion (or lesions) before melanoma was diagnosed or ruled out.

• ICD9 code included in the reference case:

  • 216: benign neoplasms—skin (e.g., pigmented naevus, dermatofibroma)

    • We assumed that this is the appropriate diagnosis code for a patient with suspected melanoma lesion, since most pigmented lesions biopsied are usually benign

• ICD9 codes included in the sensitivity analyses only:

  • 172: malignant neoplasms—melanoma of skin

    • For the reference case, we excluded skin biopsies associated with this diagnosis code because these lesions are considered obviously melanoma (biopsy is performed to confirm a melanoma diagnosis rather than to rule it out) and therefore is not our population of interest. (Pigmented lesion assay is not for lesions that are obviously benign or obviously melanoma.) However, we included 10% patients with this diagnosis code for sensitivity analysis.

  • 173: other skin malignancies

    • For the reference case, we excluded skin biopsies associated with this diagnosis code because it may be used for other skin cancers and not melanoma. However, it is possible that when patients present with suspected melanoma lesions, physicians may use a diagnosis code of a condition similar to melanoma. Therefore, we included 10% of patients with this diagnosis code for sensitivity analysis.

  • 232: carcinoma in situ—skin

    • Same comment as for 173: other skin malignancies (above)

  • 709: other disorders skin and subcutaneous tissue

    • Same comment as for 173: other skin malignancies (above)

We found that in 2016 (the most recent year of complete data), a total of 43,042 skin biopsies were performed for suspected melanoma in Ontario (10,616 by general practitioners, 26,516 by dermatologists, 3,747 by general surgeons, and 2,163 by plastic surgeons; see Table 5). The majority of skin biopsies were coded as Z113 and Z116 (incisional biopsies). Together, specialists (dermatologists, general surgeons, and plastic surgeons) were more likely to perform excisional biopsies (R codes) compared to general practitioners.

Table 5:

Number of Skin Biopsies Performed for Suspected Melanoma in Ontario in 2016

OHIP Code	Description of Different Types of Skin Biopsies	Cost of Procedurea	General Practitioner		Dermatologist		General Surgeon		Plastic Surgeon
			Number of Biopsiesb	%	Number of Biopsiesb	%	Number of Biopsiesb	%	Number of Biopsiesb	%
Z113	Biopsy without suture	$29.60	3,458	33%	10,803	41%	643	17%	207	10%
Z116	Biopsy with suture	$29.60	6,080	57%	7,868	30%	1,703	45%	589	27%
R160	Excision and biopsy: Pre-malignant, face or neck, 1	$53.20	117	1%	406	2%	140	4%	152	7%
R161	Excision and biopsy: Pre-malignant, face or neck, 2	$87.40	9	0%	44	0%	19	1%	15	1%
R162	Excision and biopsy: Pre-malignant, face or neck, 3+	$174.75	31	0%	63	0%	1	0%	15	1%
R163	Excision and biopsy: Pre-malignant, other area, 1	$43.60	533	5%	4,524	17%	590	16%	492	23%
R164	Excision and biopsy: Pre-malignant, other area, 2	$71.80	60	1%	933	4%	110	3%	130	6%
R165	Excision and biopsy: Pre-malignant, other area, 3+	$143.55	22	0%	1,826	7%	39	1%	72	3%
R048	Excision and biopsy: Malignant, face or neck, 1	$92.15	86	1%	17	0%	203	5%	258	12%
R049	Excision and biopsy: Malignant, face or neck, 2	$139.20	8	0%	0	0%	22	1%	39	2%
R050	Excision and biopsy: Malignant, face or neck, 3+	$233.00	0	0%	0	0%	9	0%	12	1%
R094	Excision and biopsy: Malignant, other area, 1	$58.15	199	2%	28	0%	237	6%	138	6%
R040	Excision and biopsy: Malignant, other area, 2	$95.70	11	0%	1	0%	23	1%	27	1%
R041	Excision and biopsy: Malignant, other area, 3+	$191.40	2	0%	3	0%	8	0%	17	1%
	Total Procedures		10,616	100%	26,516	100%	3,747	100%	2,163	100%

^aCost of procedure was based on the Ontario Schedule of Benefit for Physician Services.

^bNumber of biopsies were obtained from IntelliHealth Ontario.

To estimate the annual number of skin biopsies over the next 5 years, we obtained data from the most recent 4 years (2013 to 2016) to analyze the trend over time. The data showed that the number of skin biopsies remained stable over time. Therefore, we assumed that the number of skin biopsies performed over the next 5 years would remain the same (Table 6).

Table 6:

Size of Target Population in Primary Care and Specialist Care Settings

	Year 1	Year 2	Year 3	Year 4	Year 5
Number of people with suspected melanoma who would traditionally require biopsy in primary carea	53,080	53,080	53,080	53,080	53,080
Number of skin biopsies performed by general practitioners	10,616	10,616	10,616	10,616	10,616
Number of people with suspected melanoma who would traditionally require biopsy in specialist care	32,426	32,426	32,426	32,426	32,426
Number of skin biopsies performed by dermatologists, general surgeons and plastic surgeons	32,426	32,426	32,426	32,426	32,426

^aThis includes both patients who are biopsied by general practitioners (20%) and patients who are referred by general practitioners to specialists (80%). Please note that the specialists may conduct visual inspection again and decide not to biopsy some patients referred by general practitioners.

Next, we estimated the number of people with suspected melanoma in each setting (Table 6). In the primary care setting, people with suspected melanoma usually present to a general practitioner. The general practitioner then determines which lesions are clinically concerning enough to require a biopsy. However, not all general practitioners perform biopsies. An Ontario study (Lutz et al¹¹) showed that for various reasons, 81% of the family physicians surveyed did not always perform an excisional biopsy (half of these physicians may consider an incisional biopsy as an alternative). When family physicians did not want to perform a biopsy themselves, they would refer patients to a specialist (dermatologist, general surgeon, or plastic surgeon). According to clinical experts, the majority of patients are referred to dermatologists, and patients are more likely to be referred to surgeons if the lesion is too large to excise or is in a challenging anatomical location (Wade Mitchell, MD, phone communication, November 2019; Sakina Walji, MD, phone communication, November 2019).

Based on the Lutz et al study,¹¹ as well as IntelliHealth data showing that the majority of skin biopsies were being performed by specialists combined (Table 6), we assumed roughly 20% of patients who present with suspected melanoma would receive a biopsy by a general practitioner (with the remaining 80% being referred to a specialist who may or may not conduct a biopsy). Therefore, the number of people who present with suspected melanoma in the primary care setting was estimated to be 53,080 per year (10,616 annual biopsies by general practitioners ÷ 20%).

In the specialist care setting, we assumed that all dermatologists and surgeons perform biopsies. Therefore, the number of people with suspected melanoma who require biopsies in the specialist care setting equals the sum of skin biopsies performed by dermatologists, general surgeons, and plastic surgeons (26,516 + 3,747 + 2,163 = 32,426 biopsies per year; Tables 5 and 6).

Current Intervention Mix

Currently, pigmented lesion assay is not publicly funded or widely used in Ontario. Therefore, all patients in the current scenario would receive standard care (visual assessment followed by a biopsy and histopathology assessment).

Uptake of the New Intervention and New Intervention Mix

In the new scenario, the annual uptake of pigmented lesion assay was expected to be 1%, 3%, 5%, 7%, and 9% over the next 5 years (DermTech Canada Inc., email communication, November 2020). The number of people receiving standard care plus pigmented lesion assay in each setting was calculated based on the uptake rates (Table 7). In the sensitivity analyses, we also explored the impact of lower uptake rates of 1% to 5% in years 1 through 5 and higher uptake rates of 3% to 15% in years 1 through 5.

Table 7:

Number of People Receiving Standard Care and Pigmented Lesion Assay (Current Scenario and Future Scenario)

	Year 1	Year 2	Year 3	Year 4	Year 5
Uptake Rates	1%	3%	5%	7%	9%
Primary Care Setting
Current Scenario
Number of people receiving standard care	53,080	53,080	53,080	53,080	53,080
Future Scenario
Number of people receiving standard care	52,549	51,488	50,426	49,364	48,303
Number of people receiving standard care plus PLA	531	1,592	2,654	3,716	4,777
Specialist Care Setting
Current Scenario
Number of people receiving standard care	32,426	32,426	32,426	32,426	32,426
Future Scenario
Number of people receiving standard care	32,102	31,453	30,805	30,156	29,508
Number of people receiving standard care plus PLA	324	973	1,621	2,270	2,918

Abbreviation: PLA, pigmented lesion assay.

Clinical Pathways and Model Structures

We created two simple decision tree models to represent the diagnostic pathways for each testing strategy (Figures 8 and 9), and to estimate the per-person cost associated with each strategy.

Figure 8: Diagnostic Pathways Through Primary Care Providers, With and Without Pigmented Lesion Assay.

Abbreviations: GP, general practitioner; PLA, pigmented lesion assay.

Note: Square represents a decision node, and circles represent a chance node.

Figure 9: Diagnostic Pathways Through Specialists, With and Without Pigmented Lesion Assay.

Abbreviations: PLA, pigmented lesion assay.

Note: Square represents a decision node, and circles represent a chance node.

Standard Care:

• In the primary care setting, patients with lesions suggestive of melanoma would either receive a biopsy from a general practitioner (20%) or be referred to a specialist (80%; for biopsy or further assessment)

• In the specialist care setting, all patients with lesions suggestive of melanoma would receive a biopsy from a specialist

Standard Care Plus Pigmented Lesion Assay:

• In both settings, all patients with positive pigmented lesion assay results would then receive a surgical biopsy and subsequent histopathology assessment. If the histopathologic diagnosis is positive, patients would receive a wide excision to remove the lesion completely. Patients with negative pigmented lesion assay results would not receive a biopsy and would be scheduled for surveillance as per standard of care

Clinical Parameters

We obtained the clinical parameters from the Clinical Evidence review, such as prevalence and diagnostic accuracy of pigmented lesion assay (Table 8).

Table 8:

Clinical Parameters

Parameter	Mean	Source
Prevalence of Melanoma
Primary care setting	9% (range: 4%–16%)	Dinnes et al 201816
Specialist care setting	12% (range: 5%–21%)	Dinnes et al 201817
Pigmented Lesion Assay
Sensitivity	79% (95% CI: 58%–93%)	Clinical evidence review; Gerami et al 201721
Specificity	80% (95% CI: 73%–85%)	Clinical evidence review; Gerami et al 201721

Abbreviation: CI, confidence interval.

For the reference case, we assumed specialists would perform biopsies for all patients referred by general practitioners (e.g., to offer patients reassurances that their lesions are not cancerous). In a scenario analysis, we assumed specialists would conduct visual assessments on referred patients and decide to not biopsy in some cases. We obtained the diagnostic accuracy of visual assessment by specialists with dermoscopy (sensitivity of 78.4% and specificity of 95.5%) from a recent Cochrane systematic review.¹⁷

Resources and Costs

The budget impact analysis included the following types of short-term costs in order to evaluate the immediate impact of pigmented lesion assay on health care resources:

• Physician visits for diagnosis and post-biopsy/excision follow-up

• Pigmented lesion assay

• Biopsy and histopathologic assessment

• Excision of malignant lesions

According to clinical experts, it is uncommon to have serious complications after biopsy and excision. Less than 5% of patients may experience minor complications such as wound dehiscence (opening of a surgical wound) and infection (Wade Mitchell, MD, phone communication, November 2019; Sakina Walji, MD, phone communication, November 2019). We assumed that all patients would have a follow-up physician visit within 7 to 10 days after biopsy or excision to remove the suture, receive biopsy results, and receive treatment if there are any complications.

We did not include the following types of long-term costs because predicting these would require building a disease model and many assumptions (e.g., regarding disease progression and survival):

• Treatment for later-stage melanoma

• Physician visits for surveillance

The cost inputs we used are presented in Table 9.

Table 9:

Cost Inputs (2019 CAD)

Parameter	Value	Source
Physician Visit
General practitioner	$77.20	Schedule of Benefits (A003)
Dermatologist	$72.15	Schedule of Benefits (A025)
Pigmented Lesion Assay
Test (including test kit, shipping, assay handling, reagents, analysis, etc.)	$395.00	DermTech Canada Inc., email communication, September 24, 2019
Physician sample collection fee	$50.00	DermTech Canada Inc., email communication, September 24, 2019
Histopathology
Histopathology assessment (average cost based on values below)	$24.95	$18.75 for histopathology + ($62.04 for immunohistochemistry × 10%) = $24.95; assuming 10% are difficult cases and would need immunohistochemistry together with histopathology
Histopathology	$18.75	Schedule of Benefits (L720)
Special immunohistochemistry stain	$62.04	Schedule of Benefits ($20.68 per test [L728] × 3 = $62.04; assuming 3 markers/tests needed)
Proportion requiring special stain	10%	Clinical expert opinion
Skin Biopsy (weighted average cost)
Primary care setting	$55.52	Schedule of Benefits ($32.73 for weighted average cost of all biopsies performed by GPsa + $11.65 for cryotherapy treatment [Z117] + $11.15 if performed outside of hospital [E542] = $55.52)
Specialist care setting	$66.01	Schedule of Benefits ($43.21 for weighted average cost of all biopsies performed by specialistsa + $11.65 for cryotherapy treatment [Z117] + $11.15 if performed outside of hospital [E542] = $66.01)
Wide excision (for treatment)	$355.51	Schedule of Benefits (R010)

Abbreviations: CAD, Canadian dollars; GP, general practitioner.

^aCalculated using costs in Table 5.

Internal Validation

The secondary health economist conducted formal internal validation. This process included checking for errors and ensuring the accuracy of parameter inputs and equations in the budget impact analysis.

Analysis

The budget impact of publicly funding pigmented lesion assay was calculated by multiplying the difference in per-person cost between the two testing strategies by the number of people expected to receive pigmented lesion assay. We conducted both a reference case analysis and sensitivity analyses. Our reference case analysis represents the analysis with the most likely set of input parameters and model assumptions. In the sensitivity analyses, we explored how the results are affected by varying input parameters and model assumptions (Table 10).

Table 10:

Parameters for the Reference Case and Sensitivity Analyses

Parameter	Reference Case	Sensitivity Analysis
Size of the target population	All biopsies with ICD9 code 216: Primary care: 53,080 per year Specialist care: 32,426 per year	Add 10% of biopsies with ICD9 codes 172, 173, 232, and 709: Primary care: an additional 15,842 per year Specialist care: an additional 8,798 per year
Proportion of patients who received biopsies from GP	20%	15%; 25%
Prevalence (pre-test probability of melanoma)	Mean value provided by Dinnes et al, 201816: Primary care: 9% Specialist care: 12%	Range provided by Dinnes et al, 2018:16,17 Primary care: 4%; 16% Specialist care: 5%; 21%
Uptake rate of PLA in years 1–5	1%, 3%, 5%, 7%, 9%	Lower bound: 1%, 2%, 3%, 4%, 5% Higher bound: 3%, 6%, 9%, 12%, 15%
Diagnostic accuracy of visual assessment in referred lesions	Assuming specialists would conduct biopsy on all patients referred from GPs (sensitivity of 100% and specificity of 0%)	Assuming specialists would conduct visual assessment on referred lesions and applied diagnostic accuracy values obtained from the literature Dinnes et al, 201817: sensitivity of 74.8% and specificity of 95.5% Hornberger et al, 201845: sensitivity of 84% and specificity of 32%
Diagnostic accuracy of histopathology	Assuming 100% accuracy since histopathology is the current gold standard for diagnosis	Gerami et al, 201721: sensitivity of 89% and specificity of 91%
Diagnostic accuracy of PLA	Best quality evidence from Gerami et al, 201721: sensitivity of 79% and specificity of 80%	Hornberger et al, 201845: sensitivity of 93% and specificity of 92%
Cost of PLA	$445 per test	± 30%
Cost of biopsy	Based on real-world costs in Ontarioa: Primary care: $55.52 Specialist care: $66.01	± 50%

Abbreviations: GP, general practitioner; ICD, international classification of diseases; PLA, pigmented lesion assay.

^aSee Table 9 for source information and calculations.

Results

Reference Case

Cost Per Person

Using the diagnostic pathway decision tree models (Figures 8 and 9), we estimated the per-person cost related to each testing strategy (Table 11). In the primary care setting, the total cost per person was approximately $595.00 for standard care plus pigmented lesion assay, and $335.53 for standard care alone. Compared to standard care alone, using pigmented lesion assay reduced some costs related to physician visits (−$112.44 resulting from fewer referrals to specialists and fewer follow-up visits needed after biopsy and excision). Pigmented lesion assay also reduced some costs related to avoided biopsy and histopathology (−$66.38), as well as avoided excision (−$6.72). However, these savings were relatively small compared to the cost of the pigmented lesion assay itself ($445.00). Therefore, adding pigmented lesion assay to the current diagnostic pathway for primary care physicians would increase the per-person cost by $259.47.

Table 11:

Short-Term Cost-Effectiveness Results in Primary Care and Specialist Care Settings, Per Person

	Standard Care Plus PLA	Standard Care Alone	Difference
Primary Care Setting
Cost of physician visits	$102.23	$214.66	−$112.44
Cost of PLA	$445.00	$0.00	$445.00
Cost of biopsy and histopathology	$22.49	$88.87	−$66.38
Cost of excision of malignant lesions	$25.28	$32.00	−$6.72
Total Cost	$595.00	$335.53	$259.47
Number of biopsies	0.253	1	−0.747
Number of excisions	0.071	0.09	−0.019
Number of missed melanoma diagnoses	0.019	0	0.019
Specialist Care Setting
Cost of physician visits	$98.53	$152.96	−$54.43
Cost of PLA	$445.00	$0.00	$445.00
Cost of biopsy and histopathology	$24.63	$90.97	−$66.33
Cost of excision of malignant lesions	$33.70	$42.66	−$8.96
Total Cost	$601.86	$286.59	$315.28
Number of biopsies	0.271	1	−0.729
Number of excisions	0.095	0.12	−0.025
Number of missed melanoma diagnoses	0.025	0	0.025

Abbreviation: PLA, pigmented lesion assay.

In the specialist care setting, the total cost per person was approximately $601.86 for standard care plus pigmented lesion assay, and $286.59 for standard care alone. Compared to standard care alone, pigmented lesion assay reduced some costs related to physician visits (−$54.43 resulting from fewer follow-up visits needed after biopsy and excision). Pigmented lesion assay also reduced some costs related to avoided biopsy and histopathology (−$66.33), as well as avoided excision (−$8.96). However, these savings were relatively small compared to the cost of the pigmented lesion assay itself ($445.00). Therefore, adding pigmented lesion assay to the current diagnostic pathway for specialists would increase the per-person cost by $315.28.

Budget Impact

The reference case budget impact results are presented in Table 12. If pigmented lesion assay became publicly funded in the primary care setting, we estimated that it would increase the total net cost, ranging from $0.14 million in year 1 to $1.24 million in year 5, for a total net cost of $3.44 million over the first 5 years. Not factoring in cost savings from other areas, pigmented lesion assay alone would cost a total of $5.91 million over the first 5 years. If pigmented lesion assay became publicly funded in the specialist care setting, we estimated that it would increase the total net cost, ranging from $0.10 million in year 1 to $0.92 million in year 5, for a total net cost of $2.56 million over the first 5 years. Not factoring in cost savings from other areas, pigmented lesion assay alone would cost a total of $3.61 million over the first 5 years.

Table 12:

Budget Impact of Pigmented Lesion Assay in Primary Care and Specialist Care Settings

	Budget Impact, $ Millions
	Year 1	Year 2	Year 3	Year 4	Year 5	Total
Total Cost in Primary Care	0.14	0.41	0.69	0.96	1.24	3.44
Cost of physician visits	−0.06	−0.18	−0.30	−0.42	−0.54	−1.49
Cost of PLA	0.24	0.71	1.18	1.65	2.13	5.91
Cost of biopsy and histopathology	−0.04	−0.11	−0.18	−0.25	−0.32	−0.88
Cost of excision	0.00	−0.01	−0.02	−0.02	−0.03	−0.09
Total Cost in Specialist Care	0.10	0.31	0.51	0.72	0.92	2.56
Cost of physician visits	−0.02	−0.05	−0.09	−0.12	−0.16	−0.44
Cost of PLA	0.14	0.43	0.72	1.01	1.30	3.61
Cost of biopsy and histopathology	−0.02	−0.06	−0.11	−0.15	−0.19	−0.54
Cost of excision	0.00	−0.01	−0.01	−0.02	−0.03	−0.07

Abbreviation: PLA, pigmented lesion assay.

Sensitivity Analysis

Results of the sensitivity analyses for the primary care and specialist care settings are presented as tornado diagrams in Figures 10 and 11, respectively. The budget impact was most sensitive to assumptions regarding the uptake rate and the cost of pigmented lesion assay. The budget impact was not sensitive to cost of biopsy, diagnostic accuracy of pigmented lesion assay, or prevalence of melanoma.

Figure 10: Sensitivity Analysis Results in the Primary Care Setting (Tornado Diagram).

Abbreviations: GP, general practitioner; ICD, international classification of diseases; PLA, pigmented lesion assay.

Figure 11: Sensitivity Analysis Results in the Specialist Care Setting (Tornado Diagram).

Abbreviations: ICD, international classification of diseases; PLA, pigmented lesion assay.

Discussion

The reference case results showed that publicly funding pigmented lesion assay in Ontario would increase the provincial budget, despite a small cost-offset from reduced biopsies and excisions. The budget impact is relatively small since we assumed a very slow and gradual uptake in the reference case. However, if the uptake rates were higher, the budget impact would be much greater.

Our analysis has several strengths. The analysis was based on real-world Ontario data (types of biopsies, costs, and physician specialty). We also considered the potential use of pigmented lesion assay at different places along the clinical pathway, either used by general practitioners in the primary care setting or by specialists in the specialist care setting. In the US, dermatologists are the main target user of pigmented lesion assay. However, pigmented lesion assay could be more useful in the primary care setting because general practitioners tend to be less experienced in diagnosing melanoma compared to dermatologists. In addition, the majority of patients with suspected melanoma present first to general practitioners. An easy-to-use, effective tool could potentially be valuable in helping general practitioners determine whether surgical biopsy is needed. However, it is also possible to “overuse” pigmented lesion assay since it is easier to perform compared to surgical biopsy.

Our analysis also has several limitations. We relied on data from the OHIP claims database; however, those data were collected for billing and administrative purposes, and not for research. One limitation of the administrative data is the possibility of inaccurate coding. Therefore, we conducted extensive sensitivity analyses using different OHIP codes and diagnosis codes. Another limitation is that all accuracy studies of pigmented lesion assay were conducted in dermatology clinics. Due to a lack of data, we assumed the diagnostic accuracy of pigmented lesion assay in primary care would be similar.

Lastly, since pigmented lesion assay is less invasive and easier to perform than biopsy, physicians may choose to use the test on more people, if it was available. Therefore, there is the possibility that physicians may overuse pigmented lesion assay on inappropriate patients (e.g., giving the test just to reassure patients). We addressed this in a sensitivity analysis by increasing the size of the target population.

Conclusions

We found that publicly funding pigmented lesion assay in Ontario would increase the provincial budget, even after considering potential savings from reduced specialist referrals, biopsies, histopathology assessments, and excisions. The budget impact was small because we assumed a very low uptake of pigmented lesion assay (1%, 3%, 5%, 7% and 9% for the first 5 years). If the test was made available only to general practitioners for patients who present with suspected melanoma, we estimated that there would be an additional net cost of $3.44 million to the provincial budget over the next 5 years ($5.91 million for the cost of pigmented lesion assay alone, without the cost savings). If the test was made available only to specialists (e.g., dermatologists) for patients who present with suspected melanoma, we estimated that there would be an additional cost of $2.56 million to the provincial budget over the next 5 years ($3.61 million for the cost of pigmented lesion assay alone, without the cost savings). The results of this analysis were most sensitive to assumptions regarding the uptake rate of pigmented lesion assay, unit cost of pigmented lesion assay, and size of the target population in Ontario.

Preferences and Values Evidence

Objective

The objective of this analysis was to explore the underlying values, needs, and priorities of those who have lived experience with biopsies for suspected melanoma, as well as some opinions about the hypothetical use of a non-invasive diagnostic tool to determine if a biopsy would be necessary, such as pigmented lesion assay.

Background

Exploring patient preferences and values provides a unique source of information about people's experiences of a health condition and the health technologies or interventions used to manage or treat that health condition. It includes the impact of the condition and its treatment on the person with the health condition, their family and other caregivers, and the person's personal environment. Engagement also provides insights into how a health condition is managed by the province's health system.

Information shared from lived experience can also identify gaps or limitations in published research (e.g., outcomes important to those with lived experience that are not reflected in the literature).^56–58 Additionally, lived experience can provide information and perspectives on the ethical and social values implications of health technologies or interventions.

Because the needs, preferences, priorities, and values of those with lived experience in Ontario are often inadequately explored in the published literature, we may speak directly with people who live with a given health condition, including those with experience of the technology or intervention we are exploring.

For this analysis, we examined the preferences and values of people who have experienced a biopsy for suspected melanoma and who may consider a diagnostic tool such as a pigmented lesion assay in the future. We examined these preferences and values in two ways:

• Direct engagement by Ontario Health with people with lived-experience of biopsies for suspected melanoma through phone interviews

• A review by the Canadian Agency for Drugs and Technologies in Health (CADTH) of the published qualitative evidence

Qualitative Evidence

Ontario Health collaborated with the Canadian Agency for Drugs and Technologies in Health (CADTH) to conduct this health technology assessment. CADTH conducted a review of qualitative literature on the experiences and perspectives of people with suspected skin cancers.⁵⁹ We include some perspectives from CADTH's report in the results from our direct patient engagement, below.

This review used thematic synthesis to synthesize the results of 12 included publications and described how people with suspected skin cancer and their health care providers experienced the process of diagnosis. This report found that people with suspicious moles or lesions often described experiencing delays in being diagnosed for a variety of reasons. Once diagnosed, people with skin cancer described how the diagnosis was an emotionally destabilizing and shocking experience.

Patients emphasized the importance of communication with health care providers in the treatment process and to help manage their emotional and physical needs. After treatment, people who had been diagnosed found themselves continuing to watch their bodies and coping with feelings of uncertainty and anxiety about their future. People who had been treated described how waiting for test results was particularly fraught with anxiety during the follow-up period.

Direct Patient Engagement

Methods

PARTNERSHIP PLAN

The partnership plan for this health technology assessment focused on consultation to examine the experiences of people with lived experience of biopsy for suspected melanoma and those of their families and other caregivers. We engaged people via phone interviews.

We used a qualitative interview, as this method of engagement allowed us to explore the meaning of central themes in the experiences of people with lived experience of biopsy for suspected melanoma, as well as those of their families and caregivers.⁶⁰ The sensitive nature of exploring people's experiences of a health condition and their quality of life are other factors that support our choice of an interview methodology.

PARTICIPANT OUTREACH

We used an approach called purposive sampling,^61–64 which involves actively reaching out to people with direct experience of the health condition and health technology or intervention being reviewed. We approached a variety of partner organizations and patient networks, including the Melanoma Network of Canada, to spread the word about this engagement activity and to contact people with experience with biopsies for suspected melanoma.

Inclusion Criteria

We sought to speak with people with lived experience of skin biopsies for suspected melanoma. Participants did not need to have melanoma or direct experience with pigmented lesion assay to participate.

Exclusion Criteria

We did not set exclusion criteria.

Participants

For this project, we spoke with 16 people with lived experience of biopsy for suspected melanoma and one family member. None of the participants were familiar with the pigmented lesion assay diagnostic tool. Most participants were from southern Ontario, living in both rural and urban centres.

APPROACH

At the beginning of the interview, we explained the role of our organization, the purpose of this health technology assessment, the risks of participation, and how participants' personal health information would be protected. We gave this information to participants verbally and, if requested, in a written letter of information (Appendix 5). We then obtained participants' verbal consent before starting the interview. With participants' consent, we audio-recorded and then transcribed the interviews.

Interviews lasted approximately 15 to 40 minutes. The interview was loosely structured and consisted of a series of open-ended questions. Questions were based on a list developed by the Health Technology Assessment International Interest Group on Patient and Citizen Involvement in Health Technology Assessment.⁶⁵ Questions focused on the impact of the suspected melanoma, the impact on quality of life, and their experiences with the diagnostic process. Participants were also verbally presented with information about pigmented lesion assay and were asked for their perceptions of the benefits or limitations of this diagnostic tool and the potential impact of having this technology available in Ontario. See Appendix 6 for our interview guide.

DATA EXTRACTION AND ANALYSIS

We used a modified version of a grounded-theory methodology to analyze interview transcripts. The grounded-theory approach allowed us to organize and compare information on experiences across participants. This method consists of a repetitive process of obtaining, documenting, and analyzing responses while simultaneously collecting, analyzing, and comparing information.^66,67 We used the qualitative data analysis software program NVivo⁶⁸ to identify and interpret patterns in the data. The patterns we identified allowed us to highlight the impact of the diagnostic process for people with suspected melanoma, the impact of their melanoma diagnosis, and their opinions about a non-invasive diagnostic tool to determine the necessity of a biopsy (such as pigmented lesion assay).

Results

Familiarity With Melanoma

Prior to their own diagnoses, participants reported varying levels of familiarity with melanoma, with some not even knowing what melanoma was or how dangerous it could be:

I didn't even know what melanoma was at the time, so I didn't have any fear or suspicion … Because I still—at that point—didn't think it was anything. And I wasn't aware that if it was a melanoma, that it could spread.

Many participants acknowledged that they did not know what the possible signs were for melanoma, or what a suspicious mole or lesion looked like:

I didn't know what to look for. I had heard of skin cancer at the time. I'm not sure I really understood the changing moles. I think the ABCDEs [criteria for visually identifying melanoma] are really not out there.

Some reported that they did not know of any particular risk factors for melanoma beyond increased exposure to the sun:

No [I didn't know about risk factors]. Not at all. And, of course, being the sun worshiper I was, I pretty much ignored any kind of prevention measures that I might have come across in my limited experience. [It was] definitely not something I knew about. I didn't know what to look for.

Due to this general unfamiliarity with melanoma and suspicious moles or lesions, many participants reported that they did not seek medical attention or begin their journey to diagnosis right away. There was often low urgency for reporting or acting upon any suspicious skin blemishes. Many participants reported that they simply waited for a regularly scheduled appointment with a health care provider to speak about their mild concerns:

I didn't particularly attach importance to it because at first I thought it was just acne on my back, which I have had occasionally. So I waited for my annual checkup.

Being a working mom, I didn't get it looked after right away. So I didn't have it checked out and it progressed very quickly. But when I finally did have it looked at by my general practitioner, she immediately reacted and knew that it was a problem.

Similarly to the experiences of interview participants, the qualitative literature also reported a tendency of patients to not immediately seek medical care for their suspicious moles or lesions, and included a number of explanations: “People described not visiting a health care provider for a suspicious lesion or mole for a variety of reasons. Sometimes they stated that this was because they were not aware that it could have been skin cancer. Others attributed their symptoms to the natural process of aging … additional explanations included that they were too busy to seek care, or that they thought it was not a serious enough issue with which to ‘bother’ their doctor” (CADTH report, p. 7).⁵⁹ For participants with a familial history of melanoma, there tended to be more awareness of its possible signs, which resulted in closer self-monitoring of moles and lesions.

Care Journeys

Once a suspicious mole or lesion was identified and participants resolved to seek medical attention, multiple different care pathways were reported by the participants. Health care interactions with family doctors, dermatologists, surgeons, and/or other specialists were all reported in various different combinations and sequences by participants:

I kept my head in the sand and didn't really think about it, but then I showed it to my general practitioner and a couple days later, [she] had me in a surgeon's office in the next town over—because I'm sort of in the boonies up here in Southern Georgian Bay—and he did an excision.

And finally, my wife said to me in the summer, “you better get that looked at.” And so I did. I went to the [family] doctor and the doctor [said] “you gotta go right to a dermatologist because you've got so many [moles] … I can't deal with that.”

Often, the care pathway was dictated by availability, wait times to see specialists, or the comfort level of the health care practitioner to diagnose melanoma. Some participants reported having previously visited dermatologists or surgeons and simply scheduled appointments with those specialists directly. A number of participants lamented that their initial expressions of concern to health care providers regarding suspicious lesions were dismissed or simply not given great urgency:

I don't have a lot of moles, [so] something that's turning weird and that my husband points out is something I'm concerned about. So I asked my family doctor about it in January, [and] he says “It's nothing.” And I said, “Okay”—that's exactly what you want to hear.

[The family doctor] thought it was atypical but he says “I don't really know … I think if you're concerned, I can refer you to a dermatologist.” And I said, “Okay, I'll take the referral”—and of course I didn't do anything with it, because it didn't seem like he was concerned.

I would have been a lot happier if [the doctor] had just taken [the mole] off my head in that particular session when I went and saw her, but she made me wait several months, which put me at risk for [the melanoma] spreading.

One participant felt that they were not informed of the reason for the biopsy or the potential dangers of melanoma:

I don't ever remember any dermatologist explaining why I needed to have biopsies and what it could mean for me.

This dismissal of concerns was also found in the qualitative literature: “When people who had skin cancer brought their initial concerns to their general practitioners, they described that they sometimes responded by issuing reassurances that their symptoms were non-cancerous” (CADTH report, p. 8).⁵⁹ In many cases, it was only after repeated visits to practitioners or drastic changes to the appearance of the mole or lesion that physicians made referrals and participants felt their care progressed more quickly.

Depending on the individual care pathway for their diagnosis of melanoma, some participants reported that wait times could be an issue. Participants who later tested positive for melanoma expressed regret and frustration at these delays:

I would like to think that I might have had a different outcome if I had known back in January that it was melanoma.

Seeing a specialist such as a dermatologist could be several weeks or months, causing a great deal of frustration and anxiety. Additionally, delays to see specialists could especially be true for those living in more rural areas of the province:

If I had to wait six months [for a dermatologist appointment], I would have been pestering my doctor to find me another dermatologist. I'm lucky; I live in the big city—there are loads of dermatologists here.

It was [something that concerned me]. It took me two years to get the [gynecologist] appointment. I'm in a rural area. We don't have doctors. My doctor retired and because I had to go to a specialist … it took two and a half years to get into see [the gynecologist].

Biopsies

Participants reported various experiences with biopsies as the next step in their care journey. Some participants experienced small, relatively painless biopsies:

I wasn't particularly concerned about it. The first little [biopsy]—I think it was called a punch biopsy—was no big deal. Just stuck a Band-Aid on it and I went home.

Others experienced deeper excisions, which would result in more painful recovery and possible scarring and blemishes. The location of the biopsy on the body also often had a direct impact on how patients experienced the procedure:

Biopsies are very uncomfortable. You have to take time to heal, you might have to take time off work, depending on the location. There [are] so many complications after.

When they start taking skin out of there and stitching it back together again—it is just refreshingly painful. For me, it was refreshingly painful twice, because she took off the lesion, stitched that up, and then she said, “Okay. Listen, you're going to see [the surgeon] and she's going to slice you open like a smoked sturgeon and do it all over again; only this time it's going to be bigger.”

For those who experienced larger biopsies or excisions, there could be an emotional toll to the changes resulting from the procedure. Physical changes to their body, including divots, scars, or markings could be difficult to accept:

I'm not so worried about how I appear, but … I lost a lot of tissue on my arm. I have [what look like] two hills and a valley [on my arm from the procedures]. And I've got [scars that look like] train tracks under my arms that aren't fading. They may eventually fade, but they got infected. So, you know, I'm not going to be a beauty with my sleeves off … [Sometimes] I think, “Thank God I didn't get this on my face.”

Some participants reported feeling a sense of trauma after the procedure, or empathized with others who might feel this way about scarring and physical consequences from a biopsy or excision:

I think that it's important to have the biopsy. I'm not a person that cares about what I look like [over] my health. Some [who] might be more concerned with their looks may be a little more hesitant. A friend of mine had melanoma on his nose, and they took a big piece out of his nose. I think it did bother him a bit.

Emotional Impact

Beyond the potentially distressing aspects of the biopsy, participants were consistent in reporting about the overall emotional impact of the melanoma diagnosis. Participants also mentioned that one of the worst parts of their care journey was the fear and anxiety that came with waiting for the biopsy results to come back. Participants expressed that they experienced deep fears and worries while waiting for results and the shock and dismay that occurred when melanoma was confirmed:

I was scared. I had young kids. [I] worked so hard to get myself through university and was clueless to what melanoma was.

It was very, very shocking. It was very hard. I know they said it was early stage, but I was really traumatized.

I definitely think that's inherent; I think the minute that you take something off your body that's suspicious and you're waiting for a result … I think that's a very anxious moment and I don't think it matters who you are or what stage you are, whether you're never been diagnosed or you have been diagnosed … The minute you use the term “biopsy,” I think the anxiety goes along with it.

This emotional impact is echoed in the qualitative literature: “The experience of being diagnosed with skin cancer was incredibly emotional for most people. The wait for results was difficult and people described struggling with anxiety as they waited. With the diagnosis, people felt disoriented and that their lives had been thrown into chaos and disorder … many described being in shock at the diagnosis” (CADTH report, p. 8).⁵⁹

Pigmented Lesion Assay

Pigmented lesion assay is not currently in wide clinical use in Ontario. For this reason, none of the interview participants had direct experience with this test. However, all participants had direct experience with biopsies for suspected melanoma or were family members of such an individual. Therefore, participants were able to hypothesize about how the use of pigmented lesion assay could have impacted their own care pathway had it been available to them.

Additionally, all participants interviewed who underwent skin biopsies for suspicious moles or lesions received positive results for melanoma. It is therefore likely that these participants would be closely monitoring suspicious moles and lesions in the future. This tendency to increase vigilance was expressed in the qualitative literature as well: “People expressed their fears of cancer, raising concerns about the future and its return. They described being watchful and checking for new moles or suspicious lesions and taking action by seeing their dermatologist more regularly” (CADTH report, p. 9).⁵⁹ For this reason, pigmented lesion assay could potentially be applicable in the participants' future care pathways, and they were asked to comment on its hypothetical benefits and drawbacks.

Perceived Benefits of Pigmented Lesion Assay

Interview participants responded positively to pigmented lesion assay's perceived ease-of-use:

I really think if you can find a diagnostic tool that's easy to use; that can improve the speed of diagnosis and the quality [of care] … I'd get behind that any old day.

Participants also theorized that this test could potentially be less costly than biopsies:

It looks to me as though you've got a cost saving in there too. If you're not using high price time of a surgeon in a hospital.

Hopefully that motivates the government to take a look at this tool. That the money is allocated to the people who really do have something [melanoma] and weed out the unnecessary biopsies and the stress for people.

Interviewed participants who lived in more rural areas spoke of the challenges of accessing dermatologists or surgeons to perform biopsies or excisions and thought this type of diagnostic tool could potentially help reduce barriers to accessing specialists. Participants expressed hope that the simplicity of the tool could potentially allow family doctors to perform the test, reducing the need for the initial referrals to specialists. Prescreening suspicious lesions to reduce unnecessary specialist referrals and biopsies may reduce the total number of referrals to specialists, thereby reducing wait times to access specialists, which some participants felt could be life-saving:

I'm in Ottawa, so I'm in a metropolitan area [and] have a wonderful medical support system in my area. But [I] think [in rural areas], getting to a specialist—it's a big issue; it's a barrier just getting there. That accessibility is important. The more [that] we can bring down to the walk-in clinics [and] the family doctors … [it] could be life-saving and cost-saving.

Some participants even suggested that the test could be most effective if patients could do it themselves at home:

The other thing I would refer to is the colon cancer check; where you bring the kit home and you do the process and you mail it away. Maybe this could be implemented similarly.

If it can really narrow in on [a mole] that is risky, I could hope that it would increase the vigilance and the surveillance by the doctors at the clinics. Beyond that, I could even envision something like [making this available] to high risk individuals to do on their own at home.

Participants observed that its ease-of-use may also increase the number of melanoma cases captured. For example, if its ease-of-use allows more front-line doctors to conduct the test rather than waiting for specialist referrals, and high-risk individuals who may have been avoiding biopsies are more willing to undergo this non-invasive test, there is the potential that more melanoma cases could be identified earlier:

There [are] people who are reluctant to go to the doctor at all and so [if they're already in the office, they may agree to put a patch on] … because [you might] never get them to go to a specialist—even if [you] can get them appointment with a specialist.

Early detection [is] number one—or it will get you going in the right direction—heading towards getting a diagnosis or having [the lesion] looked at.“

One family member spoke of her daughter, who was at risk of developing melanoma, but avoided biopsies for various reasons:

Now, my daughter… I push her all the time because she has the same skin I have and she has the same history of exposure to the sun. But she really hated that and … she keeps saying, “But the [biopsies have] always come back negative, mom.” So I was telling her about [pigmented lesion assay] and she said, “I think that would be great,” because she's still at a stage in her life where she doesn't want … all these scars.

Participants also perceived that pigmented lesion assay could potentially help patients and families avoid the stress and emotional burden of waiting for the results for an unnecessary biopsy. As reported previously, waiting for biopsy results can cause anxiety and a great deal of fear. Theoretically, this device could reduce the number of unnecessary biopsies required, and therefore reduce the potential emotional strain and physical trauma associated with them:

I see my [dermatologist] for full checks and there are other lesions all the time now … as I get older. Things are cropping up now and I would like to have them looked at instead of biopsied because every biopsy is … traumatic. It brings it all back; it's almost like a post traumatic stress disorder kind of situation. It's very anxiety inducing and I wish I could just have something looked at that easily instead of biopsied because everything's biopsied now and I wish it wasn't. I would love access to a technology like that.

I suppose the biopsies [are] a bit like having a tooth out: it's briefly painful and then it sort of fades into the background fairly quickly. But at the time, [it's] very high stress. It's pretty emotional when you're going through a thing like that. That's something you could really do without. I could see a huge benefit.

Concerns About the Use of Pigmented Lesion Assay

A universal concern of participants when considering the potential use of the pigmented lesion assay was its accuracy.

Any technologies … that would rule out a melanoma [need to be accurate]—because it's got to be accurate.

You're telling me [pigmented lesion assay] can't tell you it is [melanoma], but you're telling me for sure [that it can] 100% … say it isn't [melanoma]? I'm just wondering about the accuracy of it.

Participants reflected that a false negative could have a large impact on a patient's life, potentially delaying diagnosis of a cancerous mole or lesion and delaying potential life-saving treatment. One participant wanted a mechanism or algorithm for monitoring suspicious moles or lesions—even if pigmented lesion assay results determined the lesions to be of low concern:

[If] the readings come out really clear that it is not anything to be concerned about at this time, then they [should] say, “we'll see you in six months or eight months or whatever to retest again. We won't forget about this [mole]; we'll have a look at it again.” There needs to be an algorithm to monitor things of concern.

Other participants expressed concern about the potential cost of pigmented lesion assay, fearing that its availability may be capped or limited and that this could put high-risk individuals with many moles or freckles at risk of missing a diagnosis of melanoma, or forcing them to pay for additional tests. It was reflected that limiting this test would create an inequitable situation:

I'm not the kind of person that's covered in [moles] but there [are] people that are covered in [them]. The guidelines for administrating tests have to be wide enough that the high-risk people … that their needs are being met. That it's not just “okay, you only get three tests a year.”

Discussion

Direct engagement emphasized the challenges associated with obtaining a biopsy and the impact that a positive result for diagnosis of melanoma can have.

Due to the nature of recruitment, there was a selection bias in the patients and family member interviewed. All participants had a positive diagnosis of melanoma from one or more moles or lesions. Therefore, while individuals reported that they may initially have known very little about the risks of melanoma or the signs of it in their moles or lesions, participants were extremely knowledgeable at the time of the interviews.

Due to the unavailability of pigmented lesion assay in Ontario, no participants were able to speak directly to its use or its direct impact on their own care pathway. However, the lived-experience of participants allowed them to hypothesize about the use of the device and its potential future impact on screening suspicious moles or lesions to reduce the number of unnecessary biopsies performed.

Conclusions

Biopsies and excisions for suspected melanoma can cause pain, scars, and delays in diagnosis and potential treatment. Waiting for biopsy results, and the results themselves, can also cause additional emotional stress and anxiety. Participants expressed a positive response to the pigmented lesion assay diagnostic tool, emphasizing its potential to decrease unnecessary biopsies, which could lead to increased access to specialists, earlier diagnosis, and reduction in the physical and emotional burden of the biopsy process. Participants felt that the accuracy of this tool was essential to ensure minimal false negatives.

Preferences and Values Evidence Discussion

Robust results for patient preferences and values surrounding biopsies and the potential use of pigmented lesion assay and biopsies for people with suspected melanoma were collected through both qualitative literature review and direct patient engagement. Consistent thematic results were obtained through both methodologies, emphasizing the impact that a diagnosis of melanoma can have on a patient and family member as well as the challenges and fears involved with obtaining a biopsy for diagnosis.

There was a lack of information from the literature and through direct interviews about the values and preferences of those who do not seek medical care for suspected lesions, or those who obtain a negative biopsy result from a single suspicious mole.

Preferences and Values Evidence Conclusions

Biopsies and excisions for suspected melanoma can cause pain, scars, and delays in diagnosis and potential treatment. Waiting for biopsy results, and the results themselves, can also cause additional emotional stress and anxiety, as found in both published qualitative literature and through direct interviews. When interviewed, participants supported the use of the pigmented lesion assay diagnostic tool. They commented on its potential to decrease unnecessary biopsies, which could lead to increased access to specialists, earlier diagnosis, and reduction in the physical and emotional burden of the biopsy process. However, during interviews, participants felt that the accuracy of this tool was essential to derive these potential benefits and to ensure minimal false negatives.

Conclusions of the Health Technology Assessment

Evidence suggests that pigmented lesion assay has a sensitivity of 79% (95% confidence interval [CI] 58%–93%) and a specificity of 80% (95% CI 73%–85%), which correspond to a negative predictive value of 97.3% when calculated using a 12% prevalence rate. The evidence is uncertain about the effect of pigmented lesion assay when directly compared to visual inspection alone. We did not identify any evidence of the impact of pigmented lesion assay on patient health outcomes. The evidence is uncertain if pigmented lesion assay has an impact on clinical decision making.

We found one published cost-effectiveness study with potentially serious limitations. Therefore, the cost-effectiveness of the pigmented lesion assay compared with the standard care pathway is currently uncertain. Assuming a very low uptake, we estimated that the net budget impact of publicly funding the pigmented lesion assay in Ontario over the next 5 years to be about $3.44 million if the test is used only by primary care providers or about $2.56 million if it is used only by specialists.

Although interviewed participants did not have direct experience with pigmented lesion assay, they responded positively to the theoretical benefits of the test. Participants emphasized pigmented lesion assay's likely ease of use, and its hypothetical potential to increase early detection of melanoma and reduce the physical and emotional burden of unnecessary biopsies. Participants felt that the accuracy of this tool was essential to ensure minimal false negatives.

Abbreviations

CI

Confidence interval

GRADE

Grading of Recommendations Assessment, Development, and Evaluation

ICER

Incremental cost-effectiveness ratio

NICE

National Institute for Health and Care Excellence

QALY

Quality-adjusted life-year

Glossary

Budget impact analysis

A budget impact analysis estimates the financial impact of adopting a new health care intervention on the current budget (i.e., the affordability of the new intervention). It is based on predictions of how changes in the intervention mix will impact the level of health care spending for a specific population. Budget impact analyses are typically conducted for a short-term period (e.g., 5 years). The budget impact, sometimes referred to as the net budget impact, is the estimated cost difference between the current scenario (i.e., the anticipated amount of spending for a specific population without using the new intervention) and the new scenario (i.e., the anticipated amount of spending for a specific population following the introduction of the new intervention).

Cost–consequence analysis

A cost–consequence analysis is a type of economic evaluation that estimates the costs and consequences (i.e., the health outcomes) of two or more health care interventions. In this type of analysis, the costs are presented separately from the consequences.

Cost-effective

A health care intervention is considered cost-effective when it provides additional benefits, compared with relevant alternatives, at an additional cost that is acceptable to a decision-maker based on the maximum willingness-to-pay value.

Cost-effectiveness analysis

Used broadly, “cost-effectiveness analysis” may refer to an economic evaluation used to compare the benefits of two or more health care interventions with their costs. It may encompass several types of analysis (e.g., cost-effectiveness analysis, cost–utility analysis). Used more specifically, “cost-effectiveness analysis” may refer to a type of economic evaluation in which the main outcome measure is the incremental cost per natural unit of health (e.g., life-year, symptom-free day) gained.

Cost-minimization analysis

In economic evaluations, a cost-minimization analysis compares the costs of two or more health care interventions. It is used when the intervention of interest and its relevant alternative(s) are determined to be equally effective.

Cost–utility analysis

A cost–utility analysis is a type of economic evaluation used to compare the benefits of two or more health care interventions with their costs. The benefits are measured using quality-adjusted life-years, which capture both the quality and quantity of life. In a cost–utility analysis, the main outcome measure is the incremental cost per quality-adjusted life-year gained.

Decision tree

A decision tree is a type of economic model used to assess the costs and benefits of two or more alternative health care interventions. Each intervention may be associated with different outcomes, which are represented by distinct branches in the tree. Each outcome may have a different probability of occurring and may lead to different costs and benefits.

Dominant

A health care intervention is considered dominant when it is more effective and less costly than its comparator(s).

Health state

A health state is a particular status of health (e.g., sick, well, dead). A health state is associated with some amount of benefit and may be associated with specific costs. Benefit is captured through individual or societal preferences for the time spent in each health state and is expressed in quality-adjusted weights called utility values. In a Markov model, a finite number of mutually exclusive health states are used to represent discrete states of health.

Incremental cost

The incremental cost is the additional cost, typically per person, of a health care intervention versus a comparator.

Incremental cost-effectiveness ratio (ICER)

The incremental cost-effectiveness ratio (ICER) is a summary measure that indicates, for a given health care intervention, how much more a health care consumer must pay to get an additional unit of benefit relative to an alternative intervention. It is obtained by dividing the incremental cost by the incremental effectiveness. Incremental cost-effectiveness ratios are typically presented as the cost per life-year gained or the cost per quality-adjusted life-year gained.

Ministry of Health perspective

The perspective adopted in economic evaluations determines the types of costs and health benefits to include. Ontario Health develops health technology assessment reports from the perspective of the Ontario Ministry of Health. This perspective includes all costs and health benefits attributable to the Ministry of Health, such as treatment costs (e.g., drugs, administration, monitoring, hospital stays) and costs associated with managing adverse events caused by treatments. This perspective does not include out-of-pocket costs incurred by patients related to obtaining care (e.g., transportation) or loss of productivity (e.g., absenteeism).

Probabilistic sensitivity analysis (PSA)

A probabilistic sensitivity analysis (PSA) is used in economic models to explore uncertainty in several parameters simultaneously and is done using Monte Carlo simulation. Model inputs are defined as a distribution of possible values. In each iteration, model inputs are obtained by randomly sampling from each distribution, and a single estimate of cost and effectiveness is generated. This process is repeated many times (e.g., 10,000 times) to estimate the number of times (i.e., the probability) that the health care intervention of interest is cost-effective.

Quality-adjusted life-year (QALY)

The quality-adjusted life-year (QALY) is a generic health outcome measure commonly used in cost–utility analyses to reflect the quantity and quality of life-years lived. The life-years lived are adjusted for quality of life using individual or societal preferences (i.e., utility values) for being in a particular health state. One year of perfect health is represented by one quality-adjusted life-year.

Reference case

The reference case is a preferred set of methods and principles that provide the guidelines for economic evaluations. Its purpose is to standardize the approach of conducting and reporting economic evaluations, so that results can be compared across studies.

Scenario analysis

A scenario analysis is used to explore uncertainty in the results of an economic evaluation. It is done by observing the potential impact of different scenarios on the cost-effectiveness of a health care intervention. Scenario analyses include varying structural assumptions from the reference case.

Sensitivity analysis

Every economic evaluation contains some degree of uncertainty, and results can vary depending on the values taken by key parameters and the assumptions made. Sensitivity analysis allows these factors to be varied and shows the impact of these variations on the results of the evaluation. There are various types of sensitivity analysis, including deterministic, probabilistic, and scenario.

Time trade-off

In economic evaluations, time trade-off is a direct method of measuring people's preferences for various health states. In a time-trade off, respondents are asked about their preference for either (a) living with a chronic health condition for a certain amount of time, followed by death, or (b) living in optimal health but for less time than in scenario (a). That is, respondents decide how much time in good health they would be willing to “trade off” for more time spent in poorer health. Respondents are surveyed repeatedly, with the amount of time spent in optimal health varying each time until they are indifferent about their choice.

Tornado diagram

In economic evaluations, a tornado diagram is used to determine which model parameters have the greatest influence on results. Tornado diagrams present the results of multiple one-way sensitivity analyses in a single graph.

Utility

A utility is a value that represents a person's preference for various health states. Typically, utility values are anchored at 0 (death) and 1 (perfect health). In some scoring systems, a negative utility value indicates a state of health valued as being worse than death. Utility values can be aggregated over time to derive quality-adjusted life-years, a common outcome measure in economic evaluations.

Willingness-to-pay value

A willingness-to-pay value is the monetary value a health care consumer is willing to pay for added health benefits. When conducting a cost–utility analysis, the willingness-to-pay value represents the cost a consumer is willing to pay for an additional quality-adjusted life-year. If the incremental cost-effectiveness ratio is less than the willingness-to-pay value, the health care intervention of interest is considered cost-effective. If the incremental cost-effectiveness ratio is more than the willingness-to-pay value, the intervention is considered not to be cost-effective.

Appendices

Appendix 1: Literature Search Strategies

Clinical Evidence Search

Search date: September 10, 2019

Databases searched: Ovid MEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane Centre Register of Controlled Trials, CRD Health Technology Assessment Database, and NHS Economic Evaluation Database

Database: EBM Reviews - Cochrane Central Register of Controlled Trials <August 2019>, EBM Reviews -Cochrane Database of Systematic Reviews <2005 to September 5, 2019>, EBM Reviews - Health Technology Assessment <4th Quarter 2016>, EBM Reviews - NHS Economic Evaluation Database <1st Quarter 2016>, Embase <1980 to 2019 Week 36>, Ovid MEDLINE(R) ALL <1946 to September 09, 2019>

Search strategy:

• 1exp Melanoma/ (238410)
• 2(melanoma∗ or melanocarcinoma∗ or melanomalignoma∗ or n?evocarcinoma∗ or melanocytic).ti,ab,kf. (271666)
• 3((melanotic∗ adj2 freckle∗) or (lentigo adj maligna∗)).ti,ab,kf. (2685)
• 4Skin Neoplasms/ (132821)
• 5(skin adj2 (cancer∗ or neoplasm∗ or tumo?r∗ or carcinoma∗ or malignan∗ or metastatic∗ or metastas?s or oncolog∗)).ti,ab,kf. (83546)
• 6(pigmented adj2 (lesion$1 or skin or mole$1 or n?evus or n?evi)).ti,ab,kf. (11679)
• 7or/1-6 (449704)
• 8∗Gene Expression Profiling/ (64359)
• 9∗Gene Expression/ (157974)
• 10((non-invasive or noninvasive) and ((genetic or gene or genom∗) adj3 (test or tests or testing or assay∗ or analys#s))).ti,ab,kf. (3778)
• 11((non-invasive or noninvasive) and (PRAME∗ or expressed antigen∗ or (intergenic adj2 (non-coding RNA∗ or noncoding RNA∗)) or LINC00518∗)).ti,ab,kf. (30)
• 12((pigment∗ lesion∗ adj2 assay∗) or PLA or PLAs).ti,ab,kf. (27625)
• 13((adhesive adj2 (patch∗ or tape)) or ((bladeless or blade-less) adj2 biops∗)).ti,ab,kf. (3468)
• 14(dermtech∗ or genetic information retrieval∗ or EGIR∗).ti,ab,kf. (263)
• 15or/8-14 (255909)
• 167 and 15 (2713)
• 1716 use medall,coch,cctr,clhta,cleed (840)
• 18animals/ not humans/ (5446698)
• 1917 not 18 (738)
• 20(Comment or Editorial or (Letter not (Letter and Randomized Controlled Trial)) or Congress).pt. (3497110)
• 2119 not 20 (706)
• 22limit 21 to english language [Limit not valid in CDSR; records were retained] (694)
• 23exp melanoma/ (238410)
• 24(melanoma∗ or melanocarcinoma∗ or melanomalignoma∗ or n?evocarcinoma∗ or melanocytic).tw,kw. (273631)
• 25((melanotic∗ adj2 freckle∗) or (lentigo adj maligna∗)).tw,kw. (2728)
• 26skin tumor/ or exp skin cancer/ (238167)
• 27(skin adj2 (cancer∗ or neoplasm∗ or tumo?r∗ or carcinoma∗ or malignan∗ or metastatic∗ or metastas?s or oncolog∗)).tw,kw. (83065)
• 28(pigmented adj2 (lesion$1 or skin or mole$1 or n?evus or n?evi)).tw,kw. (11585)
• 29or/23-28 (513291)
• 30∗gene expression profiling/ (64359)
• 31∗gene expression/ (157974)
• 32(pigmented lesion assay or epidermal genetic information retrieval or two gene molecular assay).sh. (12)
• 33((non-invasive or noninvasive) and ((genetic or gene or genom∗) adj3 (test or tests or testing or assay∗ or analys#s))).tw,kw,dv. (3846)
• 34((non-invasive or noninvasive) and (PRAME∗ or expressed antigen∗ or (intergenic adj2 (non-coding RNA∗ or noncoding RNA∗)) or LINC00518∗)).tw,kw,dv. (31)
• 35((pigment∗ lesion∗ adj2 assay∗) or PLA or PLAs).tw,kw,dv. (27722)
• 36((adhesive adj2 (patch∗ or tape)) or ((bladeless or blade-less) adj2 biops∗)).tw,kw,dv. (3537)
• 37(dermtech∗ or genetic information retrieval∗ or EGIR∗).tw,kw,dv. (266)
• 38or/30-37 (256145)
• 3929 and 38 (3048)
• 4039 use emez (2201)
• 41(exp animal/ or nonhuman/) not exp human/ (10427447)
• 4240 not 41 (1901)
• 43Comment/ or Editorial/ or (letter.pt. not (letter.pt. and randomized controlled trial/)) or conference abstract.pt. (6983951)
• 4442 not 43 (1281)
• 45limit 44 to english language [Limit not valid in CDSR; records were retained] (1244)
• 4622 or 45 (1938)
• 4746 use medall (688)
• 4846 use emez (1244)
• 4946 use coch (0)
• 5046 use cctr (6)
• 5146 use clhta (0)
• 5246 use cleed (0)
• 53remove duplicates from 46 (1673)

Economic Evidence Search

Economic Literature Search

Search date: September 11, 2019

Databases searched: Ovid MEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane Centre Register of Controlled Trials, CRD Health Technology Assessment Database, and NHS Economic Evaluation Database

Database: EBM Reviews - Cochrane Central Register of Controlled Trials <August 2019>, EBM Reviews -Cochrane Database of Systematic Reviews <2005 to September 5, 2019>, EBM Reviews - Database of Abstracts of Reviews of Effects <1st Quarter 2016>, EBM Reviews - Health Technology Assessment <4th Quarter 2016>, Embase <1980 to 2019 Week 36>, Ovid MEDLINE(R) ALL <1946 to September 10, 2019>

Search strategy:

• 1exp Melanoma/ (238401)
• 2(melanoma∗ or melanocarcinoma∗ or melanomalignoma∗ or n?evocarcinoma∗ or melanocytic).ti,ab,kf. (271770)
• 3((melanotic∗ adj2 freckle∗) or (lentigo adj maligna∗)).ti,ab,kf. (2686)
• 4Skin Neoplasms/ (132800)
• 5(skin adj2 (cancer∗ or neoplasm∗ or tumo?r∗ or carcinoma∗ or malignan∗ or metastatic∗ or metastas?s or oncolog∗)).ti,ab,kf. (83567)
• 6(pigmented adj2 (lesion$1 or skin or mole$1 or n?evus or n?evi)).ti,ab,kf. (11680)
• 7or/1-6 (449828)
• 8∗Gene Expression Profiling/ (64369)
• 9∗Gene Expression/ (157983)
• 10((non-invasive or noninvasive) and ((genetic or gene or genom∗) adj3 (test or tests or testing or assay∗ or analys#s))).ti,ab,kf. (3779)
• 11((non-invasive or noninvasive) and (PRAME∗ or expressed antigen∗ or (intergenic adj2 (non-coding RNA∗ or noncoding RNA∗)) or LINC00518∗)).ti,ab,kf. (30)
• 12((pigment∗ lesion∗ adj2 assay∗) or PLA or PLAs).ti,ab,kf. (27649)
• 13((adhesive adj2 (patch∗ or tape)) or ((bladeless or blade-less) adj2 biops∗)).ti,ab,kf. (3468)
• 14(dermtech∗ or genetic information retrieval∗ or EGIR∗).ti,ab,kf. (263)
• 15or/8-14 (255952)
• 167 and 15 (2713)
• 17economics/ (253559)
• 18economics, medical/ or economics, pharmaceutical/ or exp economics, hospital/ or economics, nursing/ or economics, dental/ (835640)
• 19economics.fs. (410582)
• 20(econom∗ or price or prices or pricing or priced or discount∗ or expenditure∗ or budget∗ or pharmacoeconomic∗ or pharmaco-economic∗).ti,ab,kf. (894076)
• 21exp “costs and cost analysis”/ (568237)
• 22(cost or costs or costing or costly).ti. (255972)
• 23cost effective∗.ti,ab,kf. (322253)
• 24(cost∗ adj2 (util∗ or efficacy∗ or benefit∗ or minimi∗ or analy∗ or saving∗ or estimate∗ or allocation or control or sharing or instrument∗ or technolog∗)).ab,kf. (216280)
• 25models, economic/ (11535)
• 26markov chains/ or monte carlo method/ (79403)
• 27(decision adj1 (tree∗ or analy∗ or model∗)).ti,ab,kf. (42646)
• 28(markov or markow or monte carlo).ti,ab,kf. (130345)
• 29quality-adjusted life years/ (37123)
• 30(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).ti,ab,kf. (74803)
• 31((adjusted adj1 (quality or life)) or (willing∗ adj2 pay) or sensitivity analys∗s).ti,ab,kf. (121819)
• 32or/17-31 (2552985)
• 3316 and 32 (34)
• 3433 use medall,coch,cctr,clhta (14)
• 35[16 use cleed] (0)
• 3634 or 35 (14)
• 37animals/ not humans/ (5447477)
• 3836 not 37 (13)
• 39(Comment or Editorial or (Letter not (Letter and Randomized Controlled Trial)) or Congress).pt. (3498610)
• 4038 not 39 (13)
• 41limit 40 to english language [Limit not valid in CDSR,DARE; records were retained] (13)
• 42exp melanoma/ (238401)
• 43(melanoma∗ or melanocarcinoma∗ or melanomalignoma∗ or n?evocarcinoma∗ or melanocytic).tw,kw. (273768)
• 44((melanotic∗ adj2 freckle∗) or (lentigo adj maligna∗)).tw,kw. (2729)
• 45skin tumor/ or exp skin cancer/ (238145)
• 46(skin adj2 (cancer∗ or neoplasm∗ or tumo?r∗ or carcinoma∗ or malignan∗ or metastatic∗ or metastas?s or oncolog∗)).tw,kw. (83150)
• 47(pigmented adj2 (lesion$1 or skin or mole$1 or n?evus or n?evi)).tw,kw. (11597)
• 48or/42-47 (513492)
• 49∗gene expression profiling/ (64369)
• 50∗gene expression/ (157983)
• 51(pigmented lesion assay or epidermal genetic information retrieval or two gene molecular assay).sh. (12)
• 52((non-invasive or noninvasive) and ((genetic or gene or genom∗) adj3 (test or tests or testing or assay∗ or analys#s))).tw,kw,dv. (3847)
• 53((non-invasive or noninvasive) and (PRAME∗ or expressed antigen∗ or (intergenic adj2 (non-coding RNA∗ or noncoding RNA∗)) or LINC00518∗)).tw,kw,dv. (31)
• 54((pigment∗ lesion∗ adj2 assay∗) or PLA or PLAs).tw,kw,dv. (27745)
• 55((adhesive adj2 (patch∗ or tape)) or ((bladeless or blade-less) adj2 biops∗)).tw,kw,dv. (3540)
• 56(dermtech∗ or genetic information retrieval∗ or EGIR∗).tw,kw,dv. (266)
• 57or/49-56 (256190)
• 5848 and 57 (3048)
• 59Economics/ (253559)
• 60Health Economics/ or Pharmacoeconomics/ or Drug Cost/ or Drug Formulary/ (128286)
• 61Economic Aspect/ or exp Economic Evaluation/ (457177)
• 62(econom∗ or price or prices or pricing or priced or discount∗ or expenditure∗ or budget∗ or pharmacoeconomic∗ or pharmaco-economic∗).tw,kw. (906418)
• 63exp “Cost”/ (568237)
• 64(cost or costs or costing or costly).ti. (255972)
• 65cost effective∗.tw,kw. (330637)
• 66(cost∗ adj2 (util∗ or efficac∗ or benefit∗ or minimi∗ or analy∗ or saving∗ or estimate∗ or allocation or control or sharing or instrument∗ or technolog∗)).ab,kw. (228139)
• 67Monte Carlo Method/ (64538)
• 68(decision adj1 (tree∗ or analy∗ or model∗)).tw,kw. (43630)
• 69(markov or markow or monte carlo).tw,kw. (132240)
• 70Quality-Adjusted Life Years/ (37123)
• 71(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw,kw. (75846)
• 72((adjusted adj1 (quality or life)) or (willing∗ adj2 pay) or sensitivity analys∗s).tw,kw. (135379)
• 73or/59-72 (2190318)
• 7458 and 73 (41)
• 7574 use emez (20)
• 76(exp animal/ or nonhuman/) not exp human/ (10428228)
• 7775 not 76 (19)
• 78Comment/ or Editorial/ or (letter.pt. not (letter.pt. and randomized controlled trial/)) or conference abstract.pt. (6985448)
• 7977 not 78 (11)
• 8041 or 79 (24)
• 8180 use medall (12)
• 8280 use cctr (1)
• 8380 use coch (0)
• 8480 use clhta (0)
• 85[80 use cleed] (0)
• 8680 use emez (11)
• 87remove duplicates from 80 (16)

Grey Literature Search

Search date: September 16–19, 2019. Updated December 18, 2019

Websites searched: HTA Database Canadian Repository, Alberta Health Evidence Reviews, BC Health Technology Assessments, Canadian Agency for Drugs and Technologies in Health (CADTH), Institut national d'excellence en santé et en services sociaux (INESSS), Institute of Health Economics (IHE), McGill University Health Centre Health Technology Assessment Unit, Centre Hospitalier de l'Universite de Quebec-Universite Laval, Health Technology Assessment Database, Epistemonikos, National Institute for Health and Care Excellence (NICE), Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Centers, Australian Government Medical Services Advisory Committee, Council of Australian Governments Health Technologies, Centers for Medicare & Medicaid Services Technology Assessments, Institute for Clinical and Economic Review, Ireland Health Information and Quality Authority Health Technology Assessments, Washington State Health Care Authority Health Technology Reviews, Health Technology Wales, Oregon Health Authority Health Evidence Review Commission, Veterans Affairs Health Services Research and Development, Italian National Agency for Regional Health Services (AGENAS), Australian Safety and Efficacy Register of New Interventional Procedures -Surgical (ASERNIP-S), Belgian Health Care Knowledge Centre, Ludwig Boltzmann Institute for Health Technology Assessment, Ministry of Health Malaysia Health Technology Assessment Section, Swedish Agency for Health Technology Assessment and Assessment of Social Services, ClinicalTrials.gov, PROSPERO, EUnetHTA, Tuft's Cost-Effectiveness Analysis Registry

Keywords used: pigmented lesion or pigment lesion or dermtech or ([melanoma or skin cancer or skin neoplasm] and [test or assay or gene expression or genetic or genomic or adhesive patch or adhesive tape]) or PRAME or LINC00518

Results from clinical search (included in PRISMA): 1

Results from economic search (included in PRISMA): 0

Ongoing systematic reviews (PROSPERO/EUnetHTA): 1

Ongoing clinical trials (ClinicalTrials.gov): 0

Appendix 2: Critical Appraisal of Clinical Evidence

Table A1:

Risk of Bias^a Among Diagnostic Accuracy Studies (QUADAS-2 Tool)

Author, Year	Risk of Bias				Applicability Concerns
	Patient Selection	Index Test	Reference Standard	Flow and Timing	Patient Selection	Index Test	Reference Standard
Ferris et al, 201742	High riskb	Low risk	Low risk	Low risk	Low risk	Unclearc	Low concern
Ferris et al, 201843	Uncleard	Low risk	Low risk	High riske	Uncleard	Low concern	Low concern
Gerami et al, 201721	Low riskf	Low risk	Low risk	Low risk	Low concern	Low concern	Low concern
Hornberger et al, 201845	High riskg	Low risk	Low risk	Low risk	High concerng	Low concern	Low concern

Abbreviation: QUADAS, Quality Assessment of Diagnostic Accuracy Studies.

^aPossible risk of bias levels: low, high, unclear.

^bIt is unclear how patient charts were selected and if there was bias introduced in patient selection.

^cThere is unclear applicability of using only images for assessments since there is an expectation in health care that patients be assessed in person, in addition to the understanding that in-person assessments are more accurate compared to image assessments alone.¹⁷

^dThere is unclear patient selection.

^eNot everyone received the reference standard (biopsy and histopathology), and patients who had negative pigmented lesion assay results were presumed to be true negatives at the time of follow-up if no further testing had been done. At the very least, we would like to have seen at least 50% of the negative pigmented lesion assay results confirmed by the reference standard, which would be in better alignment with the methodology of other systematic reviews on the subject.¹⁷

^fWe focused only on the consecutively enrolled patient group because the study design for this group had a lower risk of introducing bias than the archival chart group. Furthermore, we observed that the consecutively enrolled group had a prevalence of melanoma of 12%, whereas the prevalence of melanoma in the archival chart group was 32%. Given that the Gerami et al study used a prevalence of 7% for the negative predictive value calculations, we suspected that the archival chart group was likely not representative of the actual population.

^gThere are no details provided around the study design and patient selection.

Table A2:

Risk of Bias^a With Risk of Bias Assessment Tool for Non-randomized Studies (RoBANS)

Author, Year	Selection of Participants	Confounding Variables	Measurement of Exposure	Blinding of Outcome Assessments	Incomplete Outcome Data
Brouha, 202041	Low	Highc	Low	Highd	Low
Ferris, 201944	Highb	Highc	Low	Highd	Low
Varedi, 201946	Highe	Highf	Highg	Low	Low

^aPossible risk of bias levels: low, high, unclear.

^bParticipant selection was based on a retrospective review of medical charts, which is dependent on the completion of charts. We cannot be certain that all individuals exposed to the intervention were followed up appropriately and included in the study.

^cPotential confounding variables cannot be accounted for since this is a single-arm study design and therefore did not have a control group.

^dThere was no blinding, which may have introduced some inadvertent bias against conducting biopsy for individuals who received a negative test result. Additionally, the publication did not report what the minimum appropriate follow-up care was required for inclusion (if any). It was assumed that patients with an outcome of “no biopsy” was appropriate; however, it is possible that these patients did not have the same level of follow-up care among the various providers (e.g., in-person visit every 3–6 months).

^e,fThe study acknowledged the following limitations: (1) clinician surveys may be subject to respondent bias; (2) results may not be representative of all clinician experiences; (3) they were not able to adjust for any confounding variables.

^gMeasurements were collected through self-reported methods via survey.

Table A3:

GRADE Evidence Profile for Pigmented Lesion Assay

Number of Studies (Design)	Risk of Biasa	Inconsistency	Indirectness	Imprecision	Publication Bias	Upgrade Considerations	Quality
Test Accuracyb
4 (observational)	Serious limitations (−1)	Serious limitations (−1)c	No serious limitations	No serious limitations	Serious limitations (−1)d	None	⊕ Very low
1 (observational)e	No serious limitations	No serious limitations	No serious limitations	Serious limitations (−1)f	Serious limitations (−1)d	None	⊕⊕ Low
Comparative Diagnostic Accuracy
1 (observational)	Serious limitations (−1)	Serious limitations (−1)c	Serious limitations (−1)g	No serious limitations	Serious limitations (−1)d	None	⊕ Very low
Clinical Decision Making
3 (observational)	Very serious limitations (−2)	No serious limitations	No serious limitations	No serious limitations	Serious limitations (−1)h	None	⊕ Very low

Abbreviations: GRADE, Grading of Recommendations Assessment, Development, and Evaluation.

^aRisk of bias assessment details are available in Appendix 2, Tables A1 and A2.

^bA detailed examination of sensitivity and specificity are presented in Figure 3.

^cThe range of sensitivity and specificity among the different studies would likely lead to different clinical management decisions.

^dThere are only a few studies with small sample sizes, and all of them are either directly industry sponsored or include authors who have conflicts of interest as employees or consultants for the manufacturer.

^eWe considered one study with low risk of bias. Our evaluation is focused on a subgroup of patients who were consecutively enrolled in the study presented in the supplemental information in the original publication.

^fConfidence interval is considered sufficiently wide that it would lead to different clinical decision making at the low and high end.

^gVisual assessment was conducted only through the examination of dermoscopy images; however, assessment in person is more accurate than image assessment alone (relative diagnostic odds ratio 4.6, 95% confidence interval 2.4 to 9.0).¹⁷

^hThere are only three studies, and two of them were industry sponsored with authors that have conflicts of interest as employees or consultants for the manufacturer.

Appendix 3: Selected Excluded Studies—Clinical Evidence

For transparency, we provide a list of studies that readers might have expected to see but that did not meet the inclusion criteria, along with the primary reason for exclusion.

Citation	Primary Reason for Exclusion
Childs MV. Noninvasive gene expression testing in amelanotic melanoma. JAMA Dermatol. 2018;154(2):223–4.	Wrong study design (case report)
Ferris LK, Moy RL, Gerami P, Sligh JE, Jansen B, Yao Z, et al. Noninvasive analysis of high-risk driver mutations and gene expression profiles in primary cutaneous melanoma. J Invest Dermatol. 2019;139(5):1127–34.	Wrong outcomes (analytical validity and gene discovery/proof of concept)
Gerami P, Alsobrook JP 2nd, Palmer TJ, Robin HS. Development of a novel noninvasive adhesive patch test for the evaluation of pigmented lesions of the skin. J Am Acad Dermatol. 2014;71(2):237–44.	Wrong outcomes (analytical validity and gene discovery/proof of concept)
Jansen B, Hansen D, Moy R, Hanhan M, Yao Z. Gene expression analysis differentiates melanomas from Spitz nevin. J Drugs Dermatol. 2018;17(5):574–6.	Wrong outcomes (analytical validity and gene discovery/proof of concept)
Rivers JK, Rigel DS. Ruling out melanoma: a practical guide to improving performance through non-invasive gene expression testing. Skin Ther Newslett. 2019;14(1):4–6.	Wrong study design (summary of clinical management and secondary research)
Rivers JK, Copley MR, Svoboda R, Rigel DS. Non-invasive gene expression testing to rule out melanoma. Skin Ther Newslett. 2018;23(5):1–4.	Wrong study design (summary of clinical management and secondary research)
Siegel DM, Hornberger J. Further consideration of the pigmented lesion assay-reply. JAMA Dermatol. 2019;155(3):393–4.	Wrong study design (letter/response)
Wachsman W, Morhenn V, Palmer T, Walls L, Hata T, Zalla J, et al. Noninvasive genomic detection of melanoma. Br J Dermatol. 2011;164(4):797–806.	Wrong outcomes (analytical validity and gene discovery/proof of concept)
Yao Z, Allen T, Oakley M, Samons C, Garrison D, Jansen B. Analytical characteristics of a noninvasive gene expression assay for pigmented skin lesions. Assay Drug Dev Technol. 2016;14(6):355–63.	Wrong outcomes (analytical validity and gene discovery/proof of concept)
Yao Z, Moy R, Allen T, Jansen B. An adhesive patch-based skin biopsy device for molecular diagnostics and skin microbiome studies. J Drugs Dermatol. 2017;16(10):979–86.	Wrong outcomes (analytical validity and gene discovery/proof of concept)

Appendix 4: Results of Applicability and Limitation Checklists for Studies Included in the Economic Literature Review

Table A4:

Assessment of the Applicability of Studies Evaluating the Cost-Effectiveness of Pigmented Lesion Assay

Author, Year, Country	Is the study population similar to the question?	Are the interventions similar to the question?	Is the health care system studied sufficiently similar to Ontario?	Were the perspectives clearly stated? If yes, what were they?	Are all direct effects included? Are all other effects included where they are material?	Are all future costs and outcomes discounted? If yes, at what rate?	Is the value of health effects expressed in terms of quality-adjusted life-years?	Are costs and outcomes from other sectors fully and appropriately measured and valued?	Overall Judgmenta
Hornberger et al., 2018, USA45	Yes	Yes	No (biopsy practices may be different between US and Canada; cost of biopsy used in the study is much higher than Canada)	Yes, US health care payer	Yes	Unclear (time horizon and discount rate not reported)	Yes	Yes (included Partially both direct applicable medical costs and indirect costs)	Partially applicable

Abbreviation: US, United States.

Note: Response options for all items were “yes,” “partially,” “no,” “unclear,” and “NA” (not applicable).

^aOverall judgment can be “directly applicable,” “partially applicable,” or “not applicable.”

Table A5:

Assessment of the Limitations of Studies Evaluating the Cost-Effectiveness of Pigmented Lesion Assay

Author, Year, Country	Does the model structure adequately reflect the nature of the health condition under evaluation?	Is the time horizon sufficiently long to reflect all important differences in costs and outcomes?	Are all important and relevant health outcomes included?	Are the clinical inputsa obtained from the best available sources?	Do the clinical inputsa match the estimates contained in the clinical sources?	Are all important and relevant (direct) costs included in the analysis?	Are the estimates of resource use obtained from the best available sources?	Are the unit costs of resources obtained from the best available sources?	Is an appropriate incremental analysis presented, or can it be calculated from the reported data?	Are all important and uncertain parameters subjected to appropriate sensitivity analysis?	Is there a potential conflict of interest?	Overall Judgmentb
Hornberger et al., 2018, USA45	Unclear (long-term model not reported clearly)	Unclear (time horizon not reported)	Yes	No (e.g., diagnostic accuracies of PLA and standard care)	No	Yes	Unclear	Unclear (e.g., cost of biopsy and excision not reported)	Yes	No (probabilistic sensitivity analysis results not reported; 1-way sensitivity analysis not conducted on all key parameters)	Yes, study sponsored by manufacturer	Potentially serious limitations

Abbreviations: PLA, pigmented lesion assay.

Note: Response options for all items were “yes,” “partially,” “no,” “unclear,” and “NA” (not applicable).

^aClinical inputs include relative treatment effects, natural history, and utilities.

^bOverall judgment may be “minor limitations,” “potentially serious limitations,” or “very serious limitations.”

Appendix 5: Letter of Information¹

LETTER OF INFORMATION

Health Quality Ontario is conducting a review of Pigmented Lesion Assay (PLA), a diagnostic tool to help determine if a skin biopsy is needed to assess for potential melanoma. The purpose is to understand whether this test should be more broadly funded in Ontario.

An important part of this review involves speaking to patients who have experience with skin biopsies and who may be at risk for melanoma. Our goal is always to make sure the lived-experience of individuals and families are considered in the funding recommendations for this test.

WHAT DO YOU NEED FROM PARTICIPANTS?

• ✓15-25 minutes of time for a phone or in-person interview to hear about their experiences
• ✓Permission to audio- (not video-) record the interview

WHAT PARTICIPATION INVOLVES

If a participant agrees to share their experiences, they will be asked to have an interview with Health Quality Ontario staff. The interview will likely last 15–25 minutes. It will be held in a private location or over the telephone. With consent, the interview will be audio-recorded. The interviewer will ask questions about perspectives of skin biopsies, decision-making and more general thoughts about the potential use of Pigmented Lesion Assays in Ontario.

Participation is voluntary. Those who volunteer may decide not to participate, refuse to answer any questions or withdraw before the interview. Withdrawal will in no way affect the care received.

CONFIDENTIALITY

All information collected for the review will be kept confidential and privacy will be protected except as required by law. The results of this review will be published, however no identifying information will be released or published. Any records containing information from the interview will be stored securely.

RISKS TO PARTICIPATION:

There are no known physical risks to participating. Some participants may experience discomfort or anxiety after speaking about their lived experience. If this is the case, participants can speak to our staff.

If you have any questions, please contact Health Quality Ontario staff:

Appendix 6: Interview Guide²

Interview for Pigmented Lesion Assay (PLA) - DermTech HTA

Intro

Explain HQO purpose, HTA process, and purpose of interview

Lived-Experience & Therapies

History of lesion and suspicion of skin cancer

General impact and monitoring of lesion

Biopsy

Information about biopsy; reasons, procedure itself

Decision-making in choosing to do a biopsy? Would location of biopsy have been a deciding factor? Cost? Access? Etc.

If a biopsy was obtained, what the procedure was like. What were the results?

Barriers to accessing biopsy?

If results negative, thoughts on using PLA to avoid biopsy & its impact?

Overall thoughts on the use of PLA and its potential impact?

Author contributions

This report was developed by a multidisciplinary team from Ontario Health and the Canadian Agency for Drugs and Technologies in Health (CADTH). From Ontario Health, the clinical epidemiologist was Stacey Vandersluis, the health economist was Chunmei Li, the patient and public partnership analyst was David Wells, and the medical librarian was Caroline Higgins.

About Us

Ontario Health is an agency of the Government of Ontario. Our mandate is to connect and coordinate our province's health care system in ways that have not been done before to help ensure that Ontarians receive the best possible care. We work to support better health outcomes, patient experiences, provider experiences and value for money spent.

For more information about Ontario Health, visit ontariohealth.ca.