High‐frequency ultrasound for diagnosing skin cancer in adults

Abstract

Background

Early, accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and squamous cell carcinoma (SCC) are high‐risk skin cancers with the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised, with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Ultrasound is a non‐invasive imaging technique that relies on the measurement of sound wave reflections from the tissues of the body. At lower frequencies, the deeper structures of the body such as the internal organs can be visualised, while high‐frequency ultrasound (HFUS) with transducer frequencies of 20 MHz or more has a much lower depth of tissue penetration but produces a higher resolution image of tissues and structures closer to the skin surface. Used in conjunction with clinical and/or dermoscopic examination of suspected skin cancer, HFUS may offer additional diagnostic information compared to other technologies.

Objectives

To assess the diagnostic accuracy of HFUS to assist in the diagnosis of a) cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, b) cutaneous squamous cell carcinoma (cSCC), and c) basal cell carcinoma (BCC) in adults.

Search methods

We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists as well as published systematic review articles.

Selection criteria

Studies evaluating HFUS (20 MHz or more) in adults with lesions suspicious for melanoma, cSCC or BCC versus a reference standard of histological confirmation or clinical follow‐up.

Data collection and analysis

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS‐2). Due to scarcity of data and the poor quality of studies, we did not undertake a meta‐analysis for this review. For illustrative purposes, we plot estimates of sensitivity and specificity on coupled forest plots.

Main results

We included six studies, providing 29 datasets: 20 for diagnosis of melanoma (1125 lesions and 242 melanomas) and 9 for diagnosis of BCC (993 lesions and 119 BCCs). We did not identify any data relating to the diagnosis of cSCC.

Studies were generally poorly reported, limiting judgements of methodological quality. Half the studies did not set out to establish test accuracy, and all should be considered preliminary evaluations of the potential usefulness of HFUS. There were particularly high concerns for applicability of findings due to selective study populations and data‐driven thresholds for test positivity. Studies reporting qualitative assessments of HFUS images excluded up to 22% of lesions (including some melanomas) due to lack of visualisation in the test.

Derived sensitivities for qualitative HFUS characteristics were at least 83% (95% CI 75% to 90%) for the detection of melanoma; the combination of three features (lesions appearing hypoechoic, homogenous and well defined) demonstrating 100% sensitivity in two studies (lower limits of the 95% CIs were 94% and 82%), with variable corresponding specificities of 33% (95% CI 20% to 48%) and 73% (95% CI 57% to 85%), respectively. Quantitative measurement of HFUS outputs in two studies enabled decision thresholds to be set to achieve 100% sensitivity; specificities were 93% (95% CI 77% to 99%) and 65% (95% CI 51% to 76%). It was not possible to make summary statements regarding HFUS accuracy for the diagnosis of BCC due to highly variable sensitivities and specificities.

Authors' conclusions

Insufficient data are available on the potential value of HFUS in the diagnosis of melanoma or BCC. Given the between‐study heterogeneity, unclear to low methodological quality and limited volume of evidence, we cannot draw any implications for practice. The main value of the preliminary studies included may be in providing guidance on the possible components of new diagnostic rules for diagnosis of melanoma or BCC using HFUS that will require future evaluation. A prospective evaluation of HFUS added to visual inspection and dermoscopy alone in a standard healthcare setting, with a clearly defined and representative population of participants, would be required for a full and proper evaluation of accuracy.

Plain language summary

How accurate is high‐frequency ultrasound for diagnosing skin cancer in adults?

Why is improving the diagnosis of skin cancer important?

There are several types of skin cancer. Melanoma is one of the most dangerous forms, so it is important to detect it early and remove it as soon as possible. Failure to recognise melanoma for what it is (known as a false negative test result) can delay treatment, risking the spread of melanoma to other organs in the body and possibly premature death. Other skin cancers, like cutaneous squamous cell carcinoma and basal cell carcinoma, are more localised. However, cutaneous squamous cell carcinoma can spread to other parts of the body, and basal cell carcinoma can cause disfigurement if left untreated. Diagnosing a harmless lesion (a mole or area of skin with an unusual appearance in comparison with the surrounding skin) as skin cancer (a false positive result) may result in unnecessary surgery and other tests that can cause stress and anxiety to the patient. Mistaking one skin cancer for another can lead to the wrong treatment or delays in effective treatment. Thus, the correct diagnosis is important.

What is the aim of the review?

We wanted to find out whether high‐frequency ultrasound can help doctors diagnose skin cancer. We found six studies to try and answer this question. Five studies investigated the diagnosis of melanoma and three, basal cell carcinoma.

What was studied in the review?

A number of tools allow skin cancer specialists to examine the skin in more detail than by the naked eye alone. Most skin cancer specialists currently use a dermatoscope, which magnifies the skin lesion using a natural light. Ultrasound is another non‐invasive technique that measures sound wave reflections from body tissues. High‐frequency ultrasound can produce a good‐quality image of structures closer to the skin surface. When used alongside a doctor's examination and dermoscopy, high‐frequency ultrasound may help doctors make a more accurate diagnosis.

What are the main results of the review?

The review included six studies: five with 1125 skin lesions suspected of being melanoma, and three with 993 lesions suspected of being basal cell carcinoma. We did not find any studies on the diagnosis of cutaneous squamous cell carcinoma.

The included studies were small and too different from each other to allow reliable estimates of accuracy to be made for identifying melanoma or basal cell carcinoma. Half were not actually designed to establish test accuracy and all can be considered preliminary experiments on the potential value of high‐frequency ultrasound. The main value of the studies may be in helping researchers to identify the best ways of interpreting high‐frequency ultrasound for the diagnosis of melanoma or basal cell carcinoma for evaluation in future research studies.

How reliable are the results?

Study results are not very reliable when considered collectively. The small number and variability between studies reduces reliability, while all had important limitations. In particular, those taking part in the studies and the way in which the tests were used may not reflect real life situations. In all studies the final diagnosis was confirmed by biopsy. This is likely to have been a reliable method for deciding whether patients really had skin cancer*.

Who do the results of this review apply to?

Studies all took place in Europe, and only one reported participants' average age (55.3 years). The percentage of people with a final diagnosis of melanoma ranged from 14% to 58%, while 8% to 49% had basal cell carcinoma. It was not possible to tell whether doctors suspected skin cancer based on clinical examination alone or both clinical and dermoscopic examination.

What are the implications of this review?

At present, there is not enough good research to draw a conclusion on using high‐frequency ultrasound for diagnosing skin cancers. The results of this review suggest that high‐frequency ultrasound has potential to separate melanoma or basal cell carcinoma from some harmless types of lesions, but it is still unclear whether it can adequately distinguish these skin cancers from the full range of skin conditions that patients show their doctors in everyday practice. There is a need for more studies investigating high‐frequency ultrasound alongside dermoscopy or other microscopic techniques (such as reflectance confocal microscopy) in people with suspicious skin lesions.

How up‐to‐date is this review?

The review authors searched for and used studies published up to August 2016.

*In these studies biopsy was the reference standard (means of establishing the final diagnosis).

Summary of findings

Summary of findings'. '.

Question	What is the diagnostic accuracy of high‐frequency ultrasound (HFUS) for diagnosing cutaneous melanoma or BCC in adults?
Participants	Adults with suspicious skin lesions
Prior testing and prevalence	Studies varied in, or did not report, the basis for participant referral for ultrasound. One implied that half of included lesions were difficult to diagnose, and two included only 3 lesion types. Prevalence of melanoma ranged from 14% to 58% (median 30%) and BCC from 8% to 49% (median 17%).
Settings	Secondary care and specialist lesion clinics
Target condition(s)	Invasive melanoma and atypical intraepidermal melanocytic variants; basal cell carcinoma
Index test	High‐frequency ultrasound (> 20 MHz) alone or in combination with Doppler ultrasound. Lesions not visualised on ultrasound were excluded by some studies.
Reference standard	Histology
Action:	If accurate, positive results of HFUS will help to appropriately select lesions for excision
Limitations
Risk of bias:	Patient selection methods unclear or at high risk of bias due to selective inclusion of lesion types. Test interpretation was blinded to reference standard, but test thresholds were clearly prespecified in only 1 study and were data driven (2/6) or not pre‐specified (3/6) in the remainder. Reference standard blinding was not described. Timing of index and reference standards was not reported. Exclusions due to test failures were not reported (3/6).
Applicability of evidence to question:	High (4 studies) or unclear (1 study) concerns about applicability due to unrepresentative participant samples with high disease prevalence. Test observers were not described (6/6 studies) and prototype or relatively novel devices used (2/6 studies). Reference standard interpretation by experienced histopathologists was not described (5/6 studies). Half the studies were not designed to investigate test accuracy.
Total number of studies	6	Total participants with test results	1263	Total number melanoma or BCC	349
Detection of melanoma
Number of studies	5	Total participants with test results	1125	Total with melanoma	242
Findings	No pooled analysis conducted due to between‐study heterogeneity and small study numbers. Derived sensitivities for investigated HFUS characteristics were at least 83% (95% CI 75% to 90%); the combination of 3 qualitative features (lesions appearing hypoechoic, homogenous and well defined) demonstrating 100% sensitivity in 2 studies, with variable specificities of 33% (95% CI 20% to 48%) and 73% (95% CI 57% to 85%). Quantitative measurement of HFUS outputs in 2 studies enabled decision thresholds to be set to achieve 100% sensitivity; resulting specificities were 93% (95% CI 77% to 99%) and 65% (95% CI 51% to 76%). Between 7 and 38 lesions were not visualised on HFUS (reported in 3 studies); including between 3 and 5 melanomas not visualised (in each of the 3 studies).
Detection of BCC
Number of studies	3	Total participants with test results	993	Total with BCC	119
Findings	Only qualitative thresholds were assessed; sensitivities and specificities were highly variable, making summary statements difficult.

HFUS: high‐frequency ultrasound; BCC: basal cell carcinoma; CI: confidence interval.

Background

This review is one in a suite of Cochrane Diagnostic Test Accuracy (DTA) Reviews on the diagnosis and staging of melanoma and keratinocyte skin cancers, conducted for the National Institute for Health Research (NIHR) Cochrane Systematic Reviews Programme. Appendix 1 shows the content and structure of the programme. Appendix 2 provides a glossary of terms used, and Appendix 3 a table of acronyms used.

Target condition being diagnosed

There are three main forms of skin cancer. Melanoma has the highest skin cancer mortality (Cancer Research UK 2017); however, the most common skin cancers in white populations arise from keratinocyte cells: basal cell carcinoma and cutaneous squamous cell carcinoma (Gordon 2013; Madan 2010). In 2003, the World Health Organization (WHO) estimated that 2 to 3 million non‐melanoma skin cancers (of which BCC and cSCC are estimated to account for around 80% and 16% of cases, respectively) and 132,000 melanoma skin cancers occur globally each year (WHO 2003).

This DTA review has three target conditions of interest: melanoma, basal cell carcinoma (BCC), and cutaneous squamous cell carcinoma (cSCC).

Melanoma

Melanoma arises from uncontrolled proliferation of melanocytes – the epidermal cells that produce pigment or melanin. Cutaneous melanoma refers to any skin lesion with malignant melanocytes present in the dermis and includes superficial spreading, nodular, acral lentiginous and lentigo maligna melanoma variants (see Figure 1). Melanoma in situ refers to malignant melanocytes that are contained within the epidermis and have not yet invaded the dermis but are at risk of progression to melanoma if left untreated. Lentigo maligna, a subtype of melanoma in situ in chronically sun‐damaged skin, denotes another form of proliferation of abnormal melanocytes. Melanoma in situ and lentigo maligna are both atypical intraepidermal melanocytic variants. All forms of melanoma in situ can progress to invasive melanoma if growth breaches the dermo‐epidermal junction during a vertical growth phase, although malignant transformation is both lower and slower for lentigo maligna than for melanoma in situ (Kasprzak 2015). Melanoma is one of the most dangerous forms of skin cancer, with the potential to metastasise to other parts of the body via the lymphatic system and blood stream. It accounts for only a small percentage of skin cancer cases but is responsible for up to 75% of skin cancer deaths (Boring 1994; Cancer Research UK 2017).

1.

Sample photographs of superficial spreading melanoma (left), BCC (centre), and cSCC (right). Copyright © 2012 Dr Rubeta Matin: reproduced with permission.

The incidence of melanoma rose to over 200,000 newly diagnosed cases worldwide in 2012 (Erdmann 2013; Ferlay 2015), with an estimated 55,000 deaths (Ferlay 2015). The highest incidence is observed in Australia, with 13,134 new cases of melanoma of the skin in 2014 (ACIM 2017), and in New Zealand with 2341 registered cases in 2010 (HPA and MelNet NZ 2014). In the USA, the predicted incidence in 2014 was 73,870, and the predicted number of deaths 9940 (Siegel 2015). The highest rates in Europe are in north‐western Europe and Scandinavia, with the highest incidence reported in Switzerland at 25.8 per 100,000 in 2012. Rates in England have tripled from 4.6 and 6.0 per 100,000 in men and women, respectively, in 1990, to 18.6 and 19.6 per 100,000 in 2012 (EUCAN 2012). Indeed, in the UK, melanoma has one of the fastest rising incidence rates of any cancer, with the biggest projected increase in incidence between 2007 and 2030 (Mistry 2011). In the decade leading up to 2013, age‐standardised incidence increased by 46%, with 14,500 new cases in 2013 and 2459 deaths in 2014 (Cancer Research UK 2017). Rates are higher in women than in men; however, the rate of incidence in men is increasing faster than in women (Arnold 2014). The rising incidence in melanoma is thought to be primarily related to an increase in recreational sun exposure and tanning bed use and an increasingly ageing population with higher lifetime recreational ultraviolet (UV) exposure, in conjunction with possible earlier detection (Belbasis 2016; Linos 2009). Belbasis 2016 provides a detailed review of putative risk factors, including eye and hair colour, skin type and density of freckles, history of melanoma, sunburn, and presence of particular lesion types.

A database in the USA of over 40,000 patients from 1998 onwards, which assisted in the development of the 8th American Joint Committee on Cancer (AJCC) Staging System, indicated a five‐year survival of 97% to 99% for stage I melanoma, which dropped to 32% to 93% in stage III disease depending on tumour thickness, the presence of ulceration and number of involved nodes (Gershenwald 2017). While these are substantial increases relative to survival in 1975 (Cho 2014), increasing incidence between 1975 and 2010 means that reported mortality rates have remained static. This observation, coupled with increasing incidence of localised disease, suggests that improvements in survival may be due to earlier detection and heightened vigilance (Cho 2014). New targeted therapies for advanced (stage IV), melanoma (e.g. BRAF inhibitors), have improved survival, and immunotherapies are evolving such that long‐term survival is being documented (Pasquali 2018; Rozeman 2017). No new data regarding the survival prospects for patients with stage IV disease were analysed for the AJCC 8 staging guidelines due to lack of contemporary data (Gershenwald 2017).

Basal cell carcinoma

BCC can arise from multiple stem cell populations, including from the follicular bulge and interfollicular epidermis (Grachtchouk 2011). BCC growth is usually localised, but it can infiltrate and damage surrounding tissue, sometimes causing considerable destruction and disfigurement, particularly when located on the face (Figure 1). The four main subtypes of BCC are superficial, nodular, morphoeic or infiltrative, and pigmented. BCCs typically present as slow‐growing, asymptomatic papules, plaques or nodules that may subsequently bleed or form ulcers that do not heal (Firnhaber 2012). People with a BCC often present to healthcare professionals with a non‐healing lesion rather than specific symptoms such as pain. Many lesions are diagnosed incidentally (Gordon 2013).

BCC most commonly occurs on sun‐exposed sites on the head and neck (McCormack 1997), and they are more common in men and in people over the age of 40. Different authors have attributed a rising incidence of BCC in younger people to increased recreational sun exposure (Bath‐Hextall 2007; Gordon 2013; Musah 2013). Other risk factors include Fitzpatrick skin phototypes I and II (Fitzpatrick 1975; Lear 1997; Maia 1995), a history of skin cancer, immunosuppression, arsenic exposure, and genetic predisposition such as in basal cell naevus (Gorlin) syndrome (Gorlin 2004; Zak‐Prelich 2004). Annual incidence is rising worldwide; Europe has experienced an average increase of 5.5% per year over the last four decades, and the USA of 2% per year, while estimates for the UK show that incidence appears to be increasing more steeply at a rate of an additional 6 per 100,000 persons per year (Lomas 2012). The rising incidence has been explained by an ageing population, changes in the distribution of known risk factors, particularly ultraviolet radiation, and improved detection due to the increased awareness amongst both practitioners and the general population (Verkouteren 2017). Hoorens 2016 points to evidence for a gradual increase in the size of BCCs over time, with delays in diagnosis ranging from 19 to 25 months.

According to National Institute for Health and Care Excellence (NICE) guidance (NICE 2010), low‐risk BCCs are nodular lesions occurring in patients older than 24 years old who are not immunosuppressed and do not have Gorlin syndrome. Furthermore, they should be located below the clavicle, should be small (diameter of less than 1 cm) with well‐defined margins, not recurrent following incomplete excision and not in awkward or highly visible locations (NICE 2010). Superficial BCCs are also typically low risk and may be amenable to medical treatments such as photodynamic therapy or topical chemotherapy (Kelleners‐Smeets 2017). Assigning BCCs a low or high risk influences the management options (Batra 2002; Randle 1996).

Advanced locally destructive or aggressive BCC can be found on 'high‐risk' anatomical areas such as the eyebrow, eyelid, nose, ear and temple (these are at higher risk of invisible spread and therefore are more at risk of being incompletely excised (Baxter 2012; Lear 2014)), and they can arise from long‐standing untreated lesions or from a recurrence of aggressive basal cell carcinoma after primary treatment (Lear 2012). Very rarely, BCC metastasises to regional and distant sites, resulting in death, especially cases of large neglected lesions in those who are immunosuppressed or those with Gorlin syndrome (McCusker 2014). Rates of metastasis are reported at 0.0028% to 0.55% (Lo 1991), with very poor survival rates. It is recognised that basosquamous carcinoma (more like a high‐risk SCC in behaviour and not considered a true BCC) is likely to have accounted for many cases of apparent metastases of BCC, hence the spuriously high reported incidence in some studies of up to 0.55%, which is not seen in clinical practice (Garcia 2009).

Squamous cell carcinoma of the skin

Primary cSCC arises from the keratinising cells of the epidermis or its appendages. People with cSCC often present with an ulcer or firm (indurated) papule, plaque or nodule (Griffin 2016), often with an adherent crust and poorly defined margins (Madan 2010). This type of carcinoma can arise in the absence of a precursor lesion or can develop from pre‐existing actinic keratosis or Bowen's disease (considered by some to be cSCC in situ); the estimated annual risk of progression being less than 1% to 20% for newly arising lesions (Alam 2001), and 5% for pre‐existing lesions (Kao 1986). It remains locally invasive for a variable length of time, but it has the potential to spread to the regional lymph nodes or via the bloodstream to distant sites, especially in immunosuppressed individuals (Lansbury 2010). High‐risk lesions are those arising on the lip or ear, recurrent cSCC, lesions arising on non‐exposed sites, scars or chronic ulcers, tumours more than 20 mm in diameter, depth of invasion greater than 4 mm and poor differentiation on pathological examination (Motley 2009). Perineural invasion of nerves at least 0.1 mm in diameter is a further documented risk factor for high‐risk cSCC (Carter 2013).

Chronic ultraviolet light exposure through recreation or occupation is strongly linked to cSCC occurrence (Alam 2001). It is particularly common in people with fair skin and in less common genetic disorders of pigmentation, such as albinism, xeroderma pigmentosum, and recessive dystrophic epidermolysis bullosa (RDEB) (Alam 2001). Other recognised risk factors include immunosuppression; chronic wounds; arsenic or radiation exposure; certain drug treatments, such as voriconazole and BRAF mutation inhibitors; and previous skin cancer history (Baldursson 1993; Chowdri 1996; Dabski 1986; Fasching 1989; Lister 1997; Maloney 1996; O'Gorman 2014). In solid organ transplant recipients, cSCC is the most common form of skin cancer; the risk of developing cSCC has been estimated at 65 to 253 times that of the general population (Hartevelt 1990; Jensen 1999; Lansbury 2010). Overall, local and metastatic recurrence of cSCC at five years is estimated at 8% and 5%, respectively. The five‐year survival rate of metastatic cSCC of the head and neck is around 60% (Moeckelmann 2018).

Treatment

For primary melanoma, the mainstay of definitive treatment is wide local surgical excision of the lesion, to remove both the tumour and any malignant cells that might have spread into the surrounding skin (Garbe 2016; Marsden 2010; NICE 2015a; SIGN 2017; Sladden 2009). Recommended lateral surgical margins vary according to tumour thickness, as described in Garbe 2016, and to stage of disease at presentation, as recommended in NICE 2015a.

Treatment options for BCC and cSCC include surgery, other destructive techniques such as cryotherapy or electrodessication, and topical chemotherapy. A Cochrane Review of 27 randomised controlled trials (RCTs) of interventions for BCC found very little good‐quality evidence for any of the interventions used (Bath‐Hextall 2007a). Complete surgical excision of primary BCC has a reported five‐year recurrence rate of less than 2% (Griffiths 2005; Walker 2006), leading to significantly fewer recurrences than treatment with radiotherapy (Bath‐Hextall 2007a). With apparently clear histopathological margins (serial vertical sections) following standard excision biopsy with 4 mm surgical peripheral margins, there is a reported recurrence rate of around 4% at five years (Drucker 2017). Mohs micrographic surgery, whereby surgeons microscopically examine horizontal sections of the tumour intraoperatively, undertaking re‐excision until the margins are tumour‐free, are options for high‐risk lesions on the face where standard wider excision margins might lead to incomplete excision or considerable functional impairment (Bath‐Hextall 2007a; Lansbury 2010; Motley 2009; Stratigos 2015). Bath‐Hextall 2007a found a single trial comparing Mohs micrographic surgery with a 3 mm surgical margin excision in BCC (Smeets 2004); the update of this study showed non‐significantly lower recurrence at 10 years with Mohs micrographic surgery (4.4% compared to 12.2% after surgical excision, P = 0.10) (van Loo 2014).

The main treatments for high‐risk BCC are standard surgical excision, Mohs micrographic surgery or radiotherapy. For low‐risk or superficial subtypes of BCC, or for small and or multiple BCCs at low‐risk sites (Marsden 2010), destructive techniques other than excisional surgery may be used (e.g. electrodessication and curettage or cryotherapy (Alam 2001; Bath‐Hextall 2007a)). Alternatively, non‐surgical (or non‐destructive) treatments may be considered (Bath‐Hextall 2007a; Drew 2017; Kim 2014), including topical chemotherapy imiquimod (Williams 2017), 5‐fluorouracil (5‐FU) (Arits 2013), ingenol mebutate (Nart 2015), and photodynamic therapy (PDT) (Roozeboom 2016). Non‐surgical treatments are most frequently used for superficial forms of BCC, with one head‐to‐head trial suggesting topical imiquimod is superior to PDT and 5‐FU (Jansen 2018). Although use of non‐surgical techniques is increasing, these do not allow histological confirmation of tumour clearance, and their use depends on accurate characterisation of the histological subtype and depth of tumour. The 2007 Cochrane Review of BCC interventions found limited evidence from very small RCTs for these approaches (Bath‐Hextall 2007a), which have only partially been addressed by subsequent studies (Bath‐Hextall 2014; Kim 2014; Roozeboom 2012). Most BCC trials have compared interventions within the same treatment class, and few have compared medical versus surgical treatments (Kim 2014).

Vismodegib, a first‐in‐class Hedgehog signalling pathway inhibitor, is now available for treating metastatic or locally advanced BCC based on the pivotal study ERIVANCE BCC (Sekulic 2012). It is licensed for use in patients where surgery or radiotherapy is inappropriate, e.g. for treating locally advanced periocular and orbital BCCs with orbital salvage of patients who otherwise would have required exenteration (Wong 2017). However, NICE has recently advised against the use of vismodegib based on cost‐effectiveness and uncertainty of evidence (NICE 2017).

A systematic review of interventions for primary cSCC found only one RCT eligible for inclusion (Lansbury 2010). Current practice therefore relies on evidence from observational studies, as reviewed in Lansbury 2013, for example. Surgical excision with predetermined margins is usually the first‐line treatment (Motley 2009; Stratigos 2015). Estimates of recurrence after Mohs micrographic surgery, surgical excision, or radiotherapy, which are likely to have been evaluated in higher‐risk populations, have shown pooled recurrence rates of 3%, 5.4% and 6.4%, respectively, with overlapping confidence intervals; the review authors advise caution when comparing results across treatments (Lansbury 2013).

Index test(s)

Ultrasound is a non‐invasive imaging technique that essentially relies on the measurement of sound wave reflections from the tissues of the body. A transducer generates a focused beam of sound pulses, measuring the reflections (or echoes) produced by structures within the tissue. The spatial location of a tissue structure that produced an echo is determined in the lateral direction (parallel to the skin surface) by the position of the sound beam (known) and in the axial (depth) direction by the return time of the echo (measured) and the speed of sound in the tissue (known to a good approximation) (Figure 2; Barcaui 2016; Kleinerman 2012). An important parameter is the range of acoustic frequencies used to form the image. While low‐frequency ultrasound visualises the deeper structures of the body, such as the internal organs, high‐frequency ultrasound (HFUS), defined here as having centre (or median) frequency of at least 20 MHz, has a much lower depth of tissue penetration but produces a higher resolution image of tissues and structures closer to the skin surface (Kleinerman 2012). Frequencies of 20 MHz to 25 MHz allow visualisation of both the dermis and epidermis while higher frequencies of 50 MHz and above visualise the epidermis only (Kleinerman 2012). Figure 3 shows an example of a currently commercially available HFUS scanner; the cost of the system can range from EUR 5500 for a Windows tablet‐based non‐real‐time system that works at 20 MHz (not shown) to around EUR 27,000 for a laptop‐based system (Figure 3) which provides real‐time images and works up to a frequency of 50 MHz (as well as 20 MHz) (Svendson 2018).

2.

The principles of B‐mode ultrasound echographic imaging of the skin.

3.

Modern laptop based DermaScan C (2D). Copyright © 2018 Cortex Technology ApS: reproduced with permission.

In B‐mode (brightness mode) ultrasound echography, the image brightness is modulated according to the amplitude of the echoes (echogenicity). This in turn is determined by the values of sound speed and mass‐density within an echo‐producing structure relative to those values in the surrounding medium, and the size, shape, orientation, and number‐density of such structures (Barcaui 2016). Please see the following examples.

• Structural proteins, such as collagen and keratin, are dense and have high sound speed, and they generate strong echoes (termed hyperechoic or echogenic) when the fibres are thick, densely packed, and oriented mostly perpendicular to the ultrasound beam (e.g. reticular dermis).

• Adipose tissue, highly cellular lesions with little collagen or keratin, and regions where the collagen bundle size is small (some lesions) and/or oriented mostly parallel to the sound beam (e.g. papillary dermis) generate weak echoes (termed hypoechoic or echo poor).

• Liquids (e.g. as in simple cysts) generate no echoes and are referred to as anechoic (Bamber 1992; Harland 1993).

Researchers have investigated the use of HFUS for diagnosing a range of skin conditions, including skin cancer, infection, and inflammatory conditions (Kleinerman 2012), with malignant lesions reportedly appearing as hypoechogenic areas surrounded by a hyperechogenic dermis. Melanomas in particular also reportedly appear homogenous and with well‐defined margins (e.g. Harland 2000). Evaluations have also been made of the ability of HFUS to quantitatively differentiate melanomas from other lesion types using entry echogenicity and attenuation (the latter being the rate of reduction in echo signal with depth). These features have been reported to be particularly useful for distinguishing melanoma from seborrhoeic keratosis, for example (Harland 2000; Rallan 2007; see Figure 4), and they are measurable even when a given lesion cannot be visualised on ultrasound.

4.

Illustrates the well defined margins, low level and homogenous internal echoes, lack of strong entry echo and lack of acoustic shadowing for melanoma (c. and d.) and contrasting image for BCC (a. and b.) (from Harland 2000, Copyright © 2000 John Wiley and Sons, reproduced with permission)

Clinical pathway

The diagnosis of melanoma can take place in primary, secondary, and tertiary care settings by both generalist and specialist healthcare providers. In the UK, people with concerns about a new or changing lesion will usually present first to their general practitioner (GP) or, less commonly, directly to a specialist in secondary care, which could include a dermatologist, plastic surgeon, or other specialist surgeon (such as an ear, nose, and throat (ENT) specialist or maxillofacial surgeon), or ophthalmologist (Figure 5). Current UK guidelines recommend that GPs should assess all suspicious pigmented lesions presenting in primary care by taking a clinical history and visually inspecting them using the revised seven‐point checklist (MacKie 1990). Clinicians should refer those with suspected melanoma or cSCC for appropriate specialist assessment within two weeks (Chao 2013; London Cancer Alliance 2013; Marsden 2010; NICE 2015a). Evidence is emerging, however, to suggest that excision of melanoma by GPs is not associated with increased risk compared with outcomes in secondary care (Murchie 2017). In the UK, low‐risk BCC are usually recommended for routine referral, with urgent referral for those in whom a delay could have a significant impact on clinical outcomes, for example due to large lesion size or critical site (NICE 2015b). Appropriately qualified generalist care providers increasingly undertake management of low‐risk BCCs in the UK, for example by excising low‐risk lesions (NICE 2010). Similar guidance is in place in Australia (CCAAC Network 2008).

5.

Current clinical pathway for people with skin lesions.

For referred lesions, the specialist clinician will also use history‐taking, inspection of the lesion (in comparison with other lesions on the skin and usually in conjunction with dermoscopic examination), and palpation of the lesion and associated regional nodal basins to inform a clinical decision. If melanoma is suspected, then urgent 2 mm excision biopsy is recommended (Lederman 1985; Lees 1991); for cSCC, clinicians may consider predetermined surgical margin excision or a diagnostic biopsy. BCC and pre‐malignant lesions potentially eligible for non‐surgical treatment may undergo a diagnostic biopsy before initiation of therapy. Equivocal melanocytic lesions for which a definitive clinical diagnosis cannot be reached may undergo surveillance to identify any lesion changes that would indicate excisional biopsy or reassurance and discharge for those that remain stable over a period of time.

Prior test(s)

The diagnosis of skin cancer is based on history‐taking and clinical examination. In the UK, this is typically undertaken at two decision points – first in the GP surgery where a decision is made to refer or not to refer, and then a second time where a dermatologist or other secondary care clinician makes a decision whether or not to biopsy or excise. Visual inspection of the skin is on an iterative basis, using both implicit pattern recognition (non‐analytical reasoning) and more explicit 'rules' based on conscious analytical reasoning (Norman 2009), the balance of which will vary according to experience and familiarity with the diagnostic question. Various attempts have been made to formalise the 'mental rules' involved in analytical pattern recognition for melanoma (Friedman 1985; Grob 1998; MacKie 1985; MacKie 1990; Sober 1979; Thomas 1998); however, visual inspection for keratinocyte skin cancers relies primarily on pattern recognition. Accuracy has been shown to vary according to the expertise of the clinician. Some authors have reported that primary care physicians miss over half of BCC (Offidani 2002), and they misdiagnose a third (Gerbert 2000). In contrast, an Australian study found that trained dermatologists were able to detect 98% of BCC, but with a specificity of only 45% (Green 1988).

A range of technologies have emerged to aid diagnosis to reduce the number of diagnostic biopsies or inappropriate surgical procedures. Dermoscopy using a handheld microscope has become the most widely used tool for clinicians to improve diagnostic accuracy of pigmented lesions, in particular for melanoma (Argenziano 1998; Argenziano 2012; Haenssle 2010; Kittler 2002), although it is less well established for the diagnosis of BCC or cSCC. Three reviews in this series have evaluated the diagnostic and comparative accuracy of visual inspection and dermoscopy (Dinnes 2018a; Dinnes 2018b, Dinnes 2018c).

Role of index test(s)

Used in conjunction with clinical or dermoscopic suspicion of malignancy, or both, in pigmented lesions, HFUS may have a potential role in patient management as an additional test to identify lesions requiring excision. The status of current medical practice and patient benefit for melanoma is particularly suited to improvement by any cost‐effective diagnostic imaging method that might be developed, since early diagnosis that leads to complete excision of primary melanoma before metastatic spread almost always results in a cure. The probability of metastases increases dramatically with increasing depth of tumour invasion of the primary melanoma (known as the Breslow thickness). This is assessed by histological examination after excision but has the potential to be assessed by imaging in vivo. One of the postulated advantages of HFUS is its ability to rule out melanoma as a potential differential diagnosis, for example by identifying pigmented seborrhoeic keratosis (a benign skin lesion).

Although the primary aim in diagnosing potentially life‐threatening conditions such as melanoma is to minimise false negative diagnoses (to avoid delay to diagnosis and even death), a test that can reduce false positive clinical diagnoses without missing true cases of disease has clear patient and resource benefits. False‐positive diagnoses not only cause unnecessary scarring from a biopsy or excision procedure, but they also increase patient anxiety whilst they await the definitive histological results and increase healthcare costs as the number needed to remove to yield one melanoma diagnosis increases. Pigmented lesions are common, so the resource implication for even a small increase in the threshold to excise lesions in populations where melanoma rates are increasing, will avoid a considerable healthcare burden to both patient and healthcare provider, as long as lesions that are not excised turn out to be harmless.

Delay in diagnosis of a BCC as a result of a false‐negative test is not as serious as for melanoma because BCCs are usually slow‐growing and very unlikely to metastasise. However, delayed diagnosis can result in larger and more complex surgical procedures with consequently greater morbidity. Very sensitive diagnostic tests for BCC, on the other hand, may compromise on lower specificity leading to a higher false‐positive rate and an enormous burden of skin surgery, so a balance between sensitivity and specificity is necessary. As with melanoma, the consequences of falsely reassuring a person with cSCC that they do not have skin cancer can be serious and potentially fatal. Thus, a good diagnostic test for cSCC should demonstrate high sensitivity and a corresponding high negative predictive value. A test that can reduce false positive clinical diagnoses without missing true cases of disease has patient and resource benefits. False‐positive clinical diagnoses not only cause unnecessary morbidity from the biopsy but could lead to initiation of inappropriate therapies and also increase patient anxiety.

Studies have also evaluated HFUS as a method for non‐invasive measurement of melanoma thickness in vivo so that melanomas can be excised with the appropriate margin in a single surgical procedure as opposed to two separate procedures (Jasaitiene 2011; Machet 2009; Meyer 2014). In addition to its optical B‐mode imaging cousin, Wang 2013 evaluated optical coherence tomography, while Crisan 2013 studied its role in appropriate treatment planning for BCC. There is potential for refining surgical procedures, as well as increasing the use and efficacy of non‐surgical methods of treating BCC, if non‐invasive imaging can be developed that allows confirmation of tumour clearance. However, this review does not consider any of these uses.

Alternative test(s)

Doppler ultrasound, unlike B‐mode ultrasound, measures moving structures such as blood cells, as opposed to stationary tissues (Kleinerman 2012), and it shows the relative speed of blood flow as well as relative vessel size and density. In skin cancer, it can be used in combination with B‐mode HFUS and may have value for staging or assessing the aggressiveness of malignancy due to increased vascular proliferation. Doppler ultrasound may be useful in preoperative staging due to correlation between the extent of vascularisation and blood flow with Breslow thickness. As a stand‐alone technique, Doppler ultrasound is not useful to differentiate skin cancers from benign lesions (Kleinerman 2012), so we do not include it as an index test; however, its use in combination with high‐frequency ultrasound may be able to improve lesion discrimination.

Cochrane DTA Reviews have assessed a number of other tests that may have a role in the diagnosis of skin cancer as part of this series, for example, visual inspection and dermoscopy (Dinnes 2018a; Dinnes 2018b; Dinnes 2018c); reflectance confocal microscopy (RCM) (Dinnes 2018d; Dinnes 2018e); optical coherence tomography (OCT) (Ferrante di Ruffano 2018a); and computer‐assisted diagnosis (CAD) techniques applied to various types of images, including those generated by dermoscopy, diffuse reflectance spectrophotometry (DRS) and electrical impedance spectroscopy (EIS) (Ferrante di Ruffano 2018b).

RCM and OCT are two alternative ways to achieve depth‐resolved optical reflectance imaging. To attain axial resolution, RCM uses a very low numerical aperture with out‐of‐focus data suppression, while OCT uses interferometry to isolate optical reflections at a defined echo time (conceptually similar to HFUS). They are emerging as non‐invasive adjuncts to dermoscopy in a specialist setting, and RCM can potentially serve as an alternative to dermoscopy for skin cancer diagnosis (Edwards 2016).

RCM and OCT differ from each other in that RCM tends to use a shorter wavelength (830 nm as opposed to 1305 nm for OCT) and has considerably less penetration (RCM < 300 µm; OCT < 2 mm), poorer depth of focus (RCM 3 µm to 5 µm; OCT 1 mm), and a more limited basic field of view (RCM basic 500 µm × 500 µm in the horizontal plane; OCT basic 6 mm × 6 mm) than OCT, but it has better lateral resolution (RCM 1 µm, cellular; OCT 7.5 µm, near cellular). They have similar axial resolution, however (RCM 3 µm to 5 µm; OCT 5 µm), and both have fields of view that are extendible by mechanical scanning and image mosaicking, although for equivalent fields of view 3D imaging is much faster with OCT (RCM for mosaicked field of view and stack > 10 min; OCT 6 cross‐sectional frames per second, < 2 min for 6 mm × 6 mm × 2 mm volume). With RCM, the contrast for the monochrome images produced is achieved by the variation of the optical scattering properties within the skin when illuminated by a near‐infrared light. At a wavelength of 830 nm, the greatest contrast is achieved from melanin, so that RCM is recommended as particularly useful for assessing pigmented lesions (Dinnes 2018d). Similar to Doppler ultrasound but with higher resolution, vascular flow information can be extracted from OCT images, allowing the visualisation of neovascularisation, potentially enabling earlier diagnosis of melanoma (Kokolakis 2012; Themstrup 2015).

CAD or artificial intelligence‐based techniques use predefined algorithms to process and manipulate acquired data to identify the features that discriminate malignant from benign lesions. The use of CAD‐based techniques has potential for both reducing the subjectivity of, and de‐skilling, the diagnosis of skin lesions. Although such techniques have most commonly been applied to digital dermoscopy images (Esteva 2017; Rajpara 2009), they may be applied to several types of images or spectra (e.g. Wallace 2000).

For example, SIAscopy and MelaFind are based on diffuse reflectance spectrophotometry. DRS also uses optical reflectance, albeit not depth‐resolved, but it distinguishes between lesion types based on the lesion‐average spectral shape and calibrated level of reflected light for wavelengths continuously varying from the ultraviolet (320 nm) to the near infrared (1100 nm) with a high spectral resolution (4 nm) (e.g. Marchesini 1992; Wallace 2000a). The extension to imaging spectrophotometry (DRSi) to allow spatial (dermoscopic) as well as spectral information to contribute to the diagnosis, as described in Haddock 2003, has resulted in the development of handheld DRSi units (Bish 2014). Researchers have studied two such units with limited spectral capability in both primary and secondary care settings: Moncrieff 2002 and Walter 2012 have evaluated SIAscopy, while Hauschild 2014, Monheit 2011 and Wells 2012 have assessed MelaFind. It is also possible to combine DRSi with HFUS (Bamber 2007). Such approaches remain under development.

The Nevisense system is based on electrical impedance spectroscopy (EIS). EIS measures a combination of resistance and capacitance of the tissue as a function of frequency of an alternating applied voltage. At high frequencies, conduction occurs easily through all tissue components, including cells, but at low frequencies current tends to flow only through the extracellular space. The spectral shape is thus sensitive to cellular components and dimensions, internal structure and cellular arrangements. The Nevisense EIS system measures at multiple depths and at 35 frequencies logarithmically distributed from 1.0 kHz to 2.5 MHz using a 5 mm × 5 mm area electrode covered in tiny pins that penetrate into the stratum corneum. Braun 2017 and Malvehy 2014 have evaluated it, finding high sensitivity but low specificity for melanoma. Although there is concern over a possible increase in needless excision of benign atypical melanocytic lesions (Ceder 2016), this concern is counterpoised against an indication of promise for reducing the need for short‐term sequential digital dermoscopy (Rocha 2017).

DRS and EIS have not been the subject of individual test reviews due to an anticipated lack of data; however, where available, we have included CAD‐based uses of these techniques in our review of CAD for the detection of skin cancer (Ferrante di Ruffano 2018b).

Evidence permitting, we will compare the accuracy of available tests in an overview of reviews, exploiting within‐study comparisons of tests and allowing the analysis and comparison of commonly used diagnostic strategies where tests may be used alone or in combination.

Rationale

Our series of reviews of diagnostic tests used to assist clinical diagnosis of skin cancer aims to identify the most accurate approaches to diagnosis and provide clinical and policy decision‐makers with the highest possible standard of evidence on which to base decisions. With increasing melanoma and basal cell carcinoma incidence and the push towards the use of dermoscopy and other high‐resolution image analysis in primary care, the anxiety around missing early malignant lesions needs to be balanced against the risk of too many referrals, to avoid sending too many people with benign lesions for a specialist opinion. It is questionable whether all skin cancers picked up by sophisticated techniques, even in specialist settings, help to reduce morbidity and mortality, and there is concern that newer technologies run the risk of increasing false‐positive diagnoses. It is also possible that use of some technologies, e.g. widespread use of dermoscopy in primary care with little or no training, could actually result in harm by missing melanomas if they are used as replacement technologies for traditional history‐taking and clinical examination of the entire skin. Many branches of medicine have noted the danger of such 'gizmo idolatry' amongst doctors (Leff 2008).

To date, the expense (in terms of both equipment and staff time) and the need for specialised training have limited the use of tests such as RCM. If shown to be sufficiently accurate, a test such as HFUS could prove to be a relatively low‐cost tool to assist in the earlier diagnosis and better management of skin cancer.

This review follows a generic protocol that covers the full series of Cochrane DTA Reviews for the diagnosis of melanoma and keratinocyte skin cancers (Dinnes 2015a; Dinnes 2015b). The Background and Methods sections of this review therefore use some text that was originally published in the protocols (Dinnes 2015a; Dinnes 2015b), plus text that overlaps some of our other reviews (Dinnes 2018a; Dinnes 2018b; Dinnes 2018c).

Objectives

To determine the diagnostic accuracy of high‐frequency ultrasound to assist in the diagnosis of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults.
 To determine the diagnostic accuracy of high‐frequency ultrasound to assist in the diagnosis of basal cell carcinoma in adults.
 To determine the diagnostic accuracy of high‐frequency ultrasound to assist in the diagnosis of cutaneous squamous cell carcinoma in adults.

Secondary objectives

To determine the diagnostic accuracy of Doppler ultrasound plus HFUS for the diagnosis of each of the three target conditions (cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, BCC or cSCC). We set out to address a range of potential sources of heterogeneity for investigation across our series of reviews, as described in Appendix 4 and outlined in our generic protocols (Dinnes 2015a; Dinnes 2015b); however, our ability to investigate these was necessarily limited by the available data on each individual test reviewed. Ultimately, we conducted no heterogeneity investigations for this review of HFUS.

Methods

Criteria for considering studies for this review

Types of studies

We included test accuracy studies that allow comparison of the result of the index test with that of a reference standard, including the following.

• Studies where all participants received a single index test and a reference standard.

• Studies where all participants received more than one index test and reference standard.

• Studies where participants were allocated (by any method) to receive different index tests or combinations of index tests and all received a reference standard (between‐person comparative studies (BPC)).

• Studies that recruited series of participants unselected by true disease status (referred to as case series for the purposes of this review).

• Diagnostic case‐control studies that separately recruited diseased and non‐diseased groups (see Rutjes 2005).

• Both prospective and retrospective studies.

• Studies where previously acquired clinical or dermoscopic images were retrieved and prospectively interpreted for study purposes.

We excluded studies from which we could not extract 2 × 2 contingency data or if they included fewer than five melanoma, BCC or cSCC cases or fewer than five benign lesions.

Studies available only as conference abstracts were excluded; however, attempts were made to identify full papers for potentially relevant conference abstracts (Searching other resources).

Participants

We included studies in adults with lesions suspicious for skin cancer. We excluded studies that recruited only participants with malignant diagnoses and studies that compared test results in participants with malignancy compared with test results based on 'normal' skin as controls, due to the bias inherent in such comparisons (Rutjes 2006). We excluded studies in children and those that clearly reported inclusion of more than 50% of participants aged 16 and under.

Index tests

Studies evaluating HFUS alone or in combination with Doppler ultrasound were eligible. HFUS was considered to have been evaluated if the centre (or median) frequency of the transmitted pulse was at least 20 MHz.

Studies should ideally evaluate a predefined 'rule' or algorithm describing combinations of ultrasound characteristics that determine the presence or absence of melanoma, BCC or cSCC. However, as HFUS is in a relatively early phase of development, we included studies if we could extract 2 × 2 contingency table data based on the presence or absence of at least two ultrasound features related to tissue morphology or acoustic properties, for example echogenicity, homogeneity of appearance and definition of margins. Studies attempting to quantify HFUS parameters were also eligible for inclusion. There was no requirement for studies to have explicitly set out to estimate the diagnostic accuracy of the parameters assessed.

We made no exclusions according to test observer experience or qualifications.

Target conditions

We defined the target conditions as the detection of:

• any form of invasive cutaneous melanoma or atypical intraepidermal melanocytic variants (i.e. including melanoma in situ or lentigo maligna);

• BCC (all subtypes);

• cSCC.

Reference standards

The ideal reference standard was histopathological diagnosis of the excised lesion or biopsy sample in all eligible lesions. A qualified pathologist or dermatopathologist should perform histopathology. Ideally, reporting should be standardised, detailing a minimum dataset including the histopathological features of melanoma needed to determine the American Joint Committee on Cancer (AJCC) Staging System (e.g. Slater 2014). We did not apply the reporting standard as a necessary inclusion criterion but extracted any pertinent information.

Partial verification (applying the reference test only to a subset of those undergoing the index test) was of concern given that biopsy or excisions are unlikely to be carried out for all clinically benign lesions within a representative population sample. Therefore, we accepted clinical follow‐up of clinically benign lesions as an eligible reference standard, whilst recognising the risk of differential verification bias (as misclassification rates of histopathology and follow‐up will differ) in our quality assessment of studies.

Additional eligible reference standards included cancer registry follow‐up and 'expert opinion' with no histology or clinical follow‐up. Cancer registry follow‐up is considered less desirable than active clinical follow‐up, as follow‐up is not carried out within the control of the study investigators. Furthermore, if participant‐based analyses as opposed to lesion‐based analyses are presented, it may be difficult to determine whether the detection of a malignant lesion during follow‐up is the same lesion that originally tested negative on the index test.

All of the above were considered eligible reference standards with the following caveats.

• All study participants with a final diagnosis of the target disorder must have a histological diagnosis, either subsequent to the application of the index test or after a period of clinical follow‐up.

• At least 50% of all participants with benign lesions must have either a histological diagnosis or clinical follow‐up to confirm benignity.

Search methods for identification of studies

Electronic searches

The Information Specialist (SB) carried out a comprehensive search for published and unpublished studies. A single large literature search was conducted to cover all topics in the programme grant (see Appendix 1 for a summary of reviews included in the programme grant). This allowed for the screening of search results for potentially relevant papers for all reviews at the same time. A search combining disease related terms with terms related to the test names, using both text words and subject headings was formulated. The search strategy was designed to capture studies evaluating tests for the diagnosis or staging of skin cancer. As the majority of records were related to the searches for tests for staging of disease, a filter using terms related to cancer staging and to accuracy indices was applied to the staging test search, to try to eliminate irrelevant studies, for example, those using imaging tests to assess treatment effectiveness. A sample of 300 records that would be missed by applying this filter was screened and the filter adjusted to include potentially relevant studies. When piloted on MEDLINE, inclusion of the filter for the staging tests reduced the overall numbers by around 6000. The final search strategy, incorporating the filter, was subsequently applied to all bibliographic databases as listed below (Appendix 5). The final search result was cross‐checked against the list of studies included in five systematic reviews; our search identified all but one of the studies, and this study was not indexed on MEDLINE. The Information Specialist devised the search strategy, with input from the Information Specialist from Cochrane Skin. No additional limits were used.

We searched the following bibliographic databases to 29 August 2016 for relevant published studies.

• MEDLINE via OVID (from 1946).

• MEDLINE In‐Process & Other Non‐Indexed Citations via OVID.

• Embase OVID (from 1980).

We searched the following bibliographic databases to 30 August 2016 for relevant published studies.

• the Cochrane Central Register of Controlled Trials (CENTRAL; 2016, Issue 7) in the Cochrane Library;

• the Cochrane Database of Systematic Reviews (CDSR; 2016, Issue 8) in the Cochrane Library;

• Cochrane Database of Abstracts of Reviews of Effects (DARE; 2015, Issue 2);

• CRD HTA (Health Technology Assessment) database, 2016, Issue 3; and

• CINAHL (Cumulative Index to Nursing and Allied Health Literature via EBSCO from 1960).

We searched the following databases for relevant unpublished studies using a strategy based on the MEDLINE search:

• CPCI (Conference Proceedings Citation Index), via Web of Science™ (from 1990; searched 28 August 2016); and

• SCI Science Citation Index Expanded™ via Web of Science™ (from 1900, using the 'Proceedings and Meetings Abstracts' Limit function; searched 29 August 2016).

We searched the following trials registers using the search terms 'melanoma', 'squamous cell', 'basal cell' and 'skin cancer' combined with 'diagnosis':

• Zetoc (from 1993; searched 28 August 2016).

• The US National Institutes of Health Ongoing Trials Register (www.clinicaltrials.gov); searched 29 August 2016.

• NIHR Clinical Research Network Portfolio Database (www.nihr.ac.uk/research‐and‐impact/nihr‐clinical‐research‐network‐portfolio/); searched 29 August 2016.

• The World Health Organization International Clinical Trials Registry Platform (apps.who.int/trialsearch/); searched 29 August 2016.

We aimed to identify all relevant studies regardless of language or publication status (published, unpublished, in press, or in progress). We applied no date limits.

Searching other resources

We screened relevant systematic reviews identified by the searches for their included primary studies and included any missed by our searches. We checked the reference lists of all included papers, and subject experts within the author team reviewed the final list of included studies. We did not perform electronic citation searching.

Data collection and analysis

Selection of studies

At least one author (JDi or NC) screened titles and abstracts, discussing and resolving any queries by consensus. A pilot screen of 539 MEDLINE references showed good agreement (89% with a kappa of 0.77) between screeners. We included primary test accuracy studies and test accuracy reviews (for scanning of reference lists) of any test used to investigate suspected melanoma, BCC, or cSCC at initial screening. Both a clinical reviewer (from a team of 12 clinician reviewers) and a methodologist reviewer (JDi or NC) independently applied inclusion criteria (Appendix 6) to all full text articles, resolving disagreements by discussion or consultation with a third party if no consensus could be reached (JDe, CD, HW, or RM). We contacted authors of eligible studies when studies presented insufficient data to allow for the construction of 2 × 2 contingency tables.

Data extraction and management

One clinical (as detailed above) and one methodologist reviewer (JDi, NC or LFR) independently extracted data concerning details of the study design, participants, index test(s) or test combinations and criteria for index test positivity, reference standards, and data required to populate a 2 × 2 diagnostic contingency table for each index test using a piloted data extraction form. We extracted data at all available index test thresholds, resolving disagreements by discussion or in consultation with a third party, in case no consensus could be reached (JDe, CD, HW, or RM).

We contacted authors of included studies in case of missing information related to the diagnostic threshold or target condition (in particular to allow the differentiation of invasive cancers from in situ variants). We contacted authors of conference abstracts published from 2013 to 2015 to ask whether full data were available. If we could not identify a full paper, we marked conference abstracts as 'pending', and we will revisit them in a future review update.

Dealing with multiple publications and companion papers

Where we identified multiple reports of a primary study, we maximised yield of information by collating all available data. Where there were inconsistencies in reporting or overlapping study populations, we contacted study authors for clarification in the first instance. If this contact with authors was unsuccessful, we used the most complete and up‐to‐date data source where possible.

Assessment of methodological quality

We assessed risk of bias and applicability of included studies using the QUADAS‐2 checklist (Whiting 2011), tailored to the review topic (see Appendix 7). We piloted the modified QUADAS‐2 tool on a small number of included full‐text articles. One clinical (as detailed above) and one methodologist reviewer (JDi, NC or LFR) independently assessed risk of bias and applicability for the remaining studies, solving any disagreements by discussion or in consultation with a third party (JDe, CD, HW, or RM). We did not contact authors to clarify methodological uncertainties. The methodological quality assessment was therefore of the study as reported and may not always fully reflect the quality of the study as conducted.

Statistical analysis and data synthesis

Due to paucity of data and between‐study heterogeneity in the ultrasound characteristics and measurements that we investigated, we did not undertake a meta‐analysis for this review. For the diagnosis of melanoma, we considered any BCCs or invasive cSCCs that were positively identified in the 'disease negative' group to be true negative test results rather than as false positives, on the basis that excision of such lesions would be a positive outcome for the participants concerned. For the diagnosis of BCC, however, we considered any melanomas or cSCCs that were positively identified in the 'disease negative' group to be false positive results. We made this decision on the basis that the clinical management of a lesion considered to be a BCC might be quite different to that for a melanoma or cSCC and could potentially lead to a negative outcome for the participants concerned, for example if participants initiated a treatment other than excision.

We plotted estimates of sensitivity and specificity on coupled forest plots for each characteristic or threshold under consideration. Our unit of analysis was the lesion rather than the patient, as this was the most common way in which the primary studies reported data. As most participants have only one lesion to consider at a time, and as both index tests and reference standards are defined at the lesion level, the results are likely to be similar to those obtained at a participant level. We included data for Doppler ultrasound only if reported in combination with HFUS tissue morphological or acoustic property imaging; we did not evaluate the accuracy of Doppler ultrasound alone.

Investigations of heterogeneity

We examined heterogeneity between studies by visually inspecting the forest plots of sensitivity and specificity. We did not identify enough studies to allow meta‐regression to investigate potential sources of heterogeneity.

Sensitivity analyses

We did not conduct sensitivity analyses due to lack of data.

Assessment of reporting bias

Because of uncertainty about the determinants of publication bias for diagnostic accuracy studies and the inadequacy of tests for detecting funnel plot asymmetry (Deeks 2005), we did not perform any tests to detect publication bias.

Results

Results of the search

We identified and screened a total of 34,517 unique references for inclusion. Of these, we reviewed 1051 full‐text papers for eligibility for any one of the suite of reviews of tests to assist in the diagnosis of melanoma or keratinocyte skin cancer. Of the 1051 full‐text papers assessed, we excluded 848 and included 203 publications across all reviews in our series (see Figure 6 PRISMA flow diagram of search and eligibility results).

6.

PRISMA flow diagram.

Of the 41 studies tagged as potentially eligible for this review of HFUS, we included 6. One study could not be obtained from the British Library (Nitsche 1992). Exclusions were due to the use of:

• ineligible index tests (17 studies: 9 evaluations of Doppler ultrasound, 7 studies using ultrasound transducers with centre frequency less than 20 MHz, and one evaluating the accuracy of a single feature on ultrasound);

• ineligible study populations (4 studies: 2 recruiting only malignant lesions, and 2 including lesions that were not suspicious for skin cancer);

• ineligible definition of the target condition (8 studies: 4 identifying lesion thickness, 1 focusing on surgical margins, 2 investigating melanoma metastases, and 1 considering lesions such as dermatofibroma or Bowen's disease to be disease positive); and

• inadequate sample size (1 study).

The Characteristics of excluded studies provides a list of the 34 studies excluded from this review with reasons for exclusion, and a list of all studies excluded from the full series of reviews is also available as a separate pdf (please contact skin.cochrane.org for a copy of the pdf). We contacted the authors of one publication for the purposes of this review; however, they were unable to provide the additional data needed to allow the study to be included.

This review reports on a total of six cohorts of lesions published in six study publications and providing 29 datasets: 20 for melanoma and 9 for BCC. We did not identify data relating to the diagnosis of cSCC.

Studies included four case series of patients, either with pigmented lesions (Bessoud 2003; Clement 2001; Dummer 1995), or lesions described as suspicious for melanomas or BCC (Lassau 1997). Moreover, we found two case‐control type studies that included pigmented lesions with specific confirmed diagnoses of melanoma, seborrhoeic keratosis or benign naevi (Harland 2000; Rallan 2007). The Bessoud 2003 paper is from the same institution and has overlapping authorship with Lassau 1997, and there may have been an overlap in study participants. Only Rallan 2007 clearly described the basis for referral or selection for ultrasound examination, randomly selecting lesions referred from primary care due to suspicion of melanoma. Clement 2001 described the clinical diagnosis as 'hesitant' for more than half of included lesions, but none of the other studies gave any indication as to the equivocal nature or difficulty of diagnosis of the lesions included. The number of included patients ranged from 70 to 160 (reported in four studies) and lesions from 54 to 792. Only three studies reported participant characteristics such as age and gender.

All studies apart from Clement 2001, which focused primarily on the detection of BCC, reported data allowing the calculation of the accuracy of ultrasound for the detection of melanoma; two other studies also report data for detection of BCC (Dummer 1995; Lassau 1997). The prevalence of melanoma in the study samples ranged from 14% to 58%, and it appeared to be restricted to invasive melanoma in Dummer 1995 and Lassau 1997. The prevalence of BCC was 8% (Dummer 1995), 17% (Clement 2001), and 49% (Lassau 1997). In all studies apart from Lassau 1997 and Bessoud 2003, seborrhoeic keratosis made up at least 25% of the disease negative groups, and it was as high as 66% in Harland 2000, which studied seborrhoeic keratosis versus melanoma.

All six studies used 20 MHz ultrasound scanners with axial resolutions of 50 µm to 80 µm. Lateral resolutions ranged from about 100 µm in Bessoud 2003,Clement 2001,Lassau 1997 and Rallan 2007 to 300 µm in Harland 2000. Typically it was not clear how authors obtained the resolution values, and based on the example images in the papers, the instrumentation employed seemed to vary greatly in terms of other diagnostically important imaging performance properties such as signal dynamic range and signal‐to‐noise level, which trialists did not report. In some cases such performance appeared to be poor, providing little or no lesion internal detail compared with similar lesions on other systems. None of the studies described the qualifications or experience of the clinician carrying out and interpreting the ultrasound, and none reported whether the clinical or dermoscopic diagnosis of the lesion was available to aid test interpretation.

Three studies explicitly set out to establish the diagnostic accuracy of HFUS for the differentiation of melanomas from other skin lesions (Bessoud 2003; Harland 2000; Rallan 2007); the remaining three studies did not set out to evaluate test accuracy but presented data for the presence or absence of particular ultrasound characteristics that could be extracted into 2 × 2 contingency tables (Clement 2001; Dummer 1995; Lassau 1997). Qualitative HFUS characteristics that were considered were related to the echogenicity and homogeneity of appearance and to definition of margins (Bessoud 2003; Clement 2001; Dummer 1995; Lassau 1997). Four studies presented data for qualitative assessment of the presence or absence of particular structural characteristics (including echogenicity, homogeneity of appearance and definition of margins) on the HFUS image either alone (Dummer 1995; Lassau 1997; Clement 2001; Bessoud 2003) or in combination with Doppler ultrasound assessment of vascularity (Lassau 1997; Clement 2001; Bessoud 2003).

The remaining two studies examined different approaches to quantitatively interpret ultrasound findings. Harland 2000 attempted to classify lesions based on objective quantifications of the extent of ultrasound shadowing and the strength of the ultrasound entry echo to differentiate between melanoma and seborrhoeic keratosis, based on the dermal echogenicity ratio (DER) and presence of a thickened entry echo line (EEL), respectively. Rallan 2007 further developed this work with a prototype 3D HFUS C‐scan and reflex transmission imaging system to evaluate these features and make ultrasound images easier for dermatologists to interpret. This method produces three en face ultrasound images: a reflex transmission image (RTI), a lesional backscatter image (LBI) and an entry echo image (EEI), which relate to objectively quantified lesion attenuation properties, intralesional sound reflection and surface sound reflectance characteristics, respectively. For each image, investigators estimated two quantitative features (contrast and heterogeneity) and compared them between lesion groups (melanoma versus seborrhoeic keratosis, and melanoma versus other benign pigmented lesions). Mean RTI contrast, LBI relative heterogeneity, and EEI relative heterogeneity were each significantly different between melanoma and seborrhoeic keratosis and between melanoma and benign naevi; these three features were combined using an 'or' rule with specificity estimated at 100% sensitivity (Rallan 2007). Authors reported the required values for each of the three parameters to be considered 'positive' graphically but not numerically (Rallan 2007).

Three studies using qualitative HFUS interpretation reported the exclusion of lesions not visualised by ultrasound: 10% in Lassau 1997 (including 3 melanomas), 12% in Bessoud 2003 (including for 5 melanomas), and 22% in Clement 2001 (including 5 melanomas). In all studies the reference standard diagnosis was made by histology alone (i.e. all lesions either excised or biopsied). Histological diagnosis was based on excisional biopsy (Dummer 1995), surgical resection or excision (Lassau 1997; Bessoud 2003; Harland 2000; Rallan 2007), and either approach (Clement 2001).

Methodological quality of included studies

Figure 7 and Figure 8 summarize the overall methodological quality of all six included studies.

7.

Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies

8.

Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study

Two studies were at high risk of bias for participant selection due to their selective inclusion of participants with particular histological lesion types (Harland 2000; Rallan 2007). Five studies did not clearly describe participant recruitment as random or consecutive, and four did not clearly report any exclusion criteria. We judged one study as being of low concern for applicability of participants and setting (Clement 2001). We deemed four studies as being of high and one, Dummer 1995, as being of unclear concern for applicability of participants due to unrepresentative patient samples (Harland 2000; Lassau 1997; Rallan 2007), inclusion of multiple lesions per patient (Bessoud 2003), or providing insufficient information on which to make a judgement (Dummer 1995). All studies included only lesions selected for excision.

Only one study was at low risk of bias in the index test domain. We considered that ultrasound was interpreted prior to the histological reference standard in all studies, but only one clearly reported prior specification of the diagnostic threshold or ultrasound characteristics used to differentiate melanomas from other lesions (Bessoud 2003). The other studies were all rated as high risk for this item, either because they did not clearly set out to examine the accuracy of HFUS (Clement 2001; Dummer 1995; Lassau 1997), or because they deliberately set their thresholds to achieve 100% sensitivity (Harland 2000; Rallan 2007). Two studies caused high concern around the applicability of the index test: Harland 2000 due to the use of a prototype ultrasound device and Rallan 2007 due to a relatively experimental approach to the index test. All studies clearly described the criteria or diagnostic thresholds used, but no study provided information on the expertise and experience of the test operator or sonographer.

All studies reported the use of an acceptable reference standard, but only one clearly reported blinding of the reference standard to the ultrasound result (Harland 2000), and none of the studies reported blinding to the referral diagnosis (based on clinical examination or dermoscopy). For the applicability of the reference standard, no study reported using expert diagnosis to provide the final diagnosis of any lesion, and only one reported histopathology interpretation by an experienced histopathologist or by a dermatopathologist.

All studies used the same reference standard in all participants, and two were unclear on the interval between the application of the index test and excision for histology (Bessoud 2003; Harland 2000). Three studies reported exclusions due to lesions not being visualised on ultrasound (Bessoud 2003; Clement 2001; Lassau 1997); however, all three provided a breakdown of the final histologic diagnosis for these lesions, so we judged them as being at low risk on the flow and timing domain. Three studies did not report any exclusions due to lack of visualisation of lesions (Dummer 1995; Harland 2000; Rallan 2007). Two of these allowed the ultrasound features employed to be measured regardless of whether the lesions were visualised or not, and we did not judge them as being at low risk of bias on this item (Harland 2000; Rallan 2007).

Findings

Lack of data, combined with between‐study variations in populations, ultrasound techniques and characteristics, and measurements investigated precluded meta‐analysis. We summarise study results below according to target condition: melanoma or BCC. We found no data on the identification of cSCC. Appendix 8 provides summary details, while forest plots of available study data are in Figure 9 (HFUS for differentiation of melanoma), Figure 10 (for HFUS combined with Doppler US for melanoma) and Figure 11 (HFUS for differentiation of BCC).

9.

Forest plot of tests for differentiation of melanoma from other lesions using combinations of HFUS characteristics and quantitative measurements of HFUS outputs

10.

Forest plot of tests for the differentiation of melanoma from other lesions using HFUS and Doppler US

11.

Forest plot of tests for the differentiation of BCC from other lesion types using HFUS

Detection of invasive melanoma or atypical intraepidermal melanocytic variants

Combinations of subjective assessments of HFUS features

We were able to extract HFUS data related to the qualitative assessment of the presence or absence of different combinations of lesions' morphological and structural characteristics as an indicator of melanoma from three studies (Bessoud 2003; Dummer 1995; Lassau 1997), one of which set out to assess the diagnostic accuracy of these characteristics for the differentiation of melanoma from other lesions (Bessoud 2003;Figure 9; Appendix 8).

Dummer 1995 reported recruitment of a series of 792 pigmented lesions with a range of final diagnoses including melanoma (14%), BCC (8%), benign naevi (39%), seborrhoeic keratosis (27%), and dermatofibroma or angioma (13%). Sensitivities and specificities estimated from the data ranged from 64% (95% CI 60% to 87%) to 83% (95% CI 75% to 90%) for homogenous and hypoechoic lesions and from 22% (95% CI 19% to 26%) to 91% (95% CI 84% to 95%) for hypoechoic lesions with sharp lateral margins. For each of the combinations of characteristics examined in this study, a number of BCCs were found to be 'test positive', i.e. displaying the characteristics under consideration. As per our protocol, we reclassified these false positive BCCs as true negative test results (increasing specificity) on the basis that a positive test result leading to excision of the BCCs would not be a negative patient outcome. The number of BCCs that we artificially reclassified as true negative despite the presence of the HFUS image features of interest ranged from 5 (for echo‐poor lesions with homogenous internal echoes) to 57 (for echo‐poor lesions with sharp basal margins).

Bessoud 2003 and Lassau 1997 presented data for the presence of hypoechoic, homogenous, well‐defined lesions, as well as results for Doppler ultrasound that could be combined with HFUS data. Bessoud 2003 included a series of 114 pigmented lesions (7 of which did not undergo Doppler ultrasound); included lesions were primarily invasive melanomas (57%) or benign naevi (29%) with smaller percentages of BCC (4%), seborrhoeic keratosis (4%) and other benign lesions. Lassau 1997 included 70 lesions clinically suspected of being either melanoma (n = 38) or BCC (n = 32). Visualisation of 7 lesions on ultrasound was not possible, so investigators excluded them, leaving 19 (27%) invasive melanoma, 31 (44%) BCC, 1 neurosarcoma, and 12 (17%) benign naevi (3 of the 7 lesions not visualised on HFUS were melanomas).

The sensitivity of the combined HFUS characteristics was 100% in both studies (lower limits of the 95% CIs for sensitivities were 94% in Bessoud 2003 (114 lesions, 65 melanomas) and 82% in Lassau 1997 (63 lesions, 19 melanomas)), with specificities of 33% (95% CI 20% to 48%) and 73% (95% CI 57% to 85%), respectively. Excluding BCCs from Lassau 1997 resulted in a specificity of 8% (95% CI 0% to 36%; 32 lesions; 19 melanomas), the 12 benign naevi all being considered hypoechoic, homogenous and well defined. Both studies reported all BCCs as 'negative' on ultrasound, that is, with an absence of investigated characteristics (Bessoud 2003; Lassau 1997). Both studies also reported five melanomas amongst the lesions not visualised by ultrasound (Appendix 8).

Combinations of subjective assessments of HFUS features with Doppler US

Using data presented in Lassau 1997 for the presence of hypoechoic, homogenous, and well‐defined lesions on HFUS with the presence of intratumoural vessels on Doppler ultrasound (on an either/or basis) makes no difference to the sensitivity and specificity achieved using HFUS alone for discriminating between invasive melanoma (n = 19) and all other included lesions (n = 44). Only three melanomas (already picked up as test positive on HFUS) displayed any evidence of vascularity on Doppler (sensitivity 100%, 95% CI 82% to 100%; specificity 73%, 95% CI 57% to 85%; Figure 10). The HFUS and Doppler characteristics can be combined on an 'and' basis for both Bessoud 2003 and Lassau 1997, with lesions that were hypoechoic, homogenous, well defined and exhibited intralesional vessels on Doppler being considered test positive. Thus, sensitivities were 34% (95% CI 22% to 47%; n = 65 melanomas) and 16% (95% CI 3% to 40%; n = 19 melanomas), respectively, with specificities of 100% (95% CI 92% to 100%) for both studies (number of benign lesions: 45 and 44 respectively).

Quantitative assessment of HFUS features

Two studies reported quantitative assessments of the ultrasound image using the strength and heterogeneity of ultrasound shadowing and the strength and heterogeneity of the ultrasound surface entry echo (Harland 2000; Rallan 2007). Both studies included only melanoma (n = 19 and n = 14 in Harland 2000 and Rallan 2007, respectively), melanoma in situ (n = 6 and n = 11), benign naevi (n = 15 and n = 38) or seborrhoeic keratosis (n = 29 and n = 24). The main comparison in Harland 2000 was between melanoma and seborrhoeic keratosis (benign naevi excluded). Setting the DER at < 3 to ensure sensitivity of 100% produced a specificity of 79% (95% CI 60% to 92%); the absence of an EEL resulted in sensitivity of 96% (95% CI 80% to 100%) and specificity 90% (95% CI 73% to 98%) for the same comparison. Combining the two characteristics on an either/or basis (such that sensitivity was 100%) increased specificity to 93% (95% CI 77% to 99%) for discrimination of melanoma from seborrhoeic keratosis (Figure 9). Of the 15 benign naevi in this study, 6 were reported to have characteristics associated with EEL enhancement (or EEE), suggesting that 9 would be considered 'test positive' (absence of an EEL); inclusion of these lesions as disease negative would reduce the observed specificity.

Rallan 2007, on a prototype 3D HFUS C‐scan with reflex transmission imaging, found significant differences in the mean values of RTI contrast, LBI relative heterogeneity, and EEI relative heterogeneity between melanoma and seborrhoeic keratosis and between melanoma and benign naevi. When these three features were combined using an 'or' rule with sensitivity for melanoma discrimination of 100% (95% CI 86% to 100%), the resulting specificity was 65% (95% CI 51% to 76%; Figure 9).

Detection of BCC

Combinations of subjective assessments of HFUS features

Three studies reported data that could be used to derive the accuracy of ultrasound characteristics for BCC: Clement 2001 and Dummer 1995 in series of pigmented lesions, and Lassau 1997 in lesions suspicious for either melanoma or for BCC. Dummer 1995 and Lassau 1997 also reported data for melanoma. None of the three studies set out to establish the accuracy of the reported ultrasound characteristics. Clement 2001 included a series of 176 pigmented lesions, 38 of which were not visualised on ultrasound (including 5 melanomas); the remaining 138 lesions included one invasive melanoma, 23 (17%) BCC, 61 (44%) benign naevi, and 29 (21%) seborrhoeic keratoses, amongst others.

Using hypoechoic and homogenous appearance as a positive indicator for BCC, sensitivity was 91% (95% CI 72% to 99%) and specificity 14% (95% CI 8% to 22%) for Clement 2001 (138 lesions; 23 BCC), while they were 8% (95% CI 3% to 17%) and 54% (95% CI 50% to 57%), respectively, in Dummer 1995 (792 lesions; 65 BCC) (Figure 11).

Considering lesions that were hypoechoic and well defined as positive for BCC resulted in sensitivity of 83% (95% CI 61% to 95%) and specificity 32% (95% CI 24% to 42%) in Clement 2001. Dummer 1995 reported numbers of lesions with sharp basal margins and with sharp lateral margins. Considering lesions that were hypoechoic with sharp basal margins as positive for BCC resulted in sensitivity of 86% (95% CI 75% to 93%) and specificity of 20% (95% CI 17% to 23%); considering lesions that were hypoechoic with sharp lateral margins as positive for BCC resulted in sensitivity of 42% (95% CI 29% to 54%) and specificity of 13% (95% CI 11% to 16%).

Finally, data from Lassau 1997 could be derived to consider hypoechoic, homogenous and well‐defined lesions as BCC (i.e. the same characteristics previously considered to be positive indicators for melanoma); this combination resulted in sensitivity of 0% (95% CI 0% to 11%) and specificity of 3% (95% CI 0% to 16%) (63 lesions; 31 BCCs; Figure 11). All BCCs were reportedly hypoechoic but with a heterogeneous echostructure and lateral extensions with irregular margins, i.e. negative on two of the characteristics considered (Lassau 1997). If one instead considers the presence of a heterogeneous echostructure and lateral extensions with irregular margins to be positive indicators of BCC (i.e. reversing the 2 × 2 contingency table), the resulting sensitivity is 100% (95% CI 89% to 100%) and specificity 97% (95% CI 84% to 100%), with no melanomas and none of the benign naevi displaying these characteristics. Bessoud 2003 also reported all four included BCCs to be heterogenous and poorly defined; however, a further 12 lesions including keratosis, melanosis and neurosarcoma also demonstrated these characteristics.

Combinations of subjective assessments of HFUS features with Doppler US

Two of the studies reporting data for BCC also employed Doppler ultrasound in combination with HFUS (Clement 2001; Lassau 1997). Lassau 1997 allowed extraction of accuracy data only for the detection of melanoma (melanoma versus benign naevi), while for Clement 2001, consideration of lesions that were hyperechoic on HFUS and displaying vascularity on Doppler produced a sensitivity of 0% (95% CI 0% to 15%) and a specificity of 89% (95% CI 81% to 94%). Considering the absence of these characteristics as indicative of BCC (i.e. hypoechoic with no vascularity on Doppler) would reverse these estimates, giving sensitivity of 100% and specificity of 11%.

Investigations of heterogeneity

We were unable to undertake formal investigations of heterogeneity due to insufficient study numbers.

Discussion

Summary of main results

This review aimed to assess the accuracy of high‐frequency ultrasound as an aid to diagnosing melanoma, BCC and cSCC in adults. We did not identify any eligible data on cSCC. We included six studies evaluating high‐frequency ultrasound, three of which also evaluated Doppler ultrasound (Table 1).

Studies were generally poorly reported, so we could not clearly judge methodological quality and applicability of findings; this was in part due to the fact that half of the studies did not set out to establish test accuracy. We noted particularly high concerns in regard to the selection of study participants, with high proportions of malignant lesions and an unrepresentative spectrum of disease in the disease‐negative groups. Authors did not always describe the clinical pathway and referral process for ultrasound imaging well. All studies used 20 MHz ultrasound devices at a range of resolutions and a number of different qualitative and quantitative thresholds, some of which were clearly data driven or not pre‐specified. Half the studies (all using qualitative or subjective assessments of HFUS images) excluded a considerable proportion of lesions because they were not visualised by ultrasound. It is not clear from these studies whether this should be considered a failure of the test or whether the lack of visualisation of a lesion on HFUS provides further diagnostic information that may assist in the differential diagnosis. Studies applying quantitative interpretations of HFUS allowed some ultrasound features to be measured regardless of lesion visibility. Authors provided no information regarding the clinicians undertaking and interpreting the tests, limiting the generalisability of results particularly for those relying on qualitative interpretation of HFUS features. The final diagnoses were established by histology in all studies; only one study reported blinding to the ultrasound result or referral diagnosis. Sources of heterogeneity included patient selection, ultrasound techniques, test thresholds, prior testing and blinding.

For the detection of melanoma, derived sensitivities were at least 83% (95% CI 75% to 90%), with the combination of three qualitative features (lesions appearing hypoechoic, homogenous and well defined) and quantitative assessments of images demonstrated 100% sensitivity in four studies (the widest 95% CI being 80% to 100%), although in two of these the decision thresholds were deliberately set to achieve 100% sensitivity in order to discover resulting specificity. Between three and five melanomas were amongst the lesions not visualised by the HFUS in three studies, with no index test 'failures' reported by the two studies assessing quantitative metrics. Specificities varied from 8% (95% CI 0% to 36%) to 73% (95% CI 57% to 85%) for qualitative characteristics, all of which included BCC in the disease‐negative group, which tends to increase specificity, and from 65% (95% CI 51% to 76%) to 90% (95% CI 73% to 98%) for quantitative measurements, none of which included BCC in the disease‐absent group. For the detection of BCC, sensitivities and specificities were highly variable, making summary statements difficult. One study suggested that the presence of heterogeneity and poorly defined margins might differentiate BCCs from melanomas and benign naevi, although another identified other lesions demonstrating similar characteristics that might limit their usefulness in a more widely defined population.

Strengths and weaknesses of the review

The strengths of this review include an in‐depth and comprehensive electronic literature search, systematic review methods including double extraction of papers by both clinicians and methodologists, and contact with authors to allow study inclusion or clarify data. We planned a clear analysis structure to allow estimation of test accuracy in different study populations, and we undertook a detailed and replicable analysis of methodological quality.

The main concerns for the review arise from the poor reporting of primary studies and the fact that the studies were not all designed as test accuracy studies; in half of included studies, data to allow the estimation of sensitivity and specificity were derived from the information on image descriptions presented by the study authors. All three studies using qualitative interpretation of HFUS reported that they had to exclude a number of lesions (including some melanomas) because they could not be visualised, resulting in an over‐estimation of sensitivity for the characteristics assessed. However, two of the remaining studies using quantitative HFUS metrics and methodology allowed inclusion of all lesions regardless of their visualisation.

When estimating accuracy for the diagnosis of melanoma, we classed any correctly identified BCCs as true negative results as opposed to false positives, on the basis that removal of a BCC in the attempt to identify melanomas would not be a negative consequence of the test. This will have the effect of increasing specificity compared to studies excluding BCCs. When estimating accuracy for the diagnosis of BCC, however, we considered any other skin cancers that were incorrectly identified as BCC (e.g. melanomas or cSCCs) to be false positive results, as the subsequent management of a BCC can be quite different to that of a melanoma or SCC, and it is important that a test can accurately differentiate between malignancies.

Applicability of findings to the review question

The data included in this review came from preliminary exploratory studies and are unlikely to be generally applicable to predicting the diagnostic accuracies that would be expected in standard clinical practice, where people present with a broad range of different lesion types. Narrow definitions of the eligible study populations, lack of clarity regarding the patient pathway and any prior testing, and wide variation in the type and performance of the HFUS equipment employed as well as in the method used for image feature scoring, restrict generalisation and applicability. It is not always clear whether the particular test methods used could be transferred to a clinical setting.

Authors' conclusions

Implications for practice.

We could not produce any summary estimates of test accuracy to answer the research question for this review. High‐frequency ultrasound may prove to be an additional tool to assist in the differentiation of melanoma from other lesions; however, the current evidence is based on participants with highly selected lesion types, and it is unclear how their results would translate in clinical practice. The lack of visualisation of lesions on HFUS is potentially a major disadvantage unless the lack of visualisation has a clear interpretation that can be used to inform management decisions, clinicians can employ ultrasound metrics that do not depend on lesion visualisation, or ultrasound visualisation can be improved with equipment development. Given the between‐study heterogeneity, unclear to low methodological quality and applicability of findings, and limited volume of evidence, we can draw no implications for practice. The main value of the preliminary studies included in this review may be to provide guidance on the possible components of rules for diagnosing melanoma or BCC using HFUS that will require further evaluation.

Implications for research.

Prospective evaluation of high‐frequency ultrasound added to visual inspection and dermoscopy alone in a standard health care setting would be required for a full and proper evaluation of accuracy. A clearly defined and representative population of participants with a range of different lesion types is needed to establish the participant groups to whom study results can be applied in practice. HFUS technology continues to be developed, so it is important that current equipment is employed, using compatible systems across centres, appropriate harmonisation in cross‐centre training and – where possible – objective quantitative diagnostic image features so as to minimise exclusions (due to lack of visualisation) and interobserver variability.

Prospective recruitment of a consecutive series of participants, with double‐blinding between test interpretation and the reference standard diagnosis, and with pre‐specified and clearly defined diagnostic rules for determining the presence or absence of disease are all necessary and easy to achieve. Systematic follow‐up of non‐excised lesions avoids over‐reliance on a histological reference standard and allows results to be more generalisable to routine practice. A standardised approach to diagnosis and clear identification of the qualifications and level of observer training and experience required to achieve good results is also required. A multi‐centred approach would allow confirmation that results are replicable across centres and that the technology can be implemented across a health service. Any future research study needs to be clear about the diagnostic pathway followed by study participants prior to study enrolment, and reporting should conform to the updated Standards for Reporting of Diagnostic Accuracy (STARD) guideline (Bossuyt 2015).

What's new

Date	Event	Description
19 December 2018	Amended	Affiliations, Disclaimer and Sources of support updated

Appendices

Appendix 1. Current content and structure of the Programme Grant

	LIST OF REVIEWS	Number of studies
	Diagnosis of melanoma
1	Visual inspection	49
2	Dermoscopy +/‐ visual inspection	104
3	Teledermatology	22
4	Smartphone applications	2
5a	Computer‐assisted diagnosis – dermoscopy‐based techniques	42
5b	Computer‐assisted diagnosis – spectroscopy‐based techniques	Review amalgamated into 5a
6	Reflectance confocal microscopy	18
7	High‐frequency ultrasound	5
	Diagnosis of keratinocyte skin cancer (BCC and cSCC)
8	Visual inspection +/‐ Dermoscopy	24
5c	Computer‐assisted diagnosis – dermoscopy‐based techniques	Review amalgamated into 5a
5d	Computer‐assisted diagnosis – spectroscopy‐based techniques	Review amalgamated into 5a
9	Optical coherence tomography	5
10	Reflectance confocal microscopy	10
11	Exfoliative cytology	9
	Staging of melanoma
12	Imaging tests (ultrasound, CT, MRI, PET‐CT)	38
13	Sentinel lymph node biopsy	160
	Staging of cSCC
	Imaging tests review	Review dropped; only one study identified
13	Sentinel lymph node biopsy	Review amalgamated into 13 above (n = 15 studies)

Appendix 2. Glossary of terms

Term	Definition
Atypical intraepidermal melanocytic variant	Unusual area of darker pigmentation contained within the epidermis that may progress to an invasive melanoma; includes melanoma in situ and lentigo maligna
Atypical naevi	Unusual looking but non‐cancerous mole or area of darker pigmentation of the skin
BRAF V600 mutation	BRAF is a human gene that makes a protein called B‐Raf which is involved in the control of cell growth. BRAF mutations (damaged DNA) occur in around 40% of melanomas, which can then be treated with particular drugs.
BRAF inhibitors	Therapeutic agents that inhibit the serine‐threonine protein kinase BRAF mutated metastatic melanoma
Breslow thickness	A scale for measuring the thickness of melanomas by the pathologist using a microscope, measured in mm from the top layer of skin to the bottom of the tumour
Congenital naevi	A type of mole found on infants at birth
Dermoscopy	Whereby a handheld microscope is used to allow more detailed, magnified examination of the skin compared to examination by the naked eye alone
False negative	An individual who is truly positive for a disease, but whom a diagnostic test classifies as disease‐free
False positive	An individual who is truly disease‐free, but whom a diagnostic test classifies as having the disease
Histopathology/histology	The study of tissue, usually obtained by biopsy or excision, for example under a microscope
Incidence	The number of new cases of a disease in a given time period
Index test	A diagnostic test under evaluation in a primary study
Lentigo maligna	Unusual area of darker pigmentation contained within the epidermis which includes malignant cells but with no invasive growth. May progress to an invasive melanoma
Lymph node	Lymph nodes filter the lymphatic fluid (clear fluid containing white blood cells) that travels around the body to help fight disease; they are located throughout the body often in clusters (nodal basins)
Melanocytic naevus	An area of skin with darker pigmentation (or melanocytes) also referred to as 'moles'
Meta‐analysis	A form of statistical analysis used to synthesise results from a collection of individual studies
Metastases/metastatic disease	Spread of cancer away from the primary site to somewhere else through the bloodstream or the lymphatic system
Micrometastases	Micrometastases are metastases so small that they can only be seen under a microscope
Mitotic rate	Microscopic evaluation of number of cells actively dividing in a tumour
Morbidity	Detrimental effects on health
Mortality	Either the condition of being subject to death; or the death rate, which reflects the number of deaths per unit of population in relation to any specific region, age group, disease, treatment or other classification, usually expressed as deaths per 100, 1000, 10,000 or 100,000 people
Multidisciplinary team	A team with members from different healthcare professions and specialties (e.g. urology, oncology, pathology, radiology, and nursing). Cancer care in the National Health Service (NHS) uses this system to ensure that all relevant health professionals are engaged to discuss the best possible care for that patient.
Prevalence	The proportion of a population found to have a condition
Prognostic factors/indicators	Specific characteristics of a cancer or the person who has it which might affect the patient's prognosis
Receiver operating characteristic (ROC) plot	A plot of the sensitivity and 1 minus the specificity of a test at the different possible thresholds for test positivity; represents the diagnostic capability of a test with a range of binary test results
Receiver operating characteristic (ROC) analysis	The analysis of a ROC plot of a test to select an optimal threshold for test positivity
Recurrence	Recurrence is when new cancer cells are detected following treatment. This can occur either at the site of the original tumour or at other sites in the body
Reference standard	A test or combination of tests used to establish the final or 'true' diagnosis of a patient in an evaluation of a diagnostic test
Reflectance confocal microscopy (RCM)	A microscopic technique using infrared light (either in a handheld device or a static unit) that can create images of the deeper layers of the skin
Sensitivity	In this context the term is used to mean the proportion of individuals with a disease who have that disease correctly identified by the study test
Specificity	The proportion of individuals without the disease of interest (in this case with benign skin lesions) who have that absence of disease correctly identified by the study test
Staging	Clinical description of the size and spread of a patient's tumour, fitting into internationally agreed categories
Subclinical (disease)	Disease that is usually asymptomatic and not easily observable, e.g. by clinical or physical examination
Systemic treatment	Treatment, usually given by mouth or by injection, that reaches and affects cancer cells throughout the body rather than targeting one specific area

Appendix 3. Table of acronyms and abbreviations used

Acronym	Definition
μm	micrometre
B‐mode	brightness mode
BCC	basal cell carcinoma
BN	benign naevi
BPC	between‐person comparison (of tests)
CAD	computer‐assisted diagnosis
cSCC	cutaneous squamous cell carcinoma
DER	dermal echogenicity ratio
EEI	entry echo image
EEL	entry echo line
ELM	epiluminescence microscopy
GP	general practitioner
HFUS	high‐frequency ultrasound
KHz	kilohertz
LBI	lesional backscatter image
MHz	megahertz
mm	millimetre
PCPs	primary care providers
RCM	reflectance confocal microscopy
RCT	randomised controlled trial
RTI	reflex transmission image
SCC	squamous cell carcinoma
SD	standard deviation
US	ultrasound

Appendix 4. Proposed sources of heterogeneity

i. Population characteristics

• general versus higher risk populations

• patient population: Primary /secondary / specialist unit

• lesion suspicion: general suspicion/atypical/equivocal/NR

• lesion type: any pigmented; melanocytic

• inclusion of multiple lesions per participant

• ethnicity

ii. Index test characteristics

• the nature of and definition of criteria for test positivity

• observer experience with the index test

• approaches to lesion preparation (e.g. the use of oil or antiseptic gel for dermoscopy)

iii. Reference standard characteristics

• reference standard used

• whether histology‐reporting meets pathology‐reporting guidelines

• use of excisional versus diagnostic biopsy

• whether two independent dermatopathologists reviewed histological diagnosis

iv. Study quality

• consecutive or random sample of participants recruited

• index test interpreted blinded to the reference standard result

• index test interpreted blinded to the result of any other index test

• presence of partial or differential verification bias (whereby only a sample of those subject to the index test are verified by the reference test or by the same reference test with selection dependent on the index test result)

• use of an adequate reference standard

• overall risk of bias

Appendix 5. Final search strategies

Melanoma search strategies to August 2016

Database: Ovid MEDLINE(R) 1946 to August week 3 2016

Search strategy:

1 exp melanoma/

2 exp skin cancer/

3 exp basal cell carcinoma/

4 basalioma$1.ti,ab.

5 ((basal cell or skin) adj2 (cancer$1 or carcinoma$1 or mass or masses or tumour$1 or tumor$1 or neoplasm$1 or adenoma$1 or epithelioma$1 or lesion$1 or malignan$ or nodule$1)).ti,ab.

6 (pigmented adj2 (lesion$1 or mole$ or nevus or nevi or naevus or naevi or skin)).ti,ab.

7 (melanom$1 or nonmelanoma$1 or non‐melanoma$1 or melanocyt$ or non‐melanocyt$ or nonmelanocyt$ or keratinocyt$).ti,ab.

8 nmsc.ti,ab.

9 (squamous cell adj2 (cancer$1 or carcinoma$1 or mass or masses or tumor$1 or tumour$1 or neoplasm$1 or adenoma$1 or epithelioma$1 or epithelial or lesion$1 or malignan$ or nodule$1) adj2 (skin or epiderm$ or cutaneous)).ti,ab.

10 (BCC or CSCC or NMSC).ti,ab.

11 keratinocy$.ti,ab.

12 Keratinocytes/

13 or/1‐12

14 dermoscop$.ti,ab.

15 dermatoscop$.ti,ab.

16 photomicrograph$.ti,ab.

17 exp epiluminescence microscopy/

18 (epiluminescence adj2 microscop$).ti,ab.

19 (confocal adj2 microscop$).ti,ab.

20 (incident light adj2 microscop$).ti,ab.

21 (surface adj2 microscop$).ti,ab.

22 (visual adj (inspect$ or examin$)).ti,ab.

23 ((clinical or physical) adj examin$).ti,ab.

24 3 point.ti,ab.

25 three point.ti,ab.

26 pattern analys$.ti,ab.

27 ABCD$.ti,ab.

28 menzies.ti,ab.

29 7 point.ti,ab.

30 seven point.ti,ab.

31 (digital adj2 (dermoscop$ or dermatoscop$)).ti,ab.

32 artificial intelligence.ti,ab.

33 AI.ti,ab.

34 computer assisted.ti,ab.

35 computer aided.ti,ab.

36 neural network$.ti,ab.

37 exp diagnosis, computer‐assisted/

38 MoleMax.ti,ab.

39 image process$.ti,ab.

40 automatic classif$.ti,ab.

41 image analysis.ti,ab.

42 SIAscop$.ti,ab.

43 Aura.ti,ab.

44 (optical adj2 scan$).ti,ab.

45 MelaFind.ti,ab.

46 SIMSYS.ti,ab.

47 MoleMate.ti,ab.

48 SolarScan.ti,ab.

49 VivaScope.ti,ab.

50 (high adj3 ultraso$).ti,ab.

51 (canine adj2 detect$).ti,ab.

52 ((mobile or cell or cellular or smart) adj ((phone$1 adj2 app$1) or application$1)).ti,ab.

53 smartphone$.ti,ab.

54 (DermoScan or SkinVision or DermLink or SpotCheck).ti,ab.

55 Mole Detective.ti,ab.

56 Spot Check.ti,ab.

57 (mole$1 adj2 map$).ti,ab.

58 (total adj2 body).ti,ab.

59 exfoliative cytolog$.ti,ab.

60 digital analys$.ti,ab.

61 (image$1 adj3 software).ti,ab.

62 (teledermatolog$ or tele‐dermatolog$ or telederm or tele‐derm or teledermoscop$ or tele‐dermoscop$ or teledermatoscop$ or tele‐dermatoscop$).ti,ab.

63 (optical coherence adj (technolog$ or tomog$)).ti,ab.

64 (computer adj2 diagnos$).ti,ab.

65 exp sentinel lymph node biopsy/

66 (sentinel adj2 node).ti,ab.

67 nevisense.mp. or HFUS.ti,ab.

68 electrical impedance spectroscopy.ti,ab.

69 history taking.ti,ab.

70 patient history.ti,ab.

71 (naked eye adj (exam$ or assess$)).ti,ab.

72 (skin adj exam$).ti,ab.

73 physical examination/

74 ugly duckling.mp. or UD.ti,ab.

75 ((physician$ or clinical or physical) adj (exam$ or triage or recog$)).ti,ab.

76 ABCDE.mp. or VOC.ti,ab.

77 clinical accuracy.ti,ab.

78 Family Practice/ or Physicians, Family/ or clinical competence/

79 (confocal adj2 microscop$).ti,ab.

80 diagnostic algorithm$1.ti,ab.

81 checklist$.ti,ab.

82 virtual imag$1.ti,ab.

83 volatile organic compound$1.ti,ab.

84 dog$1.ti,ab.

85 gene expression analy$.ti,ab.

86 reflex transmission imag$.ti,ab.

87 thermal imaging.ti,ab.

88 elastography.ti,ab.

89 or/14‐88

90 (CT or PET).ti,ab.

91 PET‐CT.ti,ab.

92 (FDG or F18 or Fluorodeoxyglucose or radiopharmaceutical$).ti,ab.

93 exp Deoxyglucose/

94 deoxy‐glucose.ti,ab.

95 deoxyglucose.ti,ab.

96 CATSCAN.ti,ab.

97 exp Tomography, Emission‐Computed/

98 exp Tomography, X‐ray computed/

99 positron emission tomograph$.ti,ab.

100 exp magnetic resonance imaging/

101 (MRI or fMRI or NMRI or scintigraph$).ti,ab.

102 exp echography/

103 Doppler echography.ti,ab.

104 sonograph$.ti,ab.

105 ultraso$.ti,ab.

106 doppler.ti,ab.

107 magnetic resonance imag$.ti,ab.

108 or/90‐107

109 (stage$ or staging or metasta$ or recurrence or sensitivity or specificity or false negative$ or thickness$).ti,ab.

110 "Sensitivity and Specificity"/

111 exp cancer staging/

112 or/109‐111

113 108 and 112

114 89 or 113

115 13 and 114

Database: Ovid MEDLINE(R) In‐Process & Other Non‐Indexed Citations 29 August 2016

Search strategy:

1 basalioma$1.ti,ab.

2 ((basal cell or skin) adj2 (cancer$1 or carcinoma$1 or mass or masses or tumour$1 or tumor$1 or neoplasm$1 or adenoma$1 or epithelioma$1 or lesion$1 or malignan$ or nodule$1)).ti,ab.

3 (pigmented adj2 (lesion$1 or mole$ or nevus or nevi or naevus or naevi or skin)).ti,ab.

4 (melanom$1 or nonmelanoma$1 or non‐melanoma$1 or melanocyt$ or non‐melanocyt$ or nonmelanocyt$ or keratinocyt$).ti,ab.

5 nmsc.ti,ab.

6 (squamous cell adj2 (cancer$1 or carcinoma$1 or mass or masses or tumor$1 or tumour$1 or neoplasm$1 or adenoma$1 or epithelioma$1 or epithelial or lesion$1 or malignan$ or nodule$1) adj2 (skin or epiderm$ or cutaneous)).ti,ab.

7 (BCC or CSCC or NMSC).ti,ab.

8 keratinocy$.ti,ab.

9 or/1‐8

10 dermoscop$.ti,ab.

11 dermatoscop$.ti,ab.

12 photomicrograph$.ti,ab.

13 (epiluminescence adj2 microscop$).ti,ab.

14 (confocal adj2 microscop$).ti,ab.

15 (incident light adj2 microscop$).ti,ab.

16 (surface adj2 microscop$).ti,ab.

17 (visual adj (inspect$ or examin$)).ti,ab.

18 ((clinical or physical) adj examin$).ti,ab.

19 3 point.ti,ab.

20 three point.ti,ab.

21 pattern analys$.ti,ab.

22 ABCD$.ti,ab.

23 menzies.ti,ab.

24 7 point.ti,ab.

25 seven point.ti,ab.

26 (digital adj2 (dermoscop$ or dermatoscop$)).ti,ab.

27 artificial intelligence.ti,ab.

28 AI.ti,ab.

29 computer assisted.ti,ab.

30 computer aided.ti,ab.

31 neural network$.ti,ab.

32 MoleMax.ti,ab.

33 image process$.ti,ab.

34 automatic classif$.ti,ab.

35 image analysis.ti,ab.

36 SIAscop$.ti,ab.

37 Aura.ti,ab.

38 (optical adj2 scan$).ti,ab.

39 MelaFind.ti,ab.

40 SIMSYS.ti,ab.

41 MoleMate.ti,ab.

42 SolarScan.ti,ab.

43 VivaScope.ti,ab.

44 (high adj3 ultraso$).ti,ab.

45 (canine adj2 detect$).ti,ab.

46 ((mobile or cell or cellular or smart) adj ((phone$1 adj2 app$1) or application$1)).ti,ab.

47 smartphone$.ti,ab.

48 (DermoScan or SkinVision or DermLink or SpotCheck).ti,ab.

49 Mole Detective.ti,ab.

50 Spot Check.ti,ab.

51 (mole$1 adj2 map$).ti,ab.

52 (total adj2 body).ti,ab.

53 exfoliative cytolog$.ti,ab.

54 digital analys$.ti,ab.

55 (image$1 adj3 software).ti,ab.

56 (teledermatolog$ or tele‐dermatolog$ or telederm or tele‐derm or teledermoscop$ or tele‐dermoscop$ or teledermatoscop$ or tele‐dermatoscop$).ti,ab.

57 (optical coherence adj (technolog$ or tomog$)).ti,ab.

58 (computer adj2 diagnos$).ti,ab.

59 (sentinel adj2 node).ti,ab.

60 nevisense.mp. or HFUS.ti,ab.

61 electrical impedance spectroscopy.ti,ab.

62 history taking.ti,ab.

63 patient history.ti,ab.

64 (naked eye adj (exam$ or assess$)).ti,ab.

65 (skin adj exam$).ti,ab.

66 ugly duckling.mp. or UD.ti,ab.

67 ((physician$ or clinical or physical) adj (exam$ or triage or recog$)).ti,ab.

68 ABCDE.mp. or VOC.ti,ab.

69 clinical accuracy.ti,ab.

70 (Family adj (Practice or Physicians)).ti,ab.

71 (confocal adj2 microscop$).ti,ab.

72 clinical competence.ti,ab.

73 diagnostic algorithm$1.ti,ab.

74 checklist$.ti,ab.

75 virtual imag$1.ti,ab.

76 volatile organic compound$1.ti,ab.

77 dog$1.ti,ab.

78 gene expression analy$.ti,ab.

79 reflex transmission imag$.ti,ab.

80 thermal imaging.ti,ab.

81 elastography.ti,ab.

82 or/10‐81

83 (CT or PET).ti,ab.

84 PET‐CT.ti,ab.

85 (FDG or F18 or Fluorodeoxyglucose or radiopharmaceutical$).ti,ab.

86 deoxy‐glucose.ti,ab.

87 deoxyglucose.ti,ab.

88 CATSCAN.ti,ab.

89 positron emission tomograph$.ti,ab.

90 (MRI or fMRI or NMRI or scintigraph$).ti,ab.

91 Doppler echography.ti,ab.

92 sonograph$.ti,ab.

93 ultraso$.ti,ab.

94 doppler.ti,ab.

95 magnetic resonance imag$.ti,ab.

96 or/83‐95

97 (stage$ or staging or metasta$ or recurrence or sensitivity or specificity or false negative$ or thickness$).ti,ab.

98 96 and 97

99 82 or 98

100 9 and 99

Database: Embase 1974 to 29 August 2016

Search strategy:

1 *melanoma/

2 *skin cancer/

3 *basal cell carcinoma/

4 basalioma$.ti,ab.

5 ((basal cell or skin) adj2 (cancer$1 or carcinoma$1 or mass or masses or tumour$1 or tumor$1 or neoplasm$ or adenoma$ or epithelioma$ or lesion$ or malignan$ or nodule$)).ti,ab.

6 (pigmented adj2 (lesion$1 or mole$ or nevus or nevi or naevus or naevi or skin)).ti,ab.

7 (melanom$1 or nonmelanoma$1 or non‐melanoma$1 or melanocyt$ or non‐melanocyt$ or nonmelanocyt$ or keratinocyt$).ti,ab.

8 nmsc.ti,ab.

9 (squamous cell adj2 (cancer$1 or carcinoma$1 or mass or tumor$1 or tumour$1 or neoplasm$1 or adenoma$1 or epithelioma$1 or epithelial or lesion$1 or malignan$ or nodule$1) adj2 (skin or epiderm$ or cutaneous)).ti,ab.

10 (BCC or cscc).mp. or NMSC.ti,ab.

11 keratinocyte.ti,ab.

12 keratinocy$.ti,ab.

13 or/1‐12

14 dermoscop$.ti,ab.

15 dermatoscop$.ti,ab.

16 photomicrograph$.ti,ab.

17 *epiluminescence microscopy/

18 (epiluminescence adj2 microscop$).ti,ab.

19 (confocal adj2 microscop$).ti,ab.

20 (incident light adj2 microscop$).ti,ab.

21 (surface adj2 microscop$).ti,ab.

22 (visual adj (inspect$ or examin$)).ti,ab.

23 ((clinical or physical) adj examin$).ti,ab.

24 3 point.ti,ab.

25 three point.ti,ab.

26 pattern analys$.ti,ab.

27 ABCD$.ti,ab.

28 menzies.ti,ab.

29 7 point.ti,ab.

30 seven point.ti,ab.

31 (digital adj2 (dermoscop$ or dermatoscop$)).ti,ab.

32 artificial intelligence.ti,ab.

33 AI.ti,ab.

34 computer assisted.ti,ab.

35 computer aided.ti,ab.

36 neural network$.ti,ab.

37 MoleMax.ti,ab.

38 exp diagnosis, computer‐assisted/

39 image process$.ti,ab.

40 automatic classif$.ti,ab.

41 image analysis.ti,ab.

42 SIAscop$.ti,ab.

43 (optical adj2 scan$).ti,ab.

44 Aura.ti,ab.

45 MelaFind.ti,ab.

46 SIMSYS.ti,ab.

47 MoleMate.ti,ab.

48 SolarScan.ti,ab.

49 VivaScope.ti,ab.

50 confocal microscop$.ti,ab.

51 (high adj3 ultraso$).ti,ab.

52 (canine adj2 detect$).ti,ab.

53 ((mobile or cell$ or cellular or smart) adj ((phone$1 adj2 app$1) or application$1)).ti,ab.

54 smartphone$.ti,ab.

55 (DermoScan or SkinVision or DermLink or SpotCheck).ti,ab.

56 Spot Check.ti,ab.

57 Mole Detective.ti,ab.

58 (mole$1 adj2 map$).ti,ab.

59 (total adj2 body).ti,ab.

60 exfoliative cytolog$.ti,ab.

61 digital analys$.ti,ab.

62 (image$1 adj3 software).ti,ab.

63 (optical coherence adj (technolog$ or tomog$)).ti,ab.

64 (teledermatolog$ or tele‐dermatolog$ or telederm or tele‐derm or teledermoscop$ or tele‐dermoscop$ or teledermatoscop$).mp. or tele‐dermatoscop$.ti,ab.

65 (computer adj2 diagnos$).ti,ab.

66 *sentinel lymph node biopsy/

67 (sentinel adj2 node).ti,ab.

68 nevisense.ti,ab.

69 HFUS.ti,ab.

70 electrical impedance spectroscopy.ti,ab.

71 history taking.ti,ab.

72 patient history.ti,ab.

73 (naked eye adj (exam$ or assess$)).ti,ab.

74 (skin adj exam$).ti,ab.

75 *physical examination/

76 ugly duckling.ti,ab.

77 UD sign$.ti,ab.

78 ((physician$ or clinical or physical) adj (exam$ or recog$ or triage)).ti,ab.

79 ABCDE.ti,ab.

80 clinical accuracy.ti,ab.

81 *general practice/

82 (confocal adj2 microscop$).ti,ab.

83 clinical competence/

84 diagnostic algorithm$.ti,ab.

85 checklist$1.ti,ab.

86 virtual image$1.ti,ab.

87 volatile organic compound$1.ti,ab.

88 VOC.ti,ab.

89 dog$1.ti,ab.

90 gene expression analys$.ti,ab.

91 reflex transmission imaging.ti,ab.

92 thermal imaging.ti,ab.

93 elastography.ti,ab.

94 dog$1.ti,ab.

95 gene expression analys$.ti,ab.

96 reflex transmission imaging.ti,ab.

97 thermal imaging.ti,ab.

98 elastography.ti,ab.

99 or/14‐93

100 PET‐CT.ti,ab.

101 (CT or PET).ti,ab.

102 (FDG or F18 or Fluorodeoxyglucose or radiopharmaceutical$).ti,ab.

103 exp Deoxyglucose/

104 CATSCAN.ti,ab.

105 deoxyglucose.ti,ab.

106 deoxy‐glucose.ti,ab.

107 *positron emission tomography/

108 *computer assisted tomography/

109 positron emission tomograph$.ti,ab.

110 *nuclear magnetic resonance imaging/

111 (MRI or fMRI or NMRI or scintigraph$).ti,ab.

112 *echography/

113 Doppler.ti,ab.

114 sonograph$.ti,ab.

115 ultraso$.ti,ab.

116 magnetic resonance imag$.ti,ab.

117 or/100‐116

118 (stage$ or staging or metasta$ or recurrence or sensitivity or specificity or false negative$ or thickness$).ti,ab.

119 "Sensitivity and Specificity"/

120 *cancer staging/

121 or/118‐120

122 117 and 121

123 99 or 122

124 13 and 123

Database: Cochrane Library (Wiley) 2016 searched 30 August 2016 CDSR Issue 8 of 12 2016 CENTRAL Issue 7 of 12 2016 HTA Issue 3 of 4 July 2016 DARE Issue 3 of 4 2015

Search strategy:

#1 melanoma* or nonmelanoma* or non‐melanoma* or melanocyt* or non‐melanocyt* or nonmelanocyt* or keratinocyte*

#2 MeSH descriptor: [Melanoma] explode all trees

#3 "skin cancer*"

#4 MeSH descriptor: [Skin Neoplasms] explode all trees

#5 skin near/2 (cancer* or carcinoma* or mass or masses or tumour* or tumor* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*)

#6 nmsc

#7 "squamous cell" near/2 (cancer* or carcinoma* or mass or masses or tumour* or tumor* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*) near/2 (skin or epiderm* or cutaneous)

#8 "basal cell" near/2 (cancer* or carcinoma* or mass or masses or tumour* or tumor* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*)

#9 pigmented near/2 (lesion* or nevus or mole* or naevi or naevus or nevi or skin)

#10 #1 or #2 or #3 or #4 or #5 or #6 or #7 or #8 or #9

#11 dermoscop*

#12 dermatoscop*

#13 Photomicrograph*

#14 MeSH descriptor: [Dermoscopy] explode all trees

#15 confocal near/2 microscop*

#16 epiluminescence near/2 microscop*

#17 incident next light near/2 microscop*

#18 surface near/2 microscop*

#19 "visual inspect*"

#20 "visual exam*"

#21 (clinical or physical) next (exam*)

#22 "3 point"

#23 "three point"

#24 "pattern analys*"

#25 ABDC

#26 menzies

#27 "7 point"

#28 "seven point"

#29 digital near/2 (dermoscop* or dermatoscop*)

#30 "artificial intelligence"

#31 "AI"

#32 "computer assisted"

#33 "computer aided"

#34 AI

#35 "neural network*"

#36 MoleMax

#37 "computer diagnosis"

#38 "image process*"

#39 "automatic classif*"

#40 SIAscope

#41 "image analysis"

#42 "optical near/2 scan*"

#43 Aura

#44 MelaFind

#45 SIMSYS

#46 MoleMate

#47 SolarScan

#48 Vivascope

#49 "confocal microscopy"

#50 high near/3 ultraso*

#51 canine near/2 detect*

#52 Mole* near/2 map*

#53 total near/2 body

#54 mobile* or smart near/2 phone*

#55 cell next phone*

#56 smartphone*

#57 "mitotic index"

#58 DermoScan or SkinVision or DermLink or SpotCheck

#59 "Mole Detective"

#60 "Spot Check"

#61 mole* near/2 map*

#62 total near/2 body

#63 "exfoliative cytolog*"

#64 "digital analys*"

#65 image near/3 software

#66 teledermatolog* or tele‐dermatolog* or telederm or tele‐derm or teledermoscop* or tele‐dermoscop* or teledermatoscop* or tele‐dermatolog*

#67 "optical coherence" next (technolog* or tomog*)

#68 computer near/2 diagnos*

#69 sentinel near/2 node*

#70 #11 or #12 or #13 or #14 or #15 or #16 or #17 or #18 or #19 or #20 or #21 or #22 or #23 or #24 or #25 or #26 or #27 or #28 or #29 or #30 or #31 or #32 or #33 or #34 or #35 or #36 or #37 or #38 or #39 or #40 or #41 or #42 or #43 or #44 or #45 or #46 or #47 or #48 or #49 or #50 or #51 or #52 or #53 or #54 or #55 or #56 or #57 or #58 or #59 or #60 or #61 or #62 or #63 or #64 or #65 or #66 or #67 or #68 or #69

#71 ultraso*

#72 sonograph*

#73 MeSH descriptor: [Ultrasonography] explode all trees

#74 Doppler

#75 CT or PET or PET‐CT

#76 "CAT SCAN" or "CATSCAN"

#77 MeSH descriptor: [Positron‐Emission Tomography] explode all trees

#78 MeSH descriptor: [Tomography, X‐Ray Computed] explode all trees

#79 MRI

#80 MeSH descriptor: [Magnetic Resonance Imaging] explode all trees

#81 MRI or fMRI or NMRI or scintigraph*

#82 "magnetic resonance imag*"

#83 MeSH descriptor: [Deoxyglucose] explode all trees

#84 deoxyglucose or deoxy‐glucose

#85 "positron emission tomograph*"

#86 #71 or #72 or #73 or #74 or #75 or #76 or #77 or #78 or #79 or #80 or #81 or #82 or #83 or #84 or #85

#87 stage* or staging or metasta* or recurrence or sensitivity or specificity or "false negative*" or thickness*

#88 MeSH descriptor: [Neoplasm Staging] explode all trees

#89 #87 or #88

#90 #89 and #86

#91 #70 or #90

#92 #10 and #91

#93 BCC or CSCC or NMCS

#94 keratinocy*

#95 #93 or #94

#96 #10 or #95

#97 nevisense

#98 HFUS

#99 "electrical impedance spectroscopy"

#100 "history taking"

#101 "patient history"

#102 naked next eye near/1 (exam* or assess*)

#103 skin next exam*

#104 "ugly duckling" or (UD sign*)

#105 MeSH descriptor: [Physical Examination] explode all trees

#106 (physician* or clinical or physical) near/1 (exam* or recog* or triage*)

#107 ABCDE

#108 "clinical accuracy"

#109 MeSH descriptor: [General Practice] explode all trees

#110 confocal near microscop*

#111 "diagnostic algorithm*"

#112 MeSH descriptor: [Clinical Competence] explode all trees

#113 checklist*

#114 "virtual image*"

#115 "volatile organic compound*"

#116 dog or dogs

#117 VOC

#118 "gene expression analys*"

#119 "reflex transmission imaging"

#120 "thermal imaging"

#121 elastography

#122 #97 or #98 or #99 or #100 or #101 or #102 or #103 or #104 or #105 or #106 or #107 or #108 or #109 or #110 or #111 or #112 or #113 or #114 or #115 or #116 or #117 or #118 or #119 or #120 or #121

#123 #70 or #122

#124 #96 and #123

#125 #96 and #90

#126 #125 or #124

#127 #10 and #126

Database: CINAHL Plus (EBSCO) 1937 to 30 August 2016

Search strategy:

S1 (MH "Melanoma") OR (MH "Nevi and Melanomas+")

S2 (MH "Skin Neoplasms+")

S3 (MH "Carcinoma, Basal Cell+")

S4 basalioma*

S5 (basal cell) N2 (cancer* or carcinoma* or mass or masses or tumor* or tumour* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*)

S6 (pigmented) N2 (lesion* or mole* or nevus or nevi or naevus or naevi or skin)

S7 melanom* or nonmelanoma* or non‐melanoma* or melanocyt* or non‐melanocyt* or nonmelanocyt*

S8 nmsc

S9 TX BCC or cscc or NMSC

S10 (MH "Keratinocytes")

S11 keratinocyt*

S12 S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11

S13 dermoscop* or dermatoscop* or photomicrograph* or (3 point) or (three point) or ABCD* or menzies or (7 point) or (seven point) or AI or Molemax or SIASCOP* or Aura or MelaFind or SIMSYS or MoleMate or SolarScan or smartphone* or DermoScan or SkinVision or DermLink or SpotCheck

S14 (epiluminescence or confocal or incident or surface) N2 (microscop*)

S15 visual N1 (inspect* or examin*)

S16 (clinical or physical) N1 (examin*)

S17 pattern analys*

S18 (digital) N2 (dermoscop* or dermatoscop*)

S19 (artificial intelligence)

S20 (computer) N2 (assisted or aided)

S21 (neural network*)

S22 (MH "Diagnosis, Computer Assisted+")

S23 (image process*)

S24 (automatic classif*)

S25 (image analysis)

S26 SIAScop*

S27 (optical) N2 (scan*)

S28 (high) N3 (ultraso*)

S29 elastography

S30 (mobile or cell or cellular or smart) N2 (phone*) N2 (app or application*)

S31 (mole*) N2 (map*)

S32 total N2 body

S33 exfoliative cytolog*

S34 digital analys*

S35 image N3 software

S36 teledermatolog* or tele‐dermatolog* or telederm or tele‐derm or teledermoscop* or tele‐dermoscop* or teledermatoscop* or tele‐dermatoscop* teledermatolog* or tele‐dermatolog* or telederm or tele‐derm or teledermoscop*

S37 (optical coherence) N1 (technolog* or tomog*)

S38 computer N2 diagnos*

S39 sentinel N2 node

S40 (MH "Sentinel Lymph Node Biopsy")

S41 nevisense or HFUS or checklist* or VOC or dog*

S42 electrical impedance spectroscopy

S43 history taking

S44 "Patient history"

S45 naked eye

S46 skin exam*

S47 physical exam*

S48 ugly duckling

S49 UD sign*

S50 (physician* or clinical or physical) N1 (exam*)

S51 clinical accuracy

S52 general practice

S53 (physician* or clinical or physical) N1 (recog* or triage)

S54 confocal microscop*

S55 clinical competence

S56 diagnostic algorithm*

S57 checklist*

S58 virtual image*

S59 volatile organic compound*

S60 gene expression analys*

S61 reflex transmission imag*

S62 thermal imaging

S63 S13 or S14 or S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR S24 OR S25 OR S26 OR S27 OR S28 OR S29 OR S30 OR S31 OR S32 OR S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45 OR S46 OR S47 OR S48 OR S49 OR S50 OR S51 OR S52 OR S53 OR S54 OR S55 OR S56 OR S57 OR S58 OR S59 OR S60 OR S61 OR S62

S64 CT or PET

S65 PET‐CT

S66 FDG or F18 or Fluorodeoxyglucose or radiopharmaceutical*

S67 (MH "Deoxyglucose+")

S68 deoxy‐glucose or deoxyglucose

S69 CATSCAN

S70 CAT‐SCAN

S71 (MH "Deoxyglucose+")

S72 (MH "Tomography, Emission‐Computed+")

S73 (MH "Tomography, X‐Ray Computed")

S74 positron emission tomograph*

S75 (MH "Magnetic Resonance Imaging+")

S76 MRI or fMRI or NMRI or scintigraph*

S77 echography

S78 doppler

S79 sonograph*

S80 ultraso*

S81 magnetic resonance imag*

S82 S64 OR S65 OR S66 OR S67 OR S68 OR S69 OR S70 OR S71 OR S72 OR S73 OR S74 OR S75 OR S76 OR S77 OR S78 OR S79 OR S80 OR S81

S83 stage* or staging or metasta* or recurrence or sensitivity or specificity or (false negative*) or thickness

S84 (MH "Neoplasm Staging")

S85 S83 OR S84

S86 S82 AND S85

S87 S63 OR S86

S88 S12 AND S87

Database: Science Citation Index SCI Expanded (Web of Science) 1900 to 30 August 2016

Conference Proceedings Citation Index (Web of Science) 1900 to 1 September 2016

Search strategy:

#1 (melanom* or nonmelanom* or non‐melanoma* or melanocyt* or non‐melanocyt* or nonmelanocyt* or keratinocyt*)

#2 (basalioma*)

#3 ((skin) near/2 (cancer* or carcinoma or mass or masses or tumour* or tumor* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*))

#4 ((basal) near/2 (cancer* or carcinoma* or mass or masses or tumour* or tumor* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*))

#5 ((pigmented) near/2 (lesion* or mole* or nevus or nevi or naevus or naevi or skin))

#6 (nmsc or BCC or NMSC or keratinocy*)

#7 ((squamous cell (cancer* or carcinoma* or mass or masses or tumour* or tumor* or neoplasm* or adenoma* or epithelioma* or lesion* or malignan* or nodule*))

#8 (skin or epiderm* or cutaneous)

#9 #8 AND #7

#10 #9 OR #6 OR #5 OR #4 OR #3 OR #2 OR #1

#11 ((dermoscop* or dermatoscop* or photomicrograph* or epiluminescence or confocal or "incident light" or "surface microscop*" or "visual inspect*" or "physical exam*" or 3 point or three point or pattern analy* or ABCDE or menzies or 7 point or seven point or dermoscop* or dermatoscop* or AI or artificial or computer aided or computer assisted or neural network* or Molemax or image process* or automatic classif* or image analysis or siascope or optical scan* or Aura or melafind or simsys or molemate or solarscan or vivascope or confocal microscop* or high ultraso* or canine detect* or cellphone* or mobile* or phone* or smartphone or dermoscan or skinvision or dermlink or spotcheck or spot check or mole detective or mole map* or total body or exfoliative psychology or digital or image software or optical coherence or teledermatology or telederm* or teledermoscop* or teledermatoscop* or computer diagnos* or sentinel))

#12 ((nevisense or HFUS or impedance spectroscopy or history taking or patient history or naked eye or skin exam* or physical exam* or ugly duckling or UD sign* or physician* exam* or physical exam* or ABCDE or clinical accuracy or general practice or confocal microscop* or clinical competence or diagnostic algorithm* or checklist* or virtual image* or volatile organic or VOC or dog* or gene expression or reflex transmission or thermal imag* or elastography))

#13 #11 or #12

#14 ((PET or CT or FDG or deoxyglucose or deoxy‐glucose or fluorodeoxy* or radiopharma* or CATSCAN or positron emission or computer assisted or nuclear magnetic or MRI or FMRI or NMRI or scintigraph* or echograph* or Doppler or sonograph* or ultraso* or magnetic reson*))

#15 ((stage* or staging or metast* or recurrence or sensitivity or specificity or false negative* or thickness*))

#16 #14 AND #15

#17 #16 OR #13

#18 #10 AND #17

Refined by: DOCUMENT TYPES: (MEETING ABSTRACT OR PROCEEDINGS PAPER)

Appendix 6. Full text inclusion criteria

Criterion	Inclusion	Exclusion
Study design	For diagnostic and staging reviews Any study for which a 2×2 contingency table can be extracted, e.g. diagnostic case control studies 'cross‐sectional' test accuracy study with retrospective or prospective data collection studies where estimation of test accuracy was not the primary objective but test results for both index and reference standard were available RCTs of tests or testing strategies where participants were randomised between index tests and all undergo a reference standard (i.e. accuracy RCTs)	< 5 melanoma cases (diagnosis reviews) < 10 participants (staging reviews) Studies developing new criteria for diagnosis unless a separate 'test set' of images were used to evaluate the criteria (mainly digital dermoscopy) Studies using 'normal' skin as controls Letters, editorials, comment papers, narrative reviews Insufficient data to construct a 2×2 table
Target condition	Melanoma Keratinocyte skin cancer (or non‐melanoma skin cancer) BCC or epithelioma cSCC	Studies exclusively conducted in children Studies of non‐cutaneous melanoma or SCC
Population	For diagnostic reviews Adults with a skin lesion suspicious for melanoma, BCC, or cSCC (other terms include pigmented skin lesion/nevi, melanocytic, keratinocyte, etc.) Adults at high risk of developing melanoma skin cancer, BCC, or cSCC For staging reviews Adults with a diagnosis of melanoma or cSCC undergoing tests for staging of lymph nodes or distant metastases or both	People suspected of other forms of skin cancer Studies conducted exclusively in children
Index tests	For diagnosis Visual inspection/clinical examination Dermoscopy/dermatoscopy Teledermoscpoy Smartphone/mobile phone applications Digital dermoscopy/artificial intelligence Confocal microscopy Ocular coherence tomography Exfoliative cytology High‐frequency ultrasound Canine odour detection DNA expression analysis/gene chip analysis Other For staging CT PET PET‐CT MRI Ultrasound +/fine needle aspiration cytology FNAC SLNB +/high‐frequency ultrasound Other Any test combination and in any order Any test positivity threshold Any variation in testing procedure (e.g. radioisotope used)	Sentinel lymph biopsy for therapeutic rather than staging purposes Tests to determine melanoma thickness Tests to determine surgical margins/lesion borders Tests to improve histopathology diagnose LND
Reference standard	For diagnostic studies Histopathology of the excised lesion Clinical follow‐up of non‐excised/benign appearing lesions with later histopathology if suspicious Expert diagnosis (studies should not be included if expert diagnosis is the sole reference standard) For studies of imaging tests for staging Histopathology (via LND or SLMB) Clinical/radiological follow‐up A combination of the above For studies of SLNB accuracy for staging LND of both SLN+ and SLn participants to identify all diseased nodes LND of SLN+ participants and follow‐up of SLN participants to identify a subsequent nodal recurrence in a previously investigated nodal basin	For diagnostic studies Exclude if any disease positive participants have diagnosis unconfirmed by histology Exclude if > 50% of disease negative participants have diagnosis confirmed by expert opinion with no histology or follow‐up Exclude studies of referral accuracy, i.e. comparing referral decision with expert diagnosis, unless evaluations of teledermatology or mobile phone applications
BCC: basal cell carcinoma; cSCC: cutaneous squamous cell carcinoma; CT: computed tomography; FNAC: fine needle aspiration cytology; LND: lymph node dissection; MRI: magnetic resonance imaging; PET: positron emission tomography; PET‐CT: positron emission tomography computed tomography; RCT: randomised controlled trial; SCC: squamous cell carcinoma; SLN+: positive sentinel lymph node; SLn: negative sentinel lymph node; SLNB: sentinel lymph node biopsy.

Appendix 7. Quality assessment (based on QUADAS‐2)

The QUADAS‐2 checklist (Whiting 2011) was tailored to the review topic as follows below.

Patient selection domain (1)

Selective recruitment of study participants can be a key influence on test accuracy. In general terms, all participants eligible to undergo a test should be included in a study, allowing for the intended use of that test within the context of the study. We considered studies that separately sampled malignant and benign lesions to have used a case‐control design; and those that supplemented a series of suspicious lesions with additional malignant or benign lesions to be at unclear risk of bias.

In terms of exclusions, we considered studies that excluded particular lesion types (e.g. lentigo maligna), particular lesion sites, or other lesions on the basis of image quality or lack of observer agreement (e.g. on histopathology) to be at high risk of bias.

In judging the applicability of patient populations to the review question, we considered restriction to particular lesion populations, such as melanocytic, nodular, high risk or restrictions by size to be of high concern for applicability.

Given that diagnosis of skin cancer is primarily lesion‐based, there is the potential for study participants with multiple lesions to contribute disproportionately to estimates of test accuracy, especially if they are at particular risk of having skin cancer. We considered studies that include a high number of lesions in relation to the number of participants in the study to be less representative than studies conducted in a more general population of participants (i.e. if the difference between the number of included lesions and number of included participants is greater than 5%).

Index test domain (2)

Given the potential for subjective differences in test interpretation for melanoma, the interpretation of the index test blinded to the result of the reference standard is a key means of reducing bias. For prospective studies and retrospective studies that used the original index test interpretation, the diagnosis will by nature be interpreted and recorded before the result of the reference standard is known; however, studies using previously acquired images could be particularly susceptible to information bias. For these studies to be at low risk of bias, we required a clear indication that observers were unaware of the reference standard diagnosis at time of test interpretation. We also added an item to assess the presence of blinding between interpretations of different algorithms; however, we did not include this item in the overall assessment of risk of bias.

Pre‐specification of the index test threshold was considered present if the study clearly reported that the threshold used was not data driven, i.e. was not based on study results. Studies that did not clearly describe the threshold used but that required clinicians to record a diagnosis or management decision for a lesion were considered to be unclear on this criterion. Studies reporting accuracy for multiple numeric thresholds, where ROC analysis was used to select the threshold, or that reported accuracy for the presence of independently significant lesion characteristics with no separate test set of lesions were considered at high risk of bias.

In terms of applicability of the index test to the review question, we required the test to be applied and interpreted as it would be in a clinical practice setting, i.e. in‐person or face‐to‐face with the patient, and by a single observer as opposed to a consensus decision or average across multiple observers. Image‐based studies were considered to be of high concern, although for some tests (e.g. RCM) image interpretations where the observer was also supplied with a clinical or dermoscopic image of the lesion along with some patient characteristics were considered 'unclear'.

Despite the often subjective nature of test interpretation, it is also important for study authors to outline the particular lesion characteristics that were considered to be indicative of melanoma, particularly where established algorithms or checklists were not used. Studies were considered of low concern if the threshold used was established in a prior study or sufficient threshold details were presented to allow replication.

The experience of the examiner will also impact on the applicability of study results. We required studies to describe the test interpreter as 'experienced' or 'expert' to have low concern about applicability.

Reference standard domain (3)

In an ideal study, consecutively recruited participants should all undergo incisional or excisional biopsy of the skin lesion regardless of level of clinical suspicion of melanoma. In reality, both partial and differential verification bias are likely. Partial verification bias may occur where histology is the only reference standard used, and only those participants with a certain degree of suspicion of malignancy based on the result of the index test undergo verification, the others either being excluded from the study or defined as being disease‐negative without further assessment or follow‐up, as discussed above.

Differential verification bias will be present where other reference standards are used in addition to histological verification of suspicious lesions. A typical example of verification bias in skin cancer occurs when investigators do not biopsy people with benign‐appearing lesions but instead follow them up for a period of time to determine whether any malignancy subsequently develops (these would be false‐negatives on the index test). We defined an 'adequate' reference standard as: all disease‐positive individuals having a histological reference standard either at the time of application of the index test or after a period of clinical follow‐up; and at least 80% of disease‐negative participants have received a histological diagnosis, with up to 20% undergoing at least three months' follow‐up of benign‐appearing lesions.

A further challenge is the potential for incorporation bias, i.e. where the result of the index test is used to help determine the reference standard diagnosis. It is normal practice for the clinical diagnosis (usually by visual inspection or dermoscopy) to be included on pathology request forms and for the histopathologist to use this diagnosis to help with the pathology interpretation. Although inclusion of such clinical information on the histopathology request form is theoretically a form of incorporation bias, blinded interpretation of the histopathology reference standard is not normal practice, and enforcement of such conditions would significantly limit the generalisability of the study results. For studies evaluating tests other than visual inspection or dermoscopy, this item was divided into two questions, firstly whether the reference standard was blinded to the index test result, and secondly whether it was blinded to the clinical diagnosis. Only the response to the first part (i.e. blinding to index test) was included in our overall assessment of risk of bias for the reference standard domain.

In judging the applicability of the reference standard to our review question, scored studies as high concern around applicability if they used expert diagnosis (with no follow‐up) as a reference standard in any patient, or did not report histology interpretation by a dermatopathologist.

Flow and timing domain (4)

In the ideal study, the diagnosis based on the index test and reference standard should be made consecutively or as near to each other in time as possible to avoid changes in lesion over time. For lesions with a histological reference standard, we have defined a one‐month period as an appropriate interval between application of the index test and the reference standard. For studies using clinical follow‐up, a minimum three‐month follow‐up period has been defined as at low risk of bias for detecting false‐negatives. This interval was chosen based on a study showing that most false‐negative melanomas will be diagnosed within three months of the initial negative index test although a small number will be diagnosed up to 12 months subsequently (Altamura 2008).

In assessing whether all patients were included in the analysis, we considered studies at high risk of bias if participants were excluded following recruitment.

Comparative domain

A comparative domain was added to the QUADAS‐2 checklist for studies comparing the accuracy of RCM and dermoscopy. Items were included to assess the presence blinding of interpretation between tests, and to specify a maximum of one month interval between application of index tests, as intervals greater than these may be accompanied by changes in tumour characteristics. As it would not be normal practice for RCM to be interpreted blinded to the clinical or dermoscopic diagnosis, the scoring of this item did not contribute to our overall assessment of risk of bias. We also considered whether both tests were applied and interpreted in a clinically applicable manner.

The following tables use text that was originally published in the QUADAS‐2 tool by Whiting and colleagues (Whiting 2011).

Item	Response (delete as required)
PARTICIPANT SELECTION (1) ‐ RISK OF BIAS
1) Was a consecutive or random sample of participants or images enrolled?	Yes – if paper states consecutive or random No – if paper describes other method of sampling Unclear – if participant sampling not described
2) Was a case‐control design avoided?	Yes – if consecutive or random or case‐control design clearly not used No – if study described as case‐control or describes sampling specific numbers of participants with particular diagnoses Unclear – if not described
3) Did the study avoid inappropriate exclusions, e.g. 'difficult to diagnose' lesions not excluded lesions not excluded on basis of disagreement between evaluators	Yes ‐ if inappropriate exclusions were avoided No – if lesions were excluded that might affect test accuracy, e.g. 'difficult to diagnose' lesions, or where disagreement between evaluators was observed Unclear – if not clearly reported but there is suspicion that difficult to diagnose lesions may have been excluded
4) For between‐person comparative studies only (i.e. allocating different tests to different study participants): A) were the same participant selection criteria used for those allocated to each test? B) was the potential for biased allocation between tests avoided through adequate generation of a randomised sequence? C) was the potential for biased allocation between tests avoided through concealment of allocation prior to assignment?	For A) Yes – if same selection criteria were used for each index test, No – if different selection criteria were used for each index test, Unclear – if selection criteria per test were not described, NA – if only 1 index test was evaluated or all participants received all tests For B) Yes – if adequate randomisation procedures are described, No – if inadequate randomisation procedures are described, Unclear – if the method of allocation to groups is not described (a description of 'random' or 'randomised' is insufficient), NA – if only 1 index test was evaluated or all participants received all tests For C) Yes – if appropriate methods of allocation concealment are described, No – if appropriate methods of allocation concealment are not described, Unclear – if the method of allocation concealment is not described (sufficient detail to allow a definite judgement is required), NA – if only 1 index test was evaluated
Could the selection of participants have introduced bias? For non‐comparative and within person‐comparative studies If answers to all of questions 1), 2), and 3) 'Yes': If answers to any 1 of questions 1), 2), or 3) 'No': If answers to any 1 of questions 1), 2), or 3) 'Unclear': For between‐person comparative studies If answers to all of questions 1), 2), 3), and 4) 'Yes': If answers to any 1 of questions 1), 2), 3), or 4) 'No': If answers to any 1 of questions 1), 2), 3), or 4) 'Unclear':	For non‐comparative and within person‐comparative studies Risk is low Risk is high Risk unclear For between‐person comparative studies Risk is low Risk is high Risk unclear
PARTICIPANT SELECTION (1) ‐ CONCERNS REGARDING APPLICABILITY
1) Are the included participants and chosen study setting appropriate to answer the review question, i.e. are the study results generalisable? This item is not asking whether exclusion of certain participant groups might bias the study's results (as in 'Risk of bias' above), but is asking whether the chosen study participants and setting are appropriate to answer our review question. Because we are looking to establish test accuracy in both primary presentation and referred participants, a study could be appropriate for 1 setting and not for the other, or it could be unclear as to whether the study can appropriately answer either question For each study assessed, please consider whether it is more relevant for A) participants with a primary presentation of a skin lesion or B) referred participants, and respond to the questions in either A) or B) accordingly. If the study gives insufficient details, please respond Unclear to both parts of the question	A) For studies that will contribute to the analysis of participants with a primary presentation of a skin lesion (i.e. test naive) Yes – if participants included in the study appear to be generally representative of those who might present in a usual practice setting No – if study participants appear to be unrepresentative of usual practice, e.g. in terms of severity of disease, demographic features, presence of differential diagnosis or comorbidity, setting of the study, and previous testing protocols Unclear – if insufficient details are provided to determine the generalisability of study participants B) For studies that will contribute to the analysis of referred participants (i.e. who have already undergone some form of testing) Yes – if study participants appear to be representative of those who might be referred for further investigation. If the study focuses only on those with equivocal lesions, for example, we would suggest that this is not representative of the wider referred population No – if study participants appear to be unrepresentative of usual practice, e.g. if a particularly high proportion of participants have been self‐referred or referred for cosmetic reasons. Other factors to consider include severity of disease, demographic features, presence of differential diagnosis or comorbidity, setting of the study, and previous testing protocols Unclear – if insufficient details are provided to determine the generalisability of study participants
2) Did the study avoid including participants with multiple lesions?	Yes – if the difference between the number of included lesions and number of included participants is less than 5% No – if the difference between the number of included lesions and number of included participants is greater than 5% Unclear – if it is not possible to assess
Is there concern that the included participants do not match the review question? If the answer to question 1) or 2) 'Yes': If the answer to question 1) or 2) 'No': If the answer to question 1) or 2) 'Unclear':	Concern is low Concern is high Concern is unclear
INDEX TEST (2) ‐ RISK OF BIAS (to be completed per test evaluated)
1) Was the index test or testing strategy result interpreted without knowledge of the results of the reference standard?	Yes – if index test described as interpreted without knowledge of reference standard result or, for prospective studies, if index test is always conducted and interpreted prior to the reference standard No – if index test described as interpreted in knowledge of reference standard result Unclear – if index test blinding is not described
2) Was the diagnostic threshold at which the test was considered positive (i.e. melanoma present) prespecified?	Yes – if threshold was prespecified (i.e. prior to analysing study results) No – if threshold was not prespecified Unclear – if not possible to tell whether or not diagnostic threshold was prespecified
3) For within‐person comparisons of index tests or testing strategies (i.e. > 1 index test applied per participant): was each index test result interpreted without knowledge of the results of other index tests or testing strategies?	Yes – if all index tests were described as interpreted without knowledge of the results of the others No – if the index tests were described as interpreted in the knowledge of the results of the others Unclear – if it is not possible to tell whether knowledge of other index tests could have influenced test interpretation NA – if only 1 index test was evaluated
Could the conduct or interpretation of the index test have introduced bias? For non‐comparative and between‐person comparison studies If answers to questions 1) and 2) 'Yes': If answers to either questions 1) or 2) 'No': If answers to either questions 1) or 2) 'Unclear': For within‐person comparative studies If answers to all questions 1), 2), for any index test and 3) 'Yes': If answers to any 1 of questions 1) or 2) for any index test or 3) 'No': If answers to any 1 of questions 1) or 2) for any index test or 3) 'Unclear':	For non‐comparative and between‐person comparison studies Risk is low Risk is high Risk is unclear For within‐person comparative studies Risk is low Risk is high Risk is unclear
INDEX TEST (2) ‐ CONCERN ABOUT APPLICABILITY
1) Was the diagnostic threshold to determine presence or absence of disease established in a previously published study? E.g. previously evaluated/established algorithm/checklist used lesion characteristics indicative of melanoma used objective (usually numerical) threshold used	Yes – if a previously evaluated/established tool to aid diagnosis of melanoma was used or if the diagnostic threshold used was established in a previously published study No – if an unfamiliar/new tool to aid diagnosis of melanoma was used, if no particular algorithm was used, or if the objective threshold reported was chosen based on results in the current study Unclear – if insufficient information was reported
2) Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication? Study results can only be reproduced if the diagnostic threshold is described in sufficient detail. This item applies equally to studies using pattern recognition and those using checklists or algorithms to aid test interpretation	Yes – If the criteria for diagnosis of melanoma were reported in sufficient detail to allow replication No – if the criteria for diagnosis of melanoma were not reported in sufficient detail to allow replication Unclear – If some but not sufficient information on criteria for diagnosis to allow replication were provided
3) Was the test interpretation carried out by an experienced examiner?	Yes – if the test was interpreted by 1 or more speciality‐accredited dermatologists, or by examiners of any clinical background with special interest in dermatology and with any formal training in the use of the test No – if the test was not interpreted by an experienced examiner (see above) Unclear – if the experience of the examiner(s) was not reported in sufficient detail to judge or if examiners were described as 'Expert' with no further detail given NA – if system‐based diagnosis, i.e. no observer interpretation
Is there concern that the index test, its conduct, or interpretation differ from the review question? If answers to questions 1), 2), and 3) 'Yes': If answers to questions 1), 2), or 3) 'No': If answers to questions 1), 2), or 3) 'Unclear':	Concern is low Concern is high Concern is unclear
REFERENCE STANDARD (3) ‐ RISK OF BIAS
1) Is the reference standard likely to correctly classify the target condition? A) Disease‐positive – 1 or more of the following: histological confirmation of melanoma following biopsy or lesion excision clinical follow‐up of benign‐appearing lesions for at least 3 months following the application of the index test, leading to a histological diagnosis of melanoma B) Disease‐negative – 1 or more of the following: histological confirmation of absence of melanoma following biopsy or lesion excision in at least 80% of disease‐negative participants clinical follow‐up of benign‐appearing lesions for a minimum of 3 months following the index test in up to 20% of disease‐negative participants	A) Disease‐positive Yes – if all participants with a final diagnosis of melanoma underwent 1 of the listed reference standards No – If a final diagnosis of melanoma for any participant was reached without histopathology Unclear – if the method of final diagnosis was not reported for any participant with a final diagnosis of melanoma or if the length of clinical follow‐up used was not clear or if a clinical follow‐up reference standard was reported in combination with a participant‐based analysis and it was not possible to determine whether the detection of a malignant lesion during follow‐up is the same lesion that originally tested negative on the index test B) Disease‐negative Yes – If at least 80% of benign diagnoses were reached by histology and up to 20% were reached by clinical follow‐up for a minimum of 3 months following the index test No – if more than 20% of benign diagnoses were reached by clinical follow‐up for a minimum of 3 months following the index test or if clinical follow‐up period was less than 3 months Unclear – if the method of final diagnosis was not reported for any participant with benign or non‐melanoma diagnosis
2) Were the reference standard results interpreted without knowledge of the results of the index test? Please score this item for all studies even though histopathology interpretation is usually conducted with knowledge of the clinical diagnosis (from visual inspection or dermoscopy or both). We will deal with this by not including the response to this item in the 'Risk of bias' assessment for these tests. For reviews of all other tests, this item will be retained	Yes – if the reference standard diagnosis was reached blinded to the index test result No – if the reference standard diagnosis was reached with knowledge of the index test result Unclear – if blinded reference test interpretation was not clearly reported
Could the reference standard, its conduct, or its interpretation have introduced bias? For visual inspection/dermoscopy evaluations If answer to question 1) 'Yes': If answer to question 1) 'No': If answer to question 1) 'Unclear': For all other tests If answers to questions 1) and 2) 'Yes': If answers to questions 1) or 2) 'No': If answers to questions 1) or 2) 'Unclear':	For visual inspection/dermoscopy evaluations Risk is low Risk is high Risk is unclear For all other tests Risk is low Risk is high Risk is unclear
REFERENCE STANDARD (3) ‐ CONCERN ABOUT APPLICABILITY
1) Are index test results presented separately for each component of the target condition (i.e. separate results presented for those with invasive melanoma, melanoma in situ, lentigo maligna, severe dysplasia, BCC, and cSCC)?	Yes – if index test results for each component of the target condition can be disaggregated No – if index test results for the different components of the target condition cannot be disaggregated Unclear – if not clearly reported
2) Expert opinion (with no histological confirmation) was not used as a reference standard 'Expert opinion' means diagnosis based on the standard clinical examination, with no histology or lesion follow‐up ***do not complete this item for teledermatology studies	Yes – if expert opinion was not used as a reference standard for any participant No – if expert opinion was used as a reference standard for any participant Unclear – if not clearly reported
3) Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes – if histology interpretation was reported to be carried out by an experienced histopathologist or dermatopathologist No – if histology interpretation was reported to be carried out by a less experienced histopathologist Unclear – if the experience/qualifications of the pathologist were not reported
Is there concern that the target condition as defined by the reference standard does not match the review question? If answers to all questions 1), 2), and 3) 'Yes': If answers to any 1 of questions 1), 2), or 3) 'No': If answers to any 1 of questions 1), 2), or 3) 'Unclear': ***For teledermatology studies only If answers to all questions 1) and 3) 'Yes': If answers to questions 1) or 3) 'No': If answers to questions 1) or 3) 'Unclear':	Concern is low Concern is high Concern is unclear ***For teledermatology studies only Concern is low Concern is high Concern is unclear
FLOW AND TIMING (4): RISK OF BIAS
1) Was there an appropriate interval between index test and reference standard? A) For histopathological reference standard, was the interval between index test and reference standard ≤ 1 month? B) If the reference standard includes clinical follow‐up of borderline/benign‐appearing lesions, was there at least 3 months' follow‐up following application of index test(s)?	A) Yes – if study reports ≤ 1 month between index and reference standard No – if study reports > 1 month between index and reference standard Unclear – if study does not report interval between index and reference standard B) Yes – if study reports ≥ 3 months' follow‐up No – if study reports < 3 months' follow‐up Unclear – if study does not report the length of clinical follow‐up
2) Did all participants receive the same reference standard?	Yes – if all participants underwent the same reference standard No – if more than 1 reference standard was used Unclear – if not clearly reported
3) Were all participants included in the analysis?	Yes – if all participants were included in the analysis No – if some participants were excluded from the analysis Unclear– if not clearly reported
4) For within‐person comparisons of index tests Was the interval between application of index tests ≤ 1 month?	Yes – if study reports ≤ 1 month between index tests No – if study reports > 1 month between index tests Unclear – if study does not report the interval between index tests
Could the participant flow have introduced bias? For non‐comparative and between‐person comparison studies If answers to questions 1), 2), and 3) 'Yes': If answers to any 1 of questions 1), 2), or 3) 'No': If answers to any 1 of questions 1), 2), or 3) 'Unclear': For within‐person comparative studies If answers to all questions 1), 2), 3), and 4) 'Yes': If answers to any 1 of questions 1), 2), 3), or 4) 'No': If answers to any 1 of questions 1), 2), 3), or 4) 'Unclear':	For non‐comparative and between‐person comparison studies Risk is low Risk is high Risk is unclear For within‐person comparative studies Risk is low Risk is high Risk is unclear
BCC: basal cell carcinoma; cSCC: cutaneous squamous cell carcinoma; NA: not applicable.

Appendix 8. Summary study details

Study author Outcomes reported	Study type	Inclusion criteria	Number of patients (lesions)	US machine	Frequency Resolution	Threshold	Observer qualification (number) and experience	Reference standard Final diagnoses	Exclusions
Bessoud 2003 MEL	WPC P‐CS France Secondary	Patients with PSLs referred from the Dermatology Department to the Ultrasound Unit	111 (130)	AU 4 or AU 5 Idea (Esaote‐Biomedica, Genoa, Italy) Doppler	20‐MHz axial 80 µm; lateral 100 µm 7 MHz	1. Hypoechoic, homogenous and well defined HFUS + Doppler As above AND intra‐lesional vessels present	Not described	Histology MM 65; 4 BCC; 1 neurosarcoma BN 33; SK 5; Other 6 (3 melanosis, 1 thrombosing capillaritis, 1 histiocytofibroma, 1 lentigo)	16 'unseen' on ultrasound (16/130=12%); 5 melanoma; 11 benign nevi (including 1 lentigo) Further 8 lesions not imaged with Doppler (basis for selection NR)
Clement 2001 BCC	WPC P‐CS France Secondary	Patients with PSLs including melanocytic and non melanocytic examined before resection (recruited 1998‐1999)	160 (176)	AU4 then AU5 Idea, (Esaote Biomedica, Genoa, ltaly) Doppler	20 MHz 80 μm axial; 100 μm lateral 7 MHz	1. Hypoechoic 2. Hypoechoic and homogenous 3. Hypoechoic and well defined HFUS + Doppler Hypoechoic AND vascularity present	Not described	Histology MM 1; BCC 23; Mel metastases 6; BN 6; SK 29; histiocytofibroma 11; angioma 7	38 not visualised, including two that were difficult to reach (38/176 = 21.6%); 5 melanoma (all in the horizontal growth phase; 33 benign nevus)
Dummer 1995 MEL BCC	NC NR‐CS Germany Secondary	Patients with PSLs referred from the outpatient clinic to the Department of Dermatology	NR (92)	DUB 20, Taberna pro Medicum, Luneburg, Germany	20 MHz axial 80 μm; lateral 200 μm	1. Hypoechoic 2. Hypoechoic and homogenous 3. Hypoechoic and well defined (1) Sharp basal margins; 2) Sharp lateral margins)	Not described	Histology MM 108; BCC 65 BN 307; SK 211; DF 54; angioma 47	None reported
Harland 2000	NC CCS UK Specialist	Patients with PSLs with specific presumptive clinical diagnoses (SK, BN, MM) from a PSL clinic; the referring GP had considered the diagnosis of melanoma for each lesion.	NR (54)	Dermascan‐CTM B‐scanner (Cortex Technology, ApS, Hadsund, Denmark)	20 MHz axial 50 μm; lateral 300 μm	1. Dermal echogenicity ratio (DER) < 3 (to ensure sensitivity of 100%) 2. Absence of entry echo line (EEL) ‐ (either equivalent to perilesional skin (non‐enhanced) or as broadened); 3. Either 1. Or 2.	Not described	Histology MM 19; MiS 6 BN 15; SK 29	BN excluded from sensitivity and specificity estimates
Lassau 1997 MEL BCC	WPC P‐CS France Secondary	Patients with skin lesions clinically suspected of being either melanoma or BCC and scheduled for resection	70 (70)	Esaote‐Biomedica AU4 Idea (Genoa, Italy) Doppler	20 MHz axial 20 µm; lateral 100 µm 7 MHz	1. Hypoechoic, homogenous and well defined HFUS + Doppler As above OR intratumoral vessels present	Not described	Histology MM 19; BCC 31; neurosarcoma 1 BN 12	6/38 clinically suspected MEL not visualised on HFUS (including 3 melanomas); plus 1/32 suspected BCCs (which proved to be an actinic keratosis); (7/70 = 10%);
Rallan 2007	NC CCS UK Specialist	Patients referred to a skin cancer clinic with a suspicion of melanoma and with a subsequent clinical diagnosis of SK, BN, or suspicion of melanoma	87 (87)	Dermascan Cv3 Cortex ApS, Denmark); 20 MHz, modified for RTI using a transducer with an f‐number of 0.95	‐	1. Presence of either RTI contrast, LBI relative heterogeneity, OR EEI relative heterogeneity; mechanism/values indicative of presence of each feature NR	Not described	Histology MM 14; MiS 11 BN 38; SK 24	None reported
NC: non comparative study; WPC: within‐person comparison study; P: prospective; NR: not reported; CS: case series; CCS: case control study; HFUS: high‐frequency ultrasound; PSL: pigmented skin lesion; MEL: melanoma (invasive and in situ); MM: malignant melanoma; MiS: melanoma in situ; BCC: basal cell carcinoma; BN: benign naevi; SK: seborrhoeic keratosis; DF: dermatofibroma; RTI: reflex transmission imaging; LBI: lesional backscatter image; EEL: entry echo line; EEI: entry echo image; DER: dermal echogenicity ratio; GP: general practitioner. *Also reports in person diagnosis for VI and for Dermoscopy

Data

Presented below are all the data for all of the tests entered into the review.

Tests. Data tables by test.

Test	No. of studies	No. of participants
1 Melanoma ‐ hypoechoic	1	792
2 Melanoma ‐ hypoechoic and homogenous	1	792
3 Melanoma ‐ hypoechoic and sharp basal margins	1	792
4 Melanoma ‐ hypoechoic and sharp lateral margins	1	792
5 Melanoma ‐ hypoechoic, homogenous and well defined	2	177
6 Melanoma (melanoma vs benign naevi) ‐ hypoechoic, homogenous and well defined	1	32
7 Melanoma (melanoma vs seborrhoeic keratosis) ‐ dermal echogenicity ratio < 3	1	54
8 Melanoma (melanoma vs seborrhoeic keratosis) ‐ absence of entry echo line	1	54
9 Melanoma (melanoma vs seborrhoeic keratosis) ‐ dermal echogenicity ratio < 3 OR absence of entry echo line	1	54
10 Melanoma (melanoma vs seborrhoeic keratosis or benign naevi) ‐ absence of entry echo line	1	69
11 Melanoma ‐ reflex transmission image contrast/lesional backscatter image relative. Heterogeneity/entry echo image relative heterogeneity	1	87
12 Melanoma ‐ HFUS positive OR Doppler positive	1	63
13 Melanoma (melanoma vs benign naevi) ‐ HFUS positive OR Doppler positive	1	32
14 Melanoma ‐ HFUS positive AND Doppler positive	2	170
15 BCC ‐ hypoechoic	2	930
16 BCC ‐ hypoechoic and homogenous	2	930
17 BCC ‐ hypoechoic and well defined	1	138
18 BCC ‐ hypoechoic and sharp basal margins	1	792
19 BCC ‐ hypoechoic and sharp lateral margins	1	792
20 BCC ‐ hypoechoic, homogenous and well defined	1	63
21 BCC ‐ hypoechoic, heterogenous with irregular margins	1	63
22 BCC ‐ HFUS positive AND Doppler positive	1	138

1. Test.

Melanoma ‐ hypoechoic.

2. Test.

Melanoma ‐ hypoechoic and homogenous.

3. Test.

Melanoma ‐ hypoechoic and sharp basal margins.

4. Test.

Melanoma ‐ hypoechoic and sharp lateral margins.

5. Test.

Melanoma ‐ hypoechoic, homogenous and well defined.

6. Test.

Melanoma (melanoma vs benign naevi) ‐ hypoechoic, homogenous and well defined.

7. Test.

Melanoma (melanoma vs seborrhoeic keratosis) ‐ dermal echogenicity ratio < 3.

8. Test.

Melanoma (melanoma vs seborrhoeic keratosis) ‐ absence of entry echo line.

9. Test.

Melanoma (melanoma vs seborrhoeic keratosis) ‐ dermal echogenicity ratio < 3 OR absence of entry echo line.

10. Test.

Melanoma (melanoma vs seborrhoeic keratosis or benign naevi) ‐ absence of entry echo line.

11. Test.

Melanoma ‐ reflex transmission image contrast/lesional backscatter image relative. Heterogeneity/entry echo image relative heterogeneity.

12. Test.

Melanoma ‐ HFUS positive OR Doppler positive.

13. Test.

Melanoma (melanoma vs benign naevi) ‐ HFUS positive OR Doppler positive.

14. Test.

Melanoma ‐ HFUS positive AND Doppler positive.

15. Test.

BCC ‐ hypoechoic.

16. Test.

BCC ‐ hypoechoic and homogenous.

17. Test.

BCC ‐ hypoechoic and well defined.

18. Test.

BCC ‐ hypoechoic and sharp basal margins.

19. Test.

BCC ‐ hypoechoic and sharp lateral margins.

20. Test.

BCC ‐ hypoechoic, homogenous and well defined.

21. Test.

BCC ‐ hypoechoic, heterogenous with irregular margins.

22. Test.

BCC ‐ HFUS positive AND Doppler positive.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Bessoud 2003.

Study characteristics
Patient sampling	Study design: case series Data collection: prospective Period of data collection: not reported; 4‐year period Country: France
Patient characteristics and setting	Inclusion criteria: patients with pigmented skin lesions referred from the Dermatology Department to the Ultrasound Unit Setting: secondary Prior testing: referred from Dermatology; basis for referral not described Exclusion criteria: none reported Sample size (patients): no. eligible: 111 Sample size (lesions): no. eligible: 130; no. included: 114 (107 for Doppler) Participant characteristics: mean age: 55.3 (SD 18; range 6 to 92 years). Men: 47 (42%) Lesion characteristics: for melanomas visualised on ultrasound (n = 65), thickness ranged from 0.15 mm to 8 mm on histology.
Index tests	Ultrasound: high‐frequency (20 MHz) and Colour Doppler (7MHz) Test detail: AU 4 or AU 5 Idea (Esaote‐Biomedica, Genova, Italy) with a 20‐MHz annular probe (axial resolution 80 µm and lateral resolution 100 µm) and 13‐MHz linear electronic probe (axial resolution 200 µm and lateral resolution 400 µm); Colour Doppler adjustments included a pulse‐repetition frequency (PRF) of 750 Hz to 1 kHz, with a 50‐Hz filter and 9 to 16 images per second Method of diagnosis: in‐person diagnosis Prior test data available: unclear whether clinical diagnosis provided to sonographer Diagnostic threshold: HFUS ‐ hypoechoic, homogenous and well‐defined margins; HFUS plus Doppler ‐ hypoechoic, homogenous and well defined plus presence of intralesional vessels Diagnosis based on: unclear whether single or multiple observers (n not reported) Observer qualifications: not reported Experience in practice: not described Experience with index test: not described
Target condition and reference standard(s)	Type of reference standard: histological diagnosis alone Details: none provided Disease positive: 70; disease negative: 60 Target condition (final diagnoses) Melanoma (invasive or in situ): 65; BCC: 4; 1 neurosarcoma 'Benign' diagnoses: 33 benign naevi, 5 seborrhoeic keratosis, 3 melanosis, 1 thrombosing capillaritis, 1 histiocytofibroma, 1 lentigo
Flow and timing	Index test to reference standard interval: not described Exclusions: 16 lesions 'unseen' on US were excluded (5 melanoma, 1 lentigo, and 10 benign naevi) leaving 114 lesions reported for HFUS, 107 of which underwent Doppler ultrasound
Comparative
Notes	—
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	Unclear
Did the study avoid including participants with multiple lesions?	No
		Unclear	High
DOMAIN 2: Index Test All tests
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Was the test applied and interpreted in a clinically applicable manner?	Yes
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		Low	Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Unclear	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
		Low

Clement 2001.

Study characteristics
Patient sampling	Study design: case series Data collection: prospective Period of data collection: November 1998 to July 1999 Country: France
Patient characteristics and setting	Inclusion criteria: patients with pigmented skin tumours including melanocytic and non‐melanocytic examined before resection; clinical diagnoses described as 'hesitant' (NB translation from French) for more than half of lesions Setting: secondary Prior testing: not reported; basis for referral not described Exclusion criteria: difficult to reach lesions (2 dermal nevus ‐ 1 at the internal angle of the eye and the other between the toes) Sample size (patients): no. eligible: 160 Sample size (lesions): no. eligible: 176; no. included: 138 Participant characteristics: for full sample ‐ mean age: 52.7 years (range 18 to 90 years). Men: 74 (46%) Lesion characteristics: 5 melanomas not visualised on ultrasound; all had Breslow index less than 0.35 mm
Index tests	Ultrasound: high‐frequency (20 MHz) and Colour Doppler (7 MHz) Test detail: used an annular linear scanning probe with theoretical spatial resolution of 80 μm (axial) and 100 μm (lateral); equipped with an ultrasonic beam variable electronics management system to obtain an optimal focal area at penetration depths of 12.5 mm and 19 mm; for Doppler, a linear electronic probe (frequency of ultrasound: 13 MHz, axial theoretical spatial resolution: 200 μm, lateral: 400 μm) was used Method of diagnosis: in‐person diagnosis Prior test data available: unclear whether clinical diagnosis provided to sonographer Diagnostic threshold: HFUS ‐ hypoechoic; hypoechoic and homogenous; hypoechoic and well defined; HFUS + Doppler ‐ hypoechoic and presence of vascularity Diagnosis based on: unclear whether single or multiple observers (n not reported) Observer qualifications: not reported Experience in practice: not described Experience with index test: not described
Target condition and reference standard(s)	Type of reference standard: histological diagnosis alone Details: none provided Disease positive: 24; disease negative: 115 Target condition (final diagnoses) Melanoma (invasive or in situ): 1; BCC: 23; 6 melanoma metastases (considered disease negative for this review) 'Benign' diagnoses: 61 benign naevi, 29 seborrhoeic keratosis, 11 histiocytofibroma, 7 angioma
Flow and timing	Index test to reference standard interval: consecutive; each lesion scanned immediately before its biopsy or surgical excision, or both Exclusions: 36 lesions were not visualised on US and were excluded (including 5 melanomas in the horizontal growth phase (Clark levels I and II, Breslow index less than 0.35 mm)) leaving 138 lesions reported for HFUS
Comparative
Notes	—
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	Yes
Did the study avoid including participants with multiple lesions?	Yes
		Unclear	Low
DOMAIN 2: Index Test All tests
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Was the test applied and interpreted in a clinically applicable manner?	Yes
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		High	Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Unclear	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
		Low

Dummer 1995.

Study characteristics
Patient sampling	Study design: case series Data collection: unclear Period of data collection: not reported Country: Germany
Patient characteristics and setting	Inclusion criteria: patients with pigmented skin lesions referred from the outpatient clinic to the Department of Dermatology Setting: secondary Prior testing: all patients underwent physical examination and 508 underwent dermoscopy before HFUS; basis for referral not described Exclusion criteria: none reported Sample size (patients): no. eligible: not reported Sample size (lesions): no. eligible: 792; no. included: 792 Participant characteristics: mean age: not reported. Men: not reported Lesion characteristics: for the 108 melanomas, Breslow thickness was < 0.76 mm in 45; 0.76 mm to 1.5 mm in 26; 1.5 mm to 4.0 mm in 24; and > 4.0 mm in 12
Index tests	Ultrasound: high‐frequency (20 MHz) Test detail: DUB 20 (Taberna pro Medicum, Luneburg, Germany) at axial resolution 80 μm and lateral resolution 200 μm. Several sonographic scans were carried out perpendicular to the previous ones in parallel planes for each individual tumour; B‐scan section corresponds to a width of 12.8 mm and a depth of 7.5 mm Method of diagnosis: in‐person diagnosis Prior test data available: unclear whether clinical or dermoscopy diagnosis provided to sonographer; no data available for overall dermoscopy diagnosis Diagnostic threshold: echo poor (hypoechoic); echo poor with no internal echoes; echo poor with sharp basal margins; echo poor with sharp lateral margins Diagnosis based on: unclear whether single or multiple observers (n not reported) Observer qualifications: not reported Experience in practice: not described Experience with index test: not described
Target condition and reference standard(s)	Type of reference standard: histological diagnosis alone Details: excisional biopsy; reviewed by the dermatopathology staff at the departments of dermatology (Universities of Wurzburg and Munich, Germany) and 78 lesions additionally reviewed by the dermatopathology staff of the department of dermatology, University Hospital, Zurich, Switzerland Disease positive: 173; disease negative: 619 Target condition (final diagnoses) Melanoma (invasive or in situ): 108; BCC: 65 'Benign' diagnoses: 307 benign naevi, 211 seborrhoeic keratosis, 47 angioma, and 54 dermatofibroma
Flow and timing	Index test to reference standard interval: appears consecutive; patients were subjected to ELM and ultrasound examination. After excisional biopsy, the correlation between clinical, ELM, sonographic, and histologic diagnosis was established. Exclusions: no exclusions due to lack of visualisation on ultrasound
Comparative
Notes	—
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	Unclear
Did the study avoid including participants with multiple lesions?	Unclear
		Unclear	Unclear
DOMAIN 2: Index Test All tests
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Was the test applied and interpreted in a clinically applicable manner?	Yes
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		High	Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes
		Unclear	Low
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Unclear
		Unclear

Harland 2000.

Study characteristics
Patient sampling	Study design: case control (only specific diagnoses included) Data collection: unclear Period of data collection: not reported Country: UK
Patient characteristics and setting	Inclusion criteria: patients with pigmented lesions with specific presumptive clinical diagnoses (seborrhoeic keratosis, benign naevi or cutaneous malignant melanoma) recruited from a pigmented lesion clinic; the referring general practitioner had considered the diagnosis of melanoma for each lesion. Setting: specialist clinic Prior testing: clinical diagnosis made at PLC Exclusion criteria: lesions with macroscopic ulceration Sample size (patients): no. eligible: not reported Sample size (lesions): no. eligible: not reported; no. included: 69 Participant characteristics: mean age: not reported. Men: not reported Lesion characteristics: none reported
Index tests	Ultrasound: high‐frequency (20 MHz) Test detail: Dermascan‐CTM 20‐MHz B‐scanner (Cortex Technology, ApS, Hadsund, Denmark); axial resolution of 50 μm and a lateral resolution of 300 μm; in vivo slice 22'4 mm width, 13'4 mm depth (6'7 mm with zoom factor 2) and 200 μm thickness. Scanner described as "US prototype with a large unwieldy scanner head, such that certain sites, such as the inner canthus, are inaccessible to examination. However, the aim of this pilot study was not to evaluate practicality of clinical use." Method of diagnosis: in‐person diagnosis Prior test data available: unclear whether clinical diagnosis provided to sonographer Diagnostic threshold: DER ‐ dermal echogenicity ratio < 3 set to ensure sensitivity of 100% for melanoma; EEL ‐ absence of entry echo line (documented as either equivalent to perilesional skin (non‐enhanced) or as broadened); DER < 3 or absence of EEL Diagnosis based on: unclear whether single or multiple observers (n not reported) Observer qualifications: not reported Experience in practice: not described Experience with index test: not described
Target condition and reference standard(s)	Type of reference standard: histological diagnosis alone Details: histological evaluation was performed without knowledge of the ultrasound findings. Histological sections of tumours were prepared in the same plane as the B‐scans, both being centred upon the transverse reference line Target condition (final diagnoses) Melanoma (invasive): 19. Melanoma in situ: 6. BCC: 0 'Benign' diagnoses: 15 benign naevi, 29 seborrhoeic keratosis
Flow and timing	Index test to reference standard interval: not described Exclusions: no exclusions due to lack of visualisation on ultrasound
Comparative
Notes	—
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	No
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
		High	High
DOMAIN 2: Index Test All tests
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		High	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Low	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
		Unclear

Lassau 1997.

Study characteristics
Patient sampling	Study design: case series Data collection: prospective Period of data collection: not reported Country: France
Patient characteristics and setting	Inclusion criteria: patients with skin lesions clinically suspected of being either melanoma or BCC and scheduled for resection; includes only very specific lesion groups (MM, BCC, and benign naevi) Setting: secondary Prior testing: clinical diagnosis; basis for referral for US not described Exclusion criteria: none reported Sample size (patients): no. eligible: 70 Sample size (lesions): no. included: 70 Participant characteristics: mean age: not reported. Men: not reported Lesion characteristics: melanoma thickness on histology ranged from 0.25 mm to 6 mm (n = 19)
Index tests	Ultrasound: high‐frequency (20 MHz); Colour Doppler (7 MHz) Test detail: Esaote‐Biomedica AU4 Idea (Genoa, Italy). HFUS ‐ 20‐MHz annular probe with an axial resolution of 20 µm and a lateral resolution of 100 µm; Doppler ‐ a 13‐MHz linear probe with an axial resolution of 200 µm and a lateral resolution of 400 µm for performing pulsed and Colour Doppler US. Theoretical depth explored was 16 mm (HFUS) and 40 mm (linear/Doppler) Method of diagnosis: in‐person diagnosis Prior test data available: unclear whether clinical diagnosis provided to sonographer Diagnostic threshold: HFUS ‐ hypoechoic with a homogeneous echostructure and well‐defined lower and lateral margins; HFUS plus Doppler: hypoechoic with a homogeneous echostructure and well‐defined lower and lateral margins OR presence of intratumoral vessels on Doppler Diagnosis based on: unclear whether single or multiple observers (n not reported) Observer qualifications: not reported Experience in practice: not described Experience with index test: not described
Target condition and reference standard(s)	Type of reference standard: histological diagnosis alone Details: none reported Target condition (final diagnoses) Melanoma (invasive or in situ): 19; BCC: 31; plus 1 neurosarcoma 'Benign' diagnoses: 12 benign naevi, seborrhoeic keratosis
Flow and timing	Index test to reference standard interval: "After surgical resection, tumors were analyzed histologically" Exclusions: 6/38 clinically suspected MEL not visualised on HFUS (including 3 melanomas); plus 1/32 suspected BCC lesions were not visualised on US and were excluded leaving 63 lesions reported for HFUS
Comparative
Notes	—
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Yes
		Unclear	High
DOMAIN 2: Index Test All tests
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Was the test applied and interpreted in a clinically applicable manner?	Yes
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		High	Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Unclear	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
		Low

Rallan 2007.

Study characteristics
Patient sampling	Study design: case control (only selected diagnoses included) Data collection: unclear Period of data collection: not reported Country: UK
Patient characteristics and setting	Inclusion criteria: patients referred to a skin cancer clinic with a suspicion of melanoma and with a subsequent clinical diagnosis of SK, benign nevus, or suspicion of melanoma Setting: specialist clinic Prior testing: clinical diagnosis by a dermatologist; basis for referral for US not described Exclusion criteria: head/neck excluded; > 20 mm excluded Sample size (patients): no. eligible: 87 Sample size (lesions): no. included: 87 Participant characteristics: mean age: not reported (range 21 to 67 years). Men: 24; 28% Lesion characteristics: mean Breslow thickness for invasive melanomas: 0.97 ± 0.29 mm, range 0.25 mm to 2.0 mm
Index tests	Ultrasound: high‐frequency (20 MHz) with reflex transmission imaging (RTI) Test detail: Dermascan Cv3 Cortex ApS (Denmark); 3 types of images generated ‐ a reflex transmission image (RTI) predominantly influenced by ultrasonic attenuation in the focal plane, a lesional backscatter image (LBI) based on an integration zone through the lesion body and an entry echo image (EEI) based on an integration zone through the skin surface. Quote: "RTI parameters refer to lesion attenuation properties, LBI and EEI parameters depict intralesional sound reflection and surface sound reflectance characteristics, respectively". Referenced to Rallan 2006; however, relatively experimental in nature. Method of diagnosis: in‐person diagnosis Prior test data available: unclear whether clinical diagnosis provided to sonographer Diagnostic threshold: based on presence of statistically significant characteristics related to contrast and relative heterogeneity of each type of image (these were identified from comparison of mean values between MM vs SK and MM vs BN). Three significant characteristics were identified ‐ RTI contrast, LBI relative heterogeneity, and EEI relative heterogeneity Diagnosis based on: unclear whether single or multiple observers (n not reported) Observer qualifications: not reported Experience in practice: not described Experience with index test: not described
Target condition and reference standard(s)	Type of reference standard: histological diagnosis alone Details: lesions were "removed under local anaesthetic following data acquisition. Histological diagnosis was then used to classify the lesion in one of three groups, MM, SK, or other benign‐pigmented lesion. In cases of histological atypia or dyplasia, suggesting but not confirming melanoma, the lesion was classed in accordance with the clinical management protocol (usually as melanoma)." Target condition (final diagnoses) Melanoma (invasive): 14; melanoma in situ: 11; BCC: 0. 'Benign' diagnoses: 38 benign naevi, 24 seborrhoeic keratosis
Flow and timing	Index test to reference standard interval: consecutive Exclusions: no lesions reported that were not visualised on US
Comparative
Notes	—
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	No
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Yes
		High	High
DOMAIN 2: Index Test All tests
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		High	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Unclear	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
		Low

BCC: basal cell carcinoma; BN: benign naevi; DER: dermal echogenicity ratio; EEI: entry echo image; EEL: entry echo line; ELM: epiluminescence microscopy; HFUS: high‐frequency ultrasound; Hz: hertz; LBI: lesional backscatter image; MEL: melanoma (invasive and in situ);; MM: malignant melanoma; PLC: pigmented lesion clinic; PRF: pulse‐repetition frequency; RTI: reflex transmission image; SK: seborrhoeic keratosis; US: ultrasound.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Akata 1998	Inappropriate index test; Doppler US
Bens 2015	Assesses individual lesion characteristics only; assesses echogenicity alone and we required at least two HFUS features to be examined in combination
Bezugly 2015	Not a primary study
Bobadilla 2008	Inappropriate study population; only BCC lesions included
Cardenas 2009	Inappropriate index test; 17 MHz frequency
Delfino 2013	Inappropriate index test; 17‐MHz ultrasound probe
Evans 2014	Not a primary study
Fornage 1993	Inappropriate reference standard; maximum of 41% of benign group had adequate reference standard (if assume all malignant had histology). From paper ‐ pathologic diagnosis obtained for 109 lesions (54%) through shave, punch, or excisional biopsy; in the absence of pathologic analysis, the diagnosis was based on the dermatologist's assessment
Giovagnorio 2003	Inappropriate target condition; detection of metastases
Gropper 1993	Not a primary study; review
Harland 1993	Small sample size; 3 BCC; 1 SCC
Hernandez 2014	Not a primary study; comment paper
Hughes 1987	Inappropriate target condition; no breakdown of 17 malignant lesions undergoing Doppler Inappropriate index test; HFUS reported for thickness only; Doppler flow ± also reported
Hunger 2012	Inappropriate index test; high‐definition laser Doppler
Jambusaria‐Pahlajani 2009	Inappropriate study population; only biopsy confirmed BCC or SCC Inappropriate target condition; detection of surgical margin
Karaman 2001	Inappropriate index test; power Doppler
Krahn 1998	Inappropriate target condition; exclude high‐frequency ultrasound data ‐ only reports accurate detection of lesion thickness
Maj 2015	No 2 × 2 data; paper refers to Table I which contains 'detailed data' but there is no Table I in the paper Authors contacted December 2016 and May 2017
Marques 2002	Inappropriate index test; 10 MHz ultrasound ‐ not high frequency
Meyer 2014	Inappropriate target condition; identification of lesion thickness only
Ozkol 2006	Inappropriate target condition; D+ group includes 1 dermatofibroma and 1 Bowen's; cannot disaggregate Inappropriate index test; Colour Doppler
Petik 2013	Inappropriate index test; Colour Doppler plus power Doppler if vascularity not clearly identified
Rallan 2006	Derivation study; high‐resolution ultrasound with reflex transmission imaging (RTI); also combined with white light clinical (WLC) photography. No separate independent test set result is given; also 'white light' data is CAD based
Ravi 2000	Inappropriate index test; Colour Doppler
Samimi 2010	Inappropriate study population; blue naevus or melanoma metastasis Inappropriate target condition; melanoma metastasis
Schroder 1999	Inappropriate index test; not high frequency (10 MHz US)
Schröder 2001	Inappropriate index test; not HFUS
Scotto 2015	Inappropriate index test; US (5‐17 MHz) and Doppler
Song 2014	Inappropriate index test; not HFUS (7‐15 MHz)
Srivastava 1986	Inappropriate index test; Doppler US
Stucker 2002	Inappropriate index test; laser Doppler; assessment of blood flow
Stücker 1999	Inappropriate index test; laser Doppler US No 2 × 2 data; comparing mean tumour perfusion values between groups only
Wortsman 2010	Inappropriate target condition; can estimate sensitivity for detection of malignancy but cannot estimate specificity for benign lesions assessed to either rule in or rule out malignancy Inappropriate index test; up to 15 MHz ultrasound No 2 × 2 data; could get 2 × 2 from Table 1 but disease negative includes huge range of diagnoses that are not relevant to our review
Wortsman 2013	Not a primary study

BCC: basal cell carcinoma; CAD: computer‐assisted diagnosis; HFUS: high‐frequency ultrasound; SCC: squamous cell carcinoma; US: ultrasound; VI: visual inspection.

Characteristics of studies awaiting classification [ordered by study ID]

Nitsche 1992.

Study characteristics
Patient sampling	—
Patient characteristics and setting	—
Index tests	—
Target condition and reference standard(s)	—
Flow and timing	—
Comparative	—
Notes	British Library unable to supply copy of the paper

Differences between protocol and review

Due to the small number of studies available, we produced a single review that evaluates the accuracy of HFUS in all skin cancers; this replaces the two reviews intended in the protocols to address cutaneous melanoma and keratinocyte cancers.

This single review includes three primary objectives related to the detection of melanoma, BCC and cSCC. For the detection of melanoma, we changed the primary objectives and primary target condition from detection of invasive melanoma alone, to the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, as the latter is more clinically relevant to the practicing clinician. We added another secondary objective to allow the evaluation of Doppler ultrasound in combination with high‐frequency ultrasound for skin cancer diagnosis.

Availability of data limited heterogeneity investigations and sensitivity analyses.

We amended the text to clarify that studies available only as conference abstracts would be excluded from the review unless full papers could be identified; studies available only as conference abstracts do not allow a comprehensive assessment of study methods or methodological quality.

Due to the early phase nature of HFUS diagnosis for skin cancer, we replaced the following text from the protocol:

"We will include studies developing new algorithms or methods of diagnosis (i.e. derivation studies) if they use a separate independent 'test set' of participants or images to evaluate the new approach. We will also include studies using other forms of cross validation, such as 'leave‐one‐out' cross‐validation (Efron 1983). We will note for future reference (but not extract) any data on the accuracy of lesion characteristics individually, e.g. the presence or absence of a pigment network or detection of asymmetry"

with:

"Studies should ideally evaluate a predefined 'rule' or algorithm describing combinations of ultrasound characteristics that determine the presence or absence of melanoma, BCC or cSCC. However, as HFUS is in a relatively early phase of development, we included studies if we could extract 2 × 2 contingency table data based on the presence or absence of at least two ultrasound features related to tissue morphology or acoustic properties, for example echogenicity, homogeneity of appearance and definition of margins. Studies attempting to quantify HFUS parameters were also eligible for inclusion. There was no requirement for studies to have explicitly set out to estimate the diagnostic accuracy of the parameters assessed."

Although we extracted any reporting of special interest or accreditation in skin cancer according to observer expertise, we were unable to analyse the effect on accuracy.

We proposed to supplement the database searches by searching the annual meetings of appropriate organisations (e.g. British Association of Dermatologists Annual Meeting, American Academy of Dermatology Annual Meeting, European Academy of Dermatology and Venereology Meeting, Society for Melanoma Research Congress, World Congress of Dermatology, European Association of Dermato Oncology); however, due to volume of evidence retrieved from database searches and time restrictions we were unable to do this.

For quality assessment, the QUADAS‐2 tool was further tailored according to the review topic and to cover both melanoma and keratinocyte skin cancers. In terms of analysis, we did not restrict analysis of per patient data due to lack of data. Sensitivity analyses were not performed as planned due to lack of data.

Contributions of authors

JD was the contact person with the editorial base.
 JD co‐ordinated contributions from the co‐authors and wrote the final draft of the review.
 SB conducted the literature searches.
 JD and NC screened papers against eligibility criteria.
 JD and NC obtained data on ongoing and unpublished studies.
 JD and NC appraised the quality of papers.
 JD and NC extracted data for the review and sought additional information about papers.
 JD entered data into RevMan.
 JD and JJD analysed and interpreted data.
 JD, JJD, NC, YT and CD worked on the Methods sections.
 JD, JB, RNM and HCW drafted the clinical sections of the Background and responded to the clinical comments of the referees.
 JD, JJD, CD and YT responded to the methodology and statistics comments of the referees.
 KG and CO were the consumer co‐author and checked the review for readability and clarity, as well as ensuring outcomes are relevant to consumers.
 JD is the guarantor of the update.

Disclaimer

This project presents independent research supported by the National Institute for Health Research, via Cochrane Infrastructure funding to the Cochrane Skin Group and Cochrane Programme Grant funding, and the NIHR Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Systematic Reviews Programme, NIHR, NHS or the Department of Health and Social Care.

Sources of support

Internal sources

• No sources of support supplied

External sources

• NIHR Systematic Review Programme, UK.

  This project was funded by an NIHR Cochrane Systematic Reviews Programme Grant (13/89/15)

• The National Institute for Health Research (NIHR), UK.

  The NIHR, UK, is the largest single funder of the Cochrane Skin Group

• NIHR clinical fellowship, UK.

• NIHR Birmingham Biomedical Research Centre, UK.

  JD, JJD and YT receive support from the NIHR Birmingham Biomedical Research Centre

Declarations of interest

Jac Dinnes: none known.
 Jeff Bamber: J Bamber has been a member of the advisory board of Michelson Diagnostics and has received payment from Cancer Research UK and the International Breast Ultrasound School and Queen Mary University of London for lectures given. He has received book royalties from John Wiley and Sons, and acknowledges funding from the Engineering and Physical Sciences Research Council, the NIHR Biomedical Research Centre and the Royal Marsden NHS Foundation Trust and Institute of Cancer Research, and the Cancer Research UK Imaging Centre Grant to the Institute of Cancer Research. There was no involvement of grant funders or other sponsors in the study whatsoever. J Bamber is the first named inventor on a patent jointly held by his institution (The Institute of Cancer Research) and The University of Bern. The priority date was 26 July 2012. The PCT publication date was 30 January 2014. A brief description of the invention is as follows: "A method of ultrasound and photoacoustic imaging in which image clutter is reduced, thus improving image quality and penetration depth, by using a localised vibration to tag true signals so as to discriminate them from false signals which occur from sources outside of the imaged region."
 Naomi Chuchu: none known.
 Rubeta N Matin: "my institution received a grant for a Barco NV commercially sponsored study to evaluate digital dermoscopy in the skin cancer clinic. My institution also received Oxfordshire Health Services Research Charitable Funds for carrying out a study of feasibility of using the Skin Cancer Quality of Life Impact Tool (SCQOLIT) in non melanoma skin cancer. I have received royalties for the Oxford Handbook of Medical Dermatology (Oxford University Press) and payment from the UK Photopheresis Society for a lecture on cutaneous graft versus host disease (October 2017). I have received payment from Public Health England for the "Be Clear on Cancer" skin cancer report. I have no conflicts of interest to declare that directly relate to the publication of this work."
 Susan E Bayliss: none known.
 Yemisi Takwoingi: none known.
 Clare Davenport: none known.
 Kathie Godfrey: none known.
 Colette O'Sullivan: none known.
 Jonathan J Deeks: none known.
 Hywel C Williams: H Williams is director of the NIHR HTA Programme. HTA is part of the NIHR, which also supports the NIHR systematic reviews programme from which this work is funded.

Edited (no change to conclusions)