Study author Outcomes reported Pathway	Study type Country Setting	Inclusion criteria	Index tests (algorithm) Diagnostic approach	Threshold	Observer qualifcation (number) Experience	Reference standard Final diagnoses Prevalence (Any) Exclusions (if reported)
In‐person evaluations
Amirnia 2016 BCC Referred (selected on reference) (c)	NC NR‐CS Iran Secondary 61 / 61	Patients suspected of BCC or melanocytic naevi of the face who were referred to dermatology clinic	Dermoscopy (3‐point checklist plus dermatoscopic criteria of melanocytic naevi and BCC) In person	≥ 2 characteristics present; diagnosis of BCC	Dermatologist (assumed) (n = NR; experience NR) Single observer	Histology BCC 27 Benign 28 27/61; 44%
Argenziano 2006 Any Limited prior testing; selected on refererence standard (c)	BPC RCT Italy, Spain Primary NR / 85 (Full sample 1203 lesions*)	Patients asking for screening or exhibiting 1 or more skin tumours as seen during routine physical examination (patient‐finding screening). Participating PCPs randomised to either visual inspection alone or visual inspection plus dermoscopy; only excised lesions can be included for each arm.	VI (ABCD) Dermoscopy (3‐point checklist) In person	Subjective impression; dx of malignancy	GPs (n = 37) All trained in ABCD rule Single observer	Histology MEL 6 BCC 37; SCC 10 Benign 32 53/85; 62% NB: Only those patients who were considered to have lesions suggestive of skin cancer had histology and could be included; rest had expert diagnosis (making full dataset ineligible for this review)
Carli 2002a BCC (MEL) Referred (selected on reference) (u)	WPC NR‐CS Italy Secondary NR/256	Clinically equivocal or suspicious PSL subjected to excisional biopsy at the Institute of Dermatology	1. VI (no algorithm) 2. Dermoscopy (pattern) In‐person (Dermoscopy – image‐based)	Subjective impression	Dermatologist (n = 2; High experience – “extensive experience in both clinical and dermoscopic diagnosis”) Consensus of 2	Histology MM 40; MiS 14 BCC 5 BN 177; SN 16; SK 4 BCC: 5/256; 2% No exclusions reported NB: BCC (VI): 2 MMS were FP; BCC (Derm – pattern): all MM TN
Chang 2013 Any Referred (selected on reference) (u)	NC R‐CS Taiwan Secondary 676/769	Potentially malignant biopsied or excised skin lesions (nontumour specimens excluded)	VI (no algorithm) In person	Subjective impression; definitely malignant	Dermatologists; n = 25 Board‐certified Single observer	Histology MM 4; MiS 4 BCC: 110; cSCC: 20 'Benign' diagnoses: 595 Skin cancer: 152/769; 20% Exclusions: Poor‐quality index test image; mis‐registered or poor‐quality images (unfocused or containing a motion artifact)
Cooper 2002 BCC cSCC Any Follow‐up (c)	NC P‐CS UK Spec. clinic NR/102	Patients attending the open‐access dermatology renal transplant clinic with suspicious lesions	VI (No algorithm) In person	NR; correct diagnosis of malignancy	Mixed (n = 2; experience NR) Single observer	Histology BCC 12; cSCC 21 KA 2; BD 19; Solar 16; viral warts 7; other 25 BCC: 12/102; 12% SCC: 21/102; 21% Exclusions: BCC: 3 SCCs were FP
Durdu 2011 BCC Any (MEL) Referred (selected on reference) (u)	WPC P‐CS Secondary Turkey 176/200	PSL that could not be diagnosed with only dermatologic physical examination; 2 x 2 included for melanocytic subset	Dermoscopy (No algorithm (ABCD for diagnosis of melanoma only) Also evaluated exfoliative cytology In person	NR	Dermatologist (n = 1; experience NR) Single observer	Histology MEL 10; BCC: 34; Other malignant 2 SK 24; BN 100; DF 12; Warts 16; Dirt 1; Other 1 BCC: 34/200; 17% ‐
Ek 2005 BCC cSCC Any (MEL) Referred (selected on reference) (c)	NC P‐CS Aus. Specialist clinic 1223/2582	Lesions excised for which malignancy could not be excluded	VI (no algorithm) In person	Subjective impression	Plastic surgeon (n = 4 or 5; mixed experience; 3 consultants, 1 plastic surgery trainee (usually 1st year, on 6‐month rotation) and a clinical assistant) Unclear	Histology MEL 23 BCC 1214; SCC 517; BD 188; SK 63; 577 other benign (incl 330 solar keratosis) BCC: 1214/2582; 47% SCC: 517/2582; 20% Exclusions: Incomplete or incorrectly entered proformas were excluded – 79 patients with 96 lesions NB for BCC: 202 SCC and 6 MM were counted as FPs
Gokdemir 2011 BCC [MEL] Referred (selected on reference) (u)	NC NR‐CS Secondary Turkey 362/449	Patients with melanocytic and non‐melanocytic skin lesions with dermoscopic and histologic diagnoses	Dermoscopy (no algorithm) Unclear if in‐person or image‐based	Subjective assessment (dx of MM)	Dermatologist (n = NR; experience High “at least 2 years’ experience with Molemax II”) Unclear obs interp	Histology MEL 13; BCC: 45 Benign: 390 BCC: 45/448; 10% NB for BCC: 1 MM was counted as FP
Hacioglu 2013 Any Referred (selected on reference) (u)	WPC NR‐CS Turkey Secondary 76 / 80	Patients with skin lesions <12 mm diameter suspicious for malignancy; lesions that had a crusted or rough surface were excluded. NB aim is diagnose non melanoma skin cancers	VI (no algorithm) In‐person [Also evaluates image‐based dermoscopy and CAD]	Subjective impression; diagnosis of BCC/cSCC	Dermatologist (assumed) (n = 1; experience NR) Single observer	Histology MM 3; BCC 24; cSCC 3; basosquamous 2 SK 19; AK 8; intradermal nevus 4; DF 3; KA 2; Other 12 Skin cacner: 29/80; 36% Study reports 0 excluded from analysis after histopathology results NB: 3 MM considered disease negative by authors; cannot be disaggregated
Markowitz 2015 BCC Equivocal lesions (selected on reference) (u)	WPC P‐CS US Secondary 100 / 115	Adults with ≤ 3 suspicious lesions, if they had ≥ 1 clinically challenging pink lesions, on the head or neck, that was suspicious for BCC, and to be biopsied to rule BCC in or out, and if they were eligible for Mohs surgery	VI (no algorithm) Dermoscopy (2‐step algorithm Marghoob 2010) In‐person (Also evaluates OCT)	Possible BCC	Dermatologist (assumed) (n = NR; experience NR) Unclear	Histology BCC 70 Benign 45 BCC: 70/115; 61% No exclusions reported
Schwartzberg 2005 BCC Referred (selected on reference) (u)	WPC‐algs P‐CS US Secondary 141/141	Patients with suspected BCC undergoing biopsy	VI (no algorithm; own new algorithm) In‐person	BCC certain or likely (Confidence level 1 or 2)	Dermatologist (assumed) (n = 17; experience NR) Single	Histology BCC 82 Benign 59 BCC: 82/141; 58% ‐
Stanganelli 2000 BCC Any (MEL) Referred (unselected on reference) (u)	WPC R‐CS Italy Specialist clinic NR/3372	PSL referred by dermatologists and general practitioners either for pre‐surgical assessment or consultation	1. VI (ABCD) 2. Dermoscopy (pattern analysis) In person	NR Subjective impression	NR (assumed dermatologist ‐ described as one of the co‐authors; n = 1) Single observer	Histology / Registry FU MEL 55 BCC 43; Benign 3274 43/3372; 1% No exclusions reported NB for BCC: all MMs were TN for VI and for dermoscopy
Steiner 1987 BCC Any (MEL) Equivocal (selected on reference) (u)	WPC P‐CS Austria Spec. clinic NR / 318	Small (< 10 mm) diagnostically equivocal PSL; no absolute agreement on clinical diagnosis among investigating clinicians at a pigmented lesion clinic	1. VI (no algorithm) In person (also evaluated dermoscopy)	Subjective impression	Dermatologists (n = 3; High experience ‐ "experienced dermatologists") Consensus of 3 observers	Histology MM 49; MiS 24 BCC 20 BN 143; SK 20; lentigo simplex and naevoid lentigo 19; Other 15 BCC: 20/318; 9% No exclusions reported NB: Dermoscopy data excluded as no breakdown of incorrect diagnoses For BCC (VI): 3 MMs were counted as FP
Ulrich 2015 BCC Equivocal (selected on reference) (u)	WPC P‐CS Germany Secondary 155/231	Patients with non‐pigmented pink lesions with clinical suspicion of BCC requiring biopsy for diagnostic confirmation. Pink lesions defined as clinically unclear erythematous papule or plaque; either reddish macules, patches or small papules with or without scale	VI (no algorithm) Dermoscopy (2‐step algorithm Marghoob 2012) In person (Also evaluates OCT)	Clinical characteristics of BCC	Dermatologist (assumed) (n = NR; experience NR) Single observer	Histology *BCC 141 Benign 94 BCC:141/235; 60% Exclusions: Histology was missing for 21 lesions, and 1 case was found to have a combination of both BCC and SK or AK, leaving 235 lesions for analysis NB: 231 diagnoses available for VI (140 BCC) and 231 for dermoscopy (139 BCCs)
Image‐based evaluations
Altamura 2010 BCC Referred (selected on reference) (c)	NC RP‐CCS Secondary Italy; Aus; Austria NR/300	Skin lesions randomly selected from digital databases at dermatology departments and tertiary referral centre; all excised	Dermoscopy (Menzies for BCC (rev)) Image‐based (none)	Diagnosis of BCC	Dermatologist (assumed) (n = 3; experience High) observers experienced in dermatoscopic evaluation Single observer	Histology MM 40; MiS 10; BCC 150; cSCC 2 BN 50; SK 20; AK 12; DF 10; Other 6 BCC: 150/300; 50% NB: MM and cSCC results not disaggregated from Disease negative group
Carli 2002a BCC (MEL) Referred (selected on reference) (u)	WPC R‐CS Italy Secondary NR/256	Clinically equivocal or suspicious PSL subjected to excisional biopsy at the Institute of Dermatology	(Dermoscopy – image‐based) In person (Also evaluates in‐person VI and dermoscopy (see above))	Subjective impression	Dermatologist (n = 2; High experience – “extensive experience in both clinical and dermoscopic diagnosis”) Consensus of 2	Histology MM 40; MiS 14 BCC 5 BN 177; SN 16; SK 4 BCC: 5/256; 2% No exclusionsne reported NB for BCC: all MEL were test negative
Carli 2002b BCC Any (MEL) Referred (selected on reference) (u)	WPC R‐CS Italy Secondary NR / 57	Clinically suspicious or equivocal PSL undergoing excision for diagnostic purposes; all ≤ 14mm diameter	1. VI (NR) 2. Dermoscopy (NR) Image‐based (blinded)	NR	Dermatologists (n = 2) High experience ('with experience in the field of '); consensus of 2	Histology MM 6, MiS 5 BCC 10 BN 31, SK 1; Other 4 BCC; 10/57; 18% Exclusions: 4 ‘not evaluables’ excluded (NB these differ between clinical images and dermoscopic images (1 MM excluded from VI analysis)
Hacioglu 2013 Any Referred (selected on reference) (u)	WPC NR‐CS Turkey Secondary 76/80	Patients with skin lesions < 12 mm diameter suspicious for malignancy; lesions that had a crusted or rough surface were excluded. NB aim is diagnose non‐melanoma skin cancers	Dermoscopy (no algorithm) Image‐based (blinded) (Also evaluates in‐person VI and CAD)	Subjective impression; diagnosis of BCC/cSCC	Dermatologist (assumed) (n = 1; experience NR) Single observer	Histology MM 3; BCC 24; cSCC 3; basosquamous 2 SK 19; AK 8; intradermal naevus 4; DF 3; KA 2; Other 12 Skin cancer: 29/80; 36% Exclusions: Study reports 0 excluded from analysis after histopathology results B: 3 MM considered disease‐negative by study authors; cannot be disaggregated
Lorentzen 1999 BCC (MM) Referred (selected on reference) (c)	WPC P‐CS Specialist clinic Denmark 232/232	Patients with lesions suspicious for CMM referred to outpatients clinic	1. VI (no algorithm) 2. Dermoscopy (no algorithm) Image based (clinical image)	Subjective impression; correct dx of M	Mixed: Dermatologist (n = 4; experience High (4‐5 years daily experience) & 'non‐expert dermatology residents' (n = 5; 1 ‐ 2 years interest and formal training in dermatoscopy) Average	Histology MM 49; BCC 16 SK 12; BN 137 Other: 18 (SN, BD plus others) BCC: 16/232; 7% Exclusions Poor‐quality index test image 10 cases excluded NB for BCC: MM results not disaggregated
Lorentzen 2008 BCC MM Any Referred (selected on reference) (c)	WPC NR‐CS Specialist clinic Denmark 119/119	Patients referred to the specialist naevus clinic; compared classic dermoscopy to acrylic globe magnifer	Dermoscopy (Kenet risk stratification) Image‐based (blinded)	NR	Dermatologist (n = NR) Average	Histology MM 24; BCC 13 BN 69; Mild/moderate dysplasia 2; SK 9; Other 2 BCC: 13/119; 11% Exclusions: 1 dermatofibroma
Menzies 2000 BCC Any (MM‐excl) Referred (selected on reference) (u)	NC RP‐CCS Spec. clinic Aus; US Test set: NR/213 (Full sample 426)	PSL with dermoscopic images and histological diagnoses	Dermoscopy (Menzies for BCC (new)) Image‐based (none)	Absence of pigment network and ≥ 1 other char present; Dx	Dermatologist (assumed) (n = 2; experience NR) NR	Histology MM 71; BCC 71 BN 59; SK 5; Solar 3; DF 1; Other 3 BCC: 71/213; 33% NB: Included 142 BCCs, 142 invasive melanomas and 142 randomly‐sampled benign For BCC: 5 MM classed as FP
Navarrete Dechent 2016 BCC cSCC Any (MEL excl) Referred (selected on reference) (u)	NC RP‐CS Spec clinic US NR/457	Consecutively excised nonpigmented lesions; no discernible pigment on clinical or dermoscopic images.	Dermoscopy (Shiny white blotches and strands (new)) Image‐based (blinded)	≥ 1 char present	Dermatologist (assumed) and medical student (n = 2; experience NR) Consensus of 2	Histology MEL 21; BCC 287; cSCC 106 lichen planus–like keratosis 39; Naevus 4 BCC: 287/457; 63% cSCC: 106/457; 23% NB for BCC: 9 MM and 44 cSCC were counted as FP
Nori 2004 BCC Referred (selected on reference) (u)	WPC RP‐NR Secondary US; Spain 105 (VI) Full sample: 145/152	Biopsy confirmed BCC and convenience sample of non‐BCC with 'range of common diagnoses'; lesions with superior clinical image quality selected for VI	VI (no algorithm) Image based (blinded) (Also evaluates RCM)	Subjective impression: High/Med probability of BCC	Dermatologist (n = 2; experience NR) Single observer	Histology and Expert opinion* BCC 58 Benign 47 (Full sample includes 83 BCC; 4 SCC; 65 benign) BCC: 58/105; 55% NB: 15 lesions not biopsied because the clinical diagnosis was considered diagnostic (e.g.SK) cSCC results not disaggregated
Rosendahl 2011 BCC Any (MEL) Limited prior test (selected on reference) (u)	WPC‐algs R‐CS Aus. Primary 389/463	PSL submitted for histology from the primary‐care skin cancer practice of 1 author	1. VI (no algorithm) 2. Dermoscopy (pattern; chaos and clues)	1. Subjective impression 2. NR; both characteristics present	Dermatologist (n = 1) High experience (confirmed by author); Single observer	Histology MM 9; MiS 20 BCC 72; SCC 5 BN 217; BD 18; AK 14*; BNM 140 AK were considered malignant by study authors but not by review team BCC: 72/463; 16% Exclusions: 3 poor‐quality images excluded NB for BCC (VI): 3 MM were counted as FP; for BCC (Derm chaos/clues) 23 MM/MiS were counted as FPs; and for BCC (Pattern) 1 MM was counted as FP
Witkowski 2016 BCC cSCC Any (MEL excl) Equivocal (selected on reference) (u)	WPC RP‐CS Secondary Italy NR/260	Consecutive clinically equivocal ‘pink’ cutaneous lesions with absent pigmentation or containing < 10% pigment and absence of pigment network. All lesions were excised at first visit or follow‐up video dermoscopy control visit	Dermoscopy (No algorithm) Image based (blinded) (Also evaluates RCM)	NR	Dermatologist (assumed) (n = NR; experience NR) Single	Histology MEL 12; BCC 114; cSCC 13; Other malig 1 BN 47; SN 6; SL/SK/LPLK/AK 25; DF 18 Other 24 BCC: 114/260; 44% cSCC: 13/260; 5% NB for BCC: 1 MM and 1 cSCC were counted as FP
Zalaudek 2006 BCC Any (MEL) Referred (selected on reference) (u)	NC R‐CS Specialist clinic Italy NR/165	Random sample of excised, equivocal and nonequivocal, PSL and and non‐PSLs with melanin or haemoglobin pigmentation in all or part of the lesion.	Dermoscopy (3PCL) Image‐based (age, site, gender)	≥ 2 characteristics present	Mixed (n = 150; experience NR) Average result	Histology Full sample: MM 18; MiS 11 BCC: 18 79 BN; 26 SK; 8 vascular; 3 DF BCC: 18/150; 12% Exclusions: 15 used for training purposes NB for BCC: 7 MM were counted as FP
Footnotes: 3PCL ‐ three‐ point checklist; 7PCL ‐ seven‐point checklist; AK – actinic keratosis; BCC – basal cell carcinoma; BD – Bowen’s disease; BN – benign naevi; BPC – between person comparison (of tests); c ‐ clearly positioned on clinical pathway; CAD – computer‐assisted diagnosis; CCS – case control study; CS – case series; cSCC – cutaneous squamous cell carcinoma; DF – dermatofibroma; dx ‐ diagnosis; FP ‐ false positive; FU – follow‐up; KA ‐ keratoacanthoma; LPLK ‐ lichen planus‐like keratosis; LS – lentigo simplex; MEL: invasive melanoma or atypical intraepidermal melanocytic lesions; MiS – melanoma in situ (or lentigo maligna); MM – malignant (invasive) melanoma; NC – non comparative; NR – not reported; OCT ‐ optical coherence tomography; P – prospective; PLC – pigmented lesion clinic; PSL – pigmented skin lesion; R – retrospective; RCM – reflectance confocal microscopy; SK – seborrheic keratosis; SL ‐ solar lentigo; SN – Spitz naevi; TN ‐ true negative; u – unclear position on clinical pathway; WPC – within person comparison (of tests).