Friedman 2008.

Study characteristics
Patient sampling	Study design: case‐control Data collection: retrospective image selection/prospective interpretation Period of data collection: NR; lesions selected in July 2005 Country: USA Test set derived: MelaFind data randomly split into training and test sets however Melafind has previously been evaluated, the only difference here being that only small lesions were included. Full dataset included in review
Patient characteristics and setting	Inclusion criteria: a database of images of PSLs ≤ 6 mm was used to sample images of melanoma and non‐melanoma lesions; "approximately 80% of the lesions were biopsied to rule out melanoma, whereas the remaining lesions were biopsied mostly to rule out non‐melanoma skin cancer or because of patient concern." Setting: mixed (private and secondary); digital dermoscopic database acquired by Electro‐Optical Sciences Inc for the development and testing of MelaFind; 26 clinical sites have contributed (dermatologic hospital‐based clinics and private practice offices) Prior testing: selected for excision (no further detail). All lesions excised or underwent shave biopsy Setting for prior testing: NR Exclusion criteria: high‐grade dysplastic naevi were excluded. Previously biopsied, ulcerated, or bleeding lesions also excluded, as were those on mucosal surfaces and lesions that contained foreign matter (e.g. tattoos) Sample size (participants): number included: 94 Sample size (lesions): number eligible: 1977; number included: 99 Participant characteristics: none reported Lesion characteristics: 21 invasive MM: median thickness 0.32 mm (0.10‐1.40 mm). Lesion size: range: 2 mm‐22 mm
Index tests	Dermoscopy: no algorithm Method of diagnosis: dermoscopic images. Readers were provided with a CD‐ROM with colour dermoscopic images created using MelaFind multispectral image; for some cases standard dermoscopic images were also available. The equivalence of the 2 image types was assessed for a sample of 10 lesions by 3 readers. Prior test data: readers provided with participant gender, age, and lesion location; all evaluations were performed independently Diagnostic threshold: clinical diagnosis; “Is this lesion a melanoma?” and “Would you biopsy/excise this lesion?”. If readers indicated that they would biopsy the lesion because they were sure it was melanoma or to rule out melanoma, then the case was considered true‐positive Diagnosis based on: average; mean and median reported (n = 10); used mean value for review purposes Observer qualifications: 9 dermatologists; 1 nurse practitioner specialising in dermatology Experience in practice: high experience or ‘Expert’; "All 10 readers were expert dermoscopists (9 dermatologists and 1 nurse practitioner specialising in dermatology) Experience with dermoscopy: high experience /‘Expert’ users
Target condition and reference standard(s)	Reference standard: histological diagnosis alone Details: the original histology slides were evaluated by 2 out of 4 study dermatopathologists without knowledge of any additional clinical information; in cases of significant discordance in diagnoses, the slide was reviewed by a third study dermatopathologist. A lesion with at least 1 diagnosis of melanoma by the study dermatopathologists was considered melanoma. Dysplastic naevi with severe cytologic atypia were considered high grade, and those with mild to moderate atypia were considered low grade. Disease‐positive: 49; disease‐negative: 50 Target condition (final diagnoses) Melanoma (invasive): 21; melanoma (in situ): 28; BCC: 2 Mild/moderate dysplasia: 32 low‐grade dysplastic; SK: 2; 14 other benign
Flow and timing	Participant exclusions: none reported Index test to reference standard interval: timing between image acquisition and original histology NR
Comparative
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	No
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Yes
		High	High
DOMAIN 2: Index Test Dermoscopy ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	No
Was the test interpretation carried out by an experienced examiner?	Yes
		Low	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes
		Low	Low
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?
		Unclear