Binder 1995.

Study characteristics
Patient sampling	Study design: case series Data collection: retrospective image selection/prospective interpretation Period of data collection: NR Country: Austria
Patient characteristics and setting	Inclusion criteria: PSLs with available dermoscopy images, both with and without oil immersion, and histological confirmation of diagnosis. Setting: secondary (general dermatology) Prior testing: selected for excision (no further detail) Setting for prior testing: NR Exclusion criteria: none reported Sample size (participants): NR Sample size (lesions): number included: 240 Participant characteristics: NR Lesion characteristics: median thickness 0.7 mm, IQR 0.48‐0.76 mm; all < 1 cm diameter
Index tests	Dermoscopy: no algorithm Method of diagnosis: dermoscopic images of lesions with and without oil immersion (results with oil immersion used for primary analysis); images randomly presented to prevent consecutive presentation of slides for the same lesion. Each image was shown for 20 seconds with a 20‐minute break after 240 slides Prior test data: no further information presented Diagnostic threshold: correct diagnosis of melanoma. For each PSL image only one diagnosis was allowed (MM or not MM) Diagnosis based on: average (n = 19); 6 ELM experts and 13 randomly picked dermatologist 'nonexperts' Observer qualifications: dermatologist Experience in practice: high ‐ all certified dermatologists, experienced in clinical diagnosis Experience with dermoscopy: mixed. 'Nonexperts' had no formal ELM training; ‘expert’ users had been working scientifically in the development of ELM for at least 3 years Any other detail: images were obtained by photographing the PSLs on 24 x 36‐mm colour‐slide film with ELM and without oil immersion (surface microscopy ISM) using a binocular stereomicroscope (M 650, Wild AG, Heerbrugg, Switzerland) at a final magnification of x 16 using flashlight illumination
Target condition and reference standard(s)	Reference standard: histological diagnosis alone (not further described) Disease‐positive: 57; disease‐negative: 183 Target condition (final diagnoses) Melanoma (in situ and invasive, or NR): 57; BCC: 8 Severe dysplasia: 42; other 'Benign' : 133
Flow and timing	Reference interval: appears consecutive; "After photographing, all lesions were excised" Excluded participants: none reported
Comparative
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
		Unclear	High
DOMAIN 2: Index Test Dermoscopy ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?	Yes
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	No
Was the test interpretation carried out by an experienced examiner?	Yes
		Low	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Low	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?
		Low