Dreiseitl 2009.

Study characteristics
Patient sampling	Study design: case series Data collection: prospective Period of data collection: test set: February‐November 2004 Country: Austria Test set derived: study focuses on test set but gives detail of separate study in which classifier was trained
Patient characteristics and setting	Inclusion criteria: patients presenting at PSL clinic Setting: specialist unit (skin cancer/PLC) The PSL unit of the Department of Dermatology at the Medical University of Vienna serves as a secondary and tertiary referral centre Prior testing: NR Setting for prior testing: specialist unit (skin cancer/PLC) Exclusion criteria: none reported Sample size (participants): number eligible: 511; number included: 458 with complete information Sample size (lesions): number eligible: 3827; number included: 3021; however data reported on a per‐participant basis Participant characteristics: none reported Lesion characteristics: none reported
Index tests	Dermoscopy: no algorithm Method of diagnosis: in‐person diagnosis; physicians were instructed to perform an independent routine examination on the study participants Prior test data: clinical examination and/or case notes Diagnostic threshold: NR; decision to excise to rule out melanoma histopathologically Diagnosis based on: single observer (n = 1) Observer qualifications: dermatologist; data reported for 6 additional less experienced observers using MoleMax II system (reported in CAD review) Experience in practice: high experience; "Expert dermatologist" Experience with dermoscopy: high experience
Target condition and reference standard(s)	Reference standard: histological diagnosis + follow‐up Histology (excision); number patient/lesions: NR Clinical follow‐up + histology of suspicious lesions Length of follow‐up: 6 months; number participants: NR Target condition (final diagnoses) Melanoma (in situ and invasive, or NR): 27 participants; 31 lesions 'Benign' diagnoses: 431 participants; 2990 lesions
Flow and timing	Excluded participants: 806 lesions (53 participants) with inadequate follow‐up Index test to reference standard interval:
Comparative
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Are the included patients and chosen study setting appropriate?	Yes
Did the study avoid including participants with multiple lesions?	Yes
		Low	Low
DOMAIN 2: Index Test Dermoscopy ‐ in‐person
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Unclear
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	Yes
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	No
Was the test interpretation carried out by an experienced examiner?	Yes
		Unclear	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Unclear	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	No
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?	Yes
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?
		High