Lorentzen 1999a.

Study characteristics
Patient sampling	Study design: case series Data collection: prospective Period of data collection: 1994‐1997 Country: Denmark
Patient characteristics and setting	Inclusion criteria: patients with lesions suspicious for CMM referred to outpatients clinic; only excised included Setting: NR Prior testing: clinical suspicion of malignancy without dermatoscopic suspicion Setting for prior testing: NR Exclusion criteria: poor‐quality index test image (considered under flow/timing) Sample size (participants): number eligible: 242; number included: 232 Sample size (lesions): number eligible: 242; number included: 232* Participant characteristics: none reported Lesion characteristics: none reported *NB not all cases were assessed by all observers; 2x2 are based on presented sensitivity and specificity estimates for full dataset of lesions; "the dermatoscopy experts assessed almost all cases (98 ± 100%), whereas the non‐expert group completed fewer assessments, from 76%‐98%
Index tests	VI: no algorithm Method of diagnosis: clinical photographs Prior test data: no further information used; no option to change clinical diagnosis after viewing dermoscopic image Other test data: dermoscopic images presented to observer subsequent to diagnosis using clinical images alone; clinical images presented before dermoscopic images Diagnostic threshold: NR; clinical diagnosis Diagnosis based on: average; n = 9 Observer qualifications: dermatologist Experience in practice: high; moderate; mixed (average reported); 4 'experienced dermatologists' (4‐5 years' daily experience) & 5 'non‐expert dermatology residents' (1‐2 years' interest and formal training in dermatoscopy) Experience with index test: high; moderate; mixed Dermoscopy: no algorithm Method of diagnosis: dermoscopic images Prior test data: clinical image presented first Diagnostic threshold: clinical diagnosis; "observers were familiar with both the ABCD‐rule of dermatoscopy proposed by Stolz et al. (Stolz 1994b) and Kenet et al's risk‐stratifying algorithm of pigment network features of dermatoscopy (Kenet 1994). The observers were not constrained by either of the rules. The ABCD scores were not used to obtain the diagnoses. Rather a pattern recognition process was intended." Dermoscopy training: described as "formal training" Training format: non experts had undergone prior training in dermoscopy (not documented)
Target condition and reference standard(s)	Reference standard: histological diagnosis alone Details: a co‐author from Dept of Pathology "re‐evaluated all cases to confirm the pathology diagnosis, which was used as the gold standard in this study." Disease‐positive: 65; disease‐negative: 167 Target condition (final diagnoses) Melanoma (invasive): 49 'malignant melanoma' BCC: 16 Sebhorrheic keratosis: 12; benign naevus: 137 (pigmented naevi = 116; blue naevi = 16; atypical naevi = 5); other: 18 (SN, BD, sarcoid, naevus spilus, hemangioma, and others)
Flow and timing	Excluded participants: 10 cases were "considered unfit for evaluation" due to poor‐quality image Reference interval: "biopsy specimens...were obtained after the clinical and dermatoscopic photographs had been performed"
Comparative	Blinding between tests: Each observer first recorded the clinical diagnosis and then the dermatoscopic diagnosis on an entry form. Time interval between index test(s): same day; at time of face‐to‐face consultation
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
		Unclear	High
DOMAIN 2: Index Test Visual inspection ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Unclear
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	No
Was the test interpretation carried out by an experienced examiner?	Yes
		Unclear	High
DOMAIN 2: Index Test Dermoscopy ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Yes
		Low	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Low	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?
		High
DOMAIN 5: Comparative
Was each index test result interpreted without knowledge of the results of other index tests or testing strategies?	No
Was the interval between application of the index tests less than one month?	Yes
Were all tests applied and interpreted in a clinically applicable manner?	No
		Low	High