Benelli 2001.

Study characteristics
Patient sampling	Study design: case series Data collection: retrospective image selection/prospective interpretation Period of data collection: NR ‐ only dates of training course and agreement study given (April‐May 1999) Country: Italy
Patient characteristics and setting	Inclusion criteria: slides of pigmented skin tumours were selected for evaluation during a training course on dermoscopy. Lesions not located on head, palms or soles histological slide available Setting: training images; authors' institution. Institute of Dermatologic Sciences, University of Milan Prior testing: slides of pigmented skin tumours were selected for evaluation during a training course on dermoscopy Setting for prior testing: unspecified Exclusion criteria: none reported Sample size (participants): NR Sample size (lesions): number included: 49 (paper reports 50 but only 49 accounted for in text) Participant characteristics: none reported Lesion characteristics: none reported
Index tests	VI: ABCDE Method of diagnosis: clinical photographs Prior test data: no further information used Diagnostic threshold: ABCDE Score ≥ 2; presence of 2 criteria; ABCDE Score ≥ 3; presence of 3 criteria. All criteria described in full Diagnosis based on: single (n = 1); average (n = 65; attending 1/3 courses in dermoscopy held to inform dermatologists about a new dermatoscopic diagnostic method (7FFM)) Observer qualifications: dermatologists Experience in practice: expert author; not described for participating dermatologists Experience with dermoscopy: expert author; prior experience not described for participating dermatologists; all underwent dermoscopy training for study purposes Dermoscopy: 7FFM Method of diagnosis: dermoscopic images Prior test data: no further information used although clinicians had evaluated clinical images for the same 50 lesions earlier the same day Diagnostic threshold: malignant if 7FFM Score ≥ 2; i.e. presence of 1 major feature or concurrent presence of 2 minor features. All criteria described in full Test observers: as described for VI (above) Dermoscopy training: 3 one‐day dermatoscopy courses held to inform dermatologists about authors' own new dermoscopy algorithm (7FFM). Each course lasted 6 hours. Morning session participants executed pre‐test interpretation of clinical images using ABCDE. Then principles of dermatoscopy were presented during the course and as post‐test, participants evaluated 50 dermoscopic slides of same lesions using 7FFM Length of training 1 day (6 hours) Post‐training experience: < 6months Training format: in‐person teaching
Target condition and reference standard(s)	Reference standard: histological diagnosis alone Disease‐positive: 12/49 melanomas (paper reports 50 but only 49 accounted for in text) Target condition (final diagnoses) Melanoma (invasive): 10; melanoma (in situ): 2; BCC: 2 pigmented BCC 3 seborrhoeic keratoses, 2 pigmented BCC, 1 blue naevus, 2 angiokeratoma, 5 SN, 5 junctional naevi, 9 compound naevi, 10 naevi undergoing regression
Flow and timing	Excluded participants: none reported Time interval to reference test: unclear
Comparative	Blinding between tests: clinical images interpreted in the morning and dermoscopic images in the afternoon Time interval between index test(s): image capture NR
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
		Unclear	High
DOMAIN 2: Index Test Visual inspection ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Yes
		Low	High
DOMAIN 2: Index Test Dermoscopy ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?	Unclear
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Yes
		Low	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		Low	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?
		Unclear
DOMAIN 5: Comparative
Was each index test result interpreted without knowledge of the results of other index tests or testing strategies?	Yes
Was the interval between application of the index tests less than one month?	Unclear
Were all tests applied and interpreted in a clinically applicable manner?	No
		Unclear	High