Seidenari 1999.

Study characteristics
Patient sampling	Study design: Case‐control Data collection: Retrospective image selection/Prospective interpretation Period of data collection: NR Country: Italy Test set derived: Not clearly reported, but appears that the training set was randomly sampled, but the melanomas in the training set were supplemented with images of lesions of comparable size but thicker than 0.75 mm, randomly selected from other melanoma images
Patient characteristics and setting	Inclusion criteria: PSLs with x20 magnification images Setting: Secondary (general dermatology) From authors' institution Prior testing: Selected for excision (no further detail) Setting for prior testing: Unspecified Exclusion criteria: None reported Sample size (participants): NR Sample size (lesions): N eligible: 461; N included: 383 in test set, 78 in training set Participant characteristics: Thickness ≤ 1 mm: 18 (100%) < 0.75 mm (8 in situ)
Index tests	Computer‐assisted diagnosis ‐ Dermoscopy‐based Derm‐CAD system: DB–MIPS (Biomips Engineering, Italy) (Multivariate discriminant analysis classifier) System details: Dermoscopy unit, internal stereomicroscope, internal DB, DB–MIPS pattern analysis system – integrated database stores the patient's data and the description of the lesion along with the image icons. 38 features analysed (grouped into geometries, colours and Burroni's islands of colours) Derivation study (internal validation) Lesion characteristics assessed: The borders of the lesion were automatically identified, plus estimation of radius, area and perimeter of the lesion, symmetry and circularity, fractality (shape), texture analysis, colour expressed as red, green and blue components, skin lesion gradient, 'dark areas' inside the lesion. All described in detail. Approach to feature selection DBDermo‐MIPS software. "Discriminant analysis enables the identification of variables that are important for distinguishing between the groups in the training set in order to develop a procedure for predicting group membership for new cases in which group membership is undetermined (test set). Using the training set data, a threshold score was established that enabled the attribution of each malignant lesion to the right group (100% sensitivity). The same value was employed for discriminating benign and malignant lesions belonging to the test set" Additional predictors included: Unclear; for each participant personal data and information such as the site of the lesion, the magnification, the clinical and the histological diagnosis were recorded. Unclear how these were used Method of diagnosis: In‐person diagnosis CAD‐based diagnosis Prior/other test data: Unclear CAD output: NR Diagnostic threshold: Threshold not reported. Using the training set data, a threshold score was established that enabled the attribution of each malignant lesion to the right group (100% sensitivity). The same value was used for discriminating benign and malignant lesions belonging to the test set
Target condition and reference standard(s)	Reference standard: Histological diagnosis alone Histology (not further described) N participants/lesions: 461 (383 in training set) Disease‐positive: 18; Disease‐negative: 365 Target condition (Final diagnoses): Melanoma: invasive 10, in situ 8 'Benign' diagnoses: 365 non‐melanoma cases consisted of benign naevi including common naevi and clinically dysplastic naevi (> 5 mm in diameter, irregular or ill‐defined border, irregular pigmentation)
Flow and timing	Exclusions from analysis: None Time interval to reference test: NR
Comparative
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	Yes
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
Was an adequate spectrum of cases used to train the algorithm?	Unclear
		High	High
DOMAIN 2: Index Test Computer‐assisted diagnosis
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?
Was the CAD model evaluated in an independent study population?	Yes
Was model overfitting accounted for during model development?
Was the diagnostic threshold to determine presence or absence of disease established in a previously published study?	No
		High	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Was the use of expert opinion (with no histological confirmation) avoided as the reference standard?	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
Were the reference standard results likely to correctly classify the target condition (disease negative)?	Yes
		Unclear	Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
		Unclear