Ferris 2015.

Study characteristics
Patient sampling	Study design: unclear. Some dermoscopic images were collected prospectively and some were obtained from collection of existing images; selection process not described Data collection: retrospective image selection/prospective interpretation Period of data collection: NR Country: USA Test set derived: study developed a new CAD classifier using training/test set of images; plus a 'reader study'* conducted to compare accuracy with dermatologist interpretation of images (*reported here). Some dermoscopic images used to train the classifier were obtained from publicly available or purchased image libraries, these were not included in the reader study or used to test the performance of the classifier. The image set was randomly divided into 2 by diagnosis, with half used for training and half used for testing, with the exception that all high‐grade dysplastic naevi were exclusively assigned to the training set to increase the representation of dermoscopic features that could be present in melanoma. Results were extracted only for the test set.
Patient characteristics and setting	Inclusion criteria: dermoscopic images of skin lesions excised on the basis of clinical suspicion of malignancy, with available histologic diagnoses. Reader study included one melanoma that was misclassified as benign by the new CAD classifier + random sample of images determined to be of suitable quality for display on a computer screen. Setting: secondary (general dermatology) Prior testing: clinical suspicion (no further detail) Setting for prior testing: secondary (general dermatology) Exclusion criteria: high‐grade dysplastic naevi were not included in the test set or reader study Sample size (participants): number eligible: NR; number included: NR Sample size (lesions): number eligible: 473 (includes 273 randomised to training set and 27 non‐biopsied lesions); number included: CAD test set 173 lesions; dermoscopy‐ 65 lesions Participant characteristics: none reported Lesion characteristics: test set: mean lesion thickness 0.76 mm, median 0.5 mm, range 0.2‐98 mm); reader study: mean 0.93 mm, median 0.74 mm, range 0.2‐98 mm.
Index tests	Dermoscopy: no algorithm Method of diagnosis: dermoscopic images Prior test data: no further information used Diagnostic threshold: NR Diagnosis based on: average (n = 30); 35 invited to participate. Observer qualifications: 2 board‐certified dermatologists, 10 dermatology residents, and 8 physician assistants currently practicing dermatology Experience in practice: mixed Experience with dermoscopy: mixed; all observers self‐reported some training and experience with the use of dermoscopy. Among board‐certified dermatologists, 67% reported using dermoscopy ‘‘always/almost always’’ or ‘‘very frequently.’', compared to 90% of the dermatology residents and 75% of the physician assistants.
Target condition and reference standard(s)	Reference standard: histological diagnosis alone Details: all lesions were biopsied based on clinical suspicion of malignancy. All histologic diagnoses were rendered by at least 1 board‐certified dermatopathologist and were used as the reference standard for diagnosis Disease‐positive: dermoscopy 25 MM; CAD 39 MM/disease‐negative: dermoscopy 40 MM; CAD 134 MM Target condition (final diagnoses) For reader study only: Invasive melanomas 15; melanoma in situ 10 Low‐grade dysplastic naevi 16, benign naevi 14 , blue naevi 2, lentigines 4 , SK 4
Flow and timing	Excluded participants: none reported Time interval to reference test: "Dermoscopic images of skin lesions were collected before biopsy"
Comparative
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
		Unclear	High
DOMAIN 2: Index Test Dermoscopy ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Unclear
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	No
Was the test interpretation carried out by an experienced examiner?	Unclear
		Unclear	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes
		Low	Low
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?
		Low