Wolf 2013.

Study characteristics
Patient sampling	Study design: case control study Data collection: retrospective image selection/prospective interpretation Period of data collection: not reported Country: USA
Patient characteristics and setting	Inclusion criteria: lesion images selected from the institution image database with specific and clear histologic diagnoses including: melanoma, melanoma in situ, lentigo, benign naevi (including compound, junctional and low grade dysplastic naevi), dermatofibroma, sebhorrhoeic keratosis and haemangioma Setting: unspecified Prior testing: selected for excision (no further detail) Setting for prior testing: unspecified Exclusion criteria: Poor quality index test image Images that contained any identifiable features, such as facial features, tattoos, or labels with patient information, were excluded or cropped to remove the identifiable features or information Lesions with specific diagnoses including: Spitz naevi, Reed nevus, uncommon or equivocal lesions; and lesions with moderate or high‐grade atypia Sample size (patients): not reported Sample size (lesions): no. eligible 390; no. included 188 Participant characteristics: not reported Lesion characteristics: not reported Other: (note: Von Braunmühl 2015 extrapolates sensitivity and specificity to the whole dataset of 195 lesions by including the poor quality index test images ‐ excluded as overlapping populations with Maier 2015)
Index tests	1. Mobile phone application Acquisition and transmission of images: secondary care Nature of images used: not reported Any additional patient information provided: no further information used Diagnostic threshold: Application 1. The application analyses the image and gives an assessment of 'problematic' (positive test result) or 'okay' (negative test result) Application 2. The output given is 'melanoma' (positive test result) or 'looks good' (negative test result). Application 3. The output given is 'high risk' (positive test result) or 'medium risk' or 'low risk,' both of which we considered to be a negative test result. Application 4. The dermatologist assigns an output of 'atypical' (positive test result) or 'typical' (negative test result); images classified as 'send another photograph' or 'unable to categorise' were considered test failures and excluded by study authors. Observer qualifications (remote diagnosis): application 4 only: images interpreted by a board‐certified dermatologist (n = not reported)
Target condition and reference standard(s)	Reference standard: histological diagnosis alone Details: reference standard details Histology (not further described) – No. patients/lesions: a total of 188 lesions – disease positive: 60 melanomas – disease negative: 128 benign Target condition (final diagnoses) Malignant – melanoma (in situ and invasive): 60 Benign – 'Benign' diagnoses: 128
Flow and timing	1. Excluded participants: 202/390 lesion images excluded due to poor image quality, containing identifiable patient information or features, or lacking sufficient clinical or histological information. Between 3 and 29 additional lesions were analysed by the applications but considered unevaluable or test failures. The test failure rates were: 3% (n = 6; App 1), 2% (n = 3; App 2), 6% (n = 12; App 3) and 15% (n = 29; App 4). 2. Time interval to reference test: NA
Comparative
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	No
Did the study avoid inappropriate exclusions?	No
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Unclear
		High	High
DOMAIN 2: Index Test Index test
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
		Low	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Were the reference standard results interpreted without knowledge of the referral diagnosis?	Unclear
Expert opinion (with no histological confirmation) was not used as a reference standard	Yes
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes
		Low	Low
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?
		High

NA: not applicable.