Bourne 2012.

Study characteristics
Patient sampling	Study design: case series Data collection: retrospective image selection/prospective interpretation Period of data collection: June 1‐July 6 2009 Country: Australia
Patient characteristics and setting	Inclusion criteria: all skin lesions consecutively excised at a skin cancer practice to exclude skin cancer and common lesions assessed as clearly benign and not biopsied were included Setting: private; "a dedicated skin cancer practice in Brisbane, Australia" Prior testing: clinical and/or dermatoscopic suspicion. Prior testing to assemble the test set occurs in secondary care by an experienced skin cancer doctor, then the images are tested on primary care professionals Setting for prior testing: specialist unit (skin cancer/PLC) Exclusion criteria: clinically obvious BCCs, which could be easily diagnosed without dermoscopy were not included in the collection set Sample size (participants): number eligible: 46/number included: 46 Sample size (lesions): number eligible: 50/number included: 50 Participant characteristics: mean age: 58y (30‐60y); male: 22 Lesion characteristics: face = 8; neck = 1; chest = 3; back = 21; shoulder = 2; arm = 3; thigh = 4; leg = 7; foot plantar = 1
Index tests	VI: no algorithm Method of diagnosis: clinical photographs Prior test data: no further information used; image assessments were done on 4 occasions, each time using a different diagnostic approach. Diagnostic threshold: NR clinicians provided with Excel answer sheets for each method listing the various criteria used in that algorithm but no algorithm was cited for VI Diagnosis based on: average (n = 4) Observer qualifications: 3 GPs and 1 clinical nurse Experience in practice: mixed; described as varying levels of dermatoscopic experience Experience with dermoscopy: mixed; described as varying levels of dermatoscopic experience Dermoscopy 3‐point rule; Menzies criteria Method of diagnosis: dermoscopic images Prior test data: no further information used; image assessments were done on 4 occasions, each time using a different diagnostic approach. Diagnostic threshold: NR in paper; author communications states that standard thresholds were used, ≥ 2 for the 3PCL and Menzies method as described in original paper Test observers as described for VI (above)
Target condition and reference standard(s)	Reference standard: histological diagnosis + other Histopathological examination (n = 46); expert diagnosis as benign (n = 3); digital follow‐up (n = 1) Target condition (final diagnoses) Melanoma (invasive): 1; melanoma (in situ): 7; BCC: 6; lentigo maligna 1 Sebhorrheic keratosis: 5; 'Benign' diagnoses: banal naevus 10, blue naevus 1, naevus and SK/solar lentigo collision 3, solar lentigo 4, LP or LK 4, DF 1, psoriasis 1, solar keratosis 2, intraepidermal carcinoma 3, regressed keratoacanthoma 1
Flow and timing	Excluded participants: as 2 of the methods (Menzies and 3PCL) related to only pigmented lesions, the 5 non‐pigmented specimens in the set of 50 were excluded from the contingency tables for these methods. Time interval to reference test: "all skin lesions consecutively excised to exclude skin cancer were recorded"
Comparative	Blinding between tests: image assessments on 4 different occasions different algorithms Time interval between index test(s): same day; images acquired at time of face‐to‐face consultation
Notes	‐
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Are the included patients and chosen study setting appropriate?	No
Did the study avoid including participants with multiple lesions?	Yes
		Low	High
DOMAIN 2: Index Test Visual inspection ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Unclear
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	No
Was the test interpretation carried out by an experienced examiner?	Unclear
		Unclear	High
DOMAIN 2: Index Test Dermoscopy ‐ image‐based
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
For studies reporting the accuracy of multiple diagnostic thresholds, was each threshold or algorithm interpreted without knowledge of the results of the others?	Unclear
Was the test applied and interpreted in a clinically applicable manner?	No
Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication?	Yes
Was the test interpretation carried out by an experienced examiner?	Unclear
		Low	High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	No
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Expert opinion (with no histological confirmation) was not used as a reference standard	No
Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Unclear
		High	High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	No
If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of at least: 3 months for melanoma or cSCC or 6 months for BCC?	Unclear
If more than one algorithm was evaluated for the same test, was the interval between application of the different algorithms 1 month or less?	Yes
		High
DOMAIN 5: Comparative
Was each index test result interpreted without knowledge of the results of other index tests or testing strategies?	Yes
Was the interval between application of the index tests less than one month?	Yes
Were all tests applied and interpreted in a clinically applicable manner?	No
		Low	High