Item	Response (delete as required)
Participant selection (1) –risk of bias
1. Was a consecutive or random sample of participants or images enrolled?	Yes – if paper states consecutive or random No – if paper describes other method of sampling Unclear – if participant sampling not described
2. Was a case‐control design avoided? (note: a diagnostic case‐control study separately recruits participants according to selected final diagnoses, e.g. those with melanoma, with BCC, with severe dysplasia and with mild dysplasia AND will usually deliberately sample certain numbers from each group such that the overall case mix of included participants and disease prevalence is not reflective of usual care.)	Yes – if case‐control design clearly not used No – if study described as case‐control or describes sampling specific numbers of participants with particular diagnoses Unclear – if not clearly described or you have any concerns that the authors have not selected a series of participants
3. Did the study avoid inappropriate exclusions, e.g. 'difficult to diagnose' lesions not excluded lesions not excluded on basis of disagreement between evaluators or histopathologists	Yes – if inappropriate exclusions were avoided No – if lesions were excluded that might affect test accuracy, e.g. 'difficult to diagnose' lesions, OR where disagreement between evaluators was observed Unclear – if not clearly reported but there is suspicion that difficult to diagnose lesions may have been excluded
Could the selection of participants have introduced bias? If answers to all of questions 1. AND 2. AND 3. 'Yes' If answers to any 1 of questions 1. OR 2. OR 3. 'No' If answers to any 1 of questions 1. OR 2. OR 3. 'Unclear'	Risk is low Risk is high Risk is unclear
Participant selection (1) –concerns regarding applicability
1. Are the included participants and chosen study setting appropriate to answer the review question, i.e. are the study results generalisable? This item is not asking whether exclusion of certain participant groups might bias the study's results (as in 'Risk of bias' above), but is asking whether the chosen study participants and setting are appropriate to answer our review question. Because we are looking to establish test accuracy in both primary presentation and referred participants, a study could be appropriate for 1 setting and not for the other, or it could be unclear as to whether the study can appropriately answer either question.	Yes – if participants included in the study appear to be generally representative of those who might present in a usual practice setting No – if study participants were restricted to those in lesion subgroups, e.g. melanocytic only, or small lesions only, if only excised lesions were included, or lesions were selected from referred populations rather than selected by general practitioners in a primary care setting Unclear – if insufficient details are provided to determine the generalisability of study participants
2. Did the study avoid including participants with multiple lesions?	Yes – if the difference between the number of included lesions and number of included participants is less than 5% No – if the difference between the number of included lesions and number of included participants is greater than 5% Unclear – if it is not possible to assess
Is there concern that the included participants do not match the review question? If the answer to question 1. and 2. 'Yes' If the answer to question 1. or 2. 'No' If the answer to question 1. or 2. 'Unclear'	Concern is low Concern is high Concern is unclear
Index test (2) –risk of bias (to be completed per test evaluated)
1. Was the index test or testing strategy result interpreted without knowledge of the results of the reference standard?	Yes – if index test described as interpreted without knowledge of the reference standard result or, for prospective studies, if index test is always conducted and interpreted prior to the reference standard No – if index test described as interpreted in knowledge of reference standard result Unclear – if index test blinding is not described
2. Was the diagnostic threshold at which the test was considered positive (i.e. melanoma, BCC or cSCC present) prespecified?	Yes – if threshold was prespecified (i.e. prior to analysing study results), i.e. results were not data driven No – if threshold was not prespecified but was selected after analysis of results usually to maximise sensitivity or specificity (or both), or multiple thresholds were tested Unclear – if not possible to tell whether or not diagnostic threshold was prespecified
Could the conduct or interpretation of the index test have introduced bias? For NC and BPC studies If answers to questions 1. and 2. 'Yes' If answers to either questions 1. or 2. 'No' If answers to either questions 1). or 2. 'Unclear' For WPC studies If answers to all questions 'Yes' If answers to any 1 question 'No' If answers to any 1 of questions is 'Unclear'	For NC and BPC studies Risk is low Risk is high Risk is unclear For WPC studies Risk is low Risk is high Risk is unclear
Index test (2) –concern about applicability
1. Was the test applied and interpreted in a clinically applicable manner?	Yes – in‐person evaluation and single observer result present No – image based, or mean or consensus result presented, or both Unclear – if cannot tell
2. Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication? Study results can only be reproduced if the diagnostic threshold is described in sufficient detail. This item applies equally to studies using pattern recognition and those using checklists or algorithms to aid test interpretation	Yes – if the criteria for diagnosis of the target disorder were reported in sufficient detail to allow replication. If the study does not describe the threshold in detail BUT evaluates an established test/algorithm AND provides a citation to a previous study of the test in the Methods or Results, then respond Yes. No – if the criteria for diagnosis of the target disorder were not reported in sufficient detail to allow replication. Unclear – if some but not sufficient information on criteria for diagnosis to allow replication were provided. If the study does not describe the threshold in detail BUT evaluates an established test/algorithm but with NO citation to a previous study of the test in the methods, then respond Unclear.
3) Was the test interpretation carried out by an experienced examiner?	Yes – if the test was interpreted by ≥ 1 speciality accredited dermatologists, or by examiners of any clinical background with special interest in dermatology and with any formal training in the use of the test No – if the test was not interpreted by an experienced examiner (see above) Unclear – if the experience of the examiner(s) was not reported in sufficient detail to judge OR if examiners described as 'Expert' with no further detail given N/A – if system‐based diagnosis, i.e. no observer interpretation
Is there concern that the index test, its conduct, or interpretation differ from the review question? If answers to questions 1., 2. AND 3. 'Yes' If answers to questions 1., 2. OR 3. 'No' If answers to questions 1., 2. OR 3. 'Unclear'	Concern is low Concern is high Concern is unclear
Reference standard (3) –risk of bias
1). Is the reference standard likely to correctly classify the target condition? a) Disease positive – ≥ 1 of: Histological confirmation of the target disorder following biopsy or lesion excision Clinical follow‐up of benign appearing lesions following the application of the index test, leading to a histological diagnosis of the target disorder of interest: ≥ 3 months for melanoma or cSCC ≥ 6 months for BCC b) Disease negative – ≥ 1 of: Histological confirmation of absence of the target disorder following biopsy or lesion excision in ≥ 80% of disease‐negative participants Clinical follow up of benign appearing lesions following the index test in up to 20% of disease negative participants of: ≥ 3 months for melanoma or cSCC ≥ 6 months for BCC	a) Disease positive Yes – if all disease‐positive participants underwent 1 of the listed reference standards No – if a final diagnosis for any disease‐positive participant was reached without histopathology Unclear – if the method of final diagnosis was not reported for any disease‐positive participant OR if the length of clinical follow‐up used was not clear OR if a clinical follow‐up reference standard was reported in combination with a participant‐based analysis and it was not possible to determine whether the detection of a malignant lesion during follow‐up is the same lesion that originally tested negative on the index test a) Disease negative Yes – if ≥ 80% of benign diagnoses were reached by histology and up to 20% were reached by clinical follow‐up of for a minimum of 3 (or 6) months following the index test No – if > 20% of benign diagnoses were reached by clinical follow‐up of a minimum of 3 (or 6) months following the index test OR if clinical follow‐up period was less than 3 (or 6) months Unclear – if the method of final diagnosis was not reported for any participant with benign or disease‐negative diagnosis
2. Were the reference standard results interpreted without knowledge of the results of the index test? Please score this item for all studies. Response to the item will not be incorporated into the overall risk of bias assessment for comparisons against a histological reference standard as histopathology interpretation is usually conducted with knowledge of the clinical diagnosis (from visual inspection or dermoscopy (or both)). Response to the item will be incorporated into the overall risk of bias assessment for comparisons against a face‐to‐face reference standard	For studies comparing teledermatology against a histological reference standard Yes – if the histological reference standard diagnosis was reached blinded to the index test result No – if the histological reference standard diagnosis was reached with knowledge of the index test result Unclear – if blinded reference test interpretation was not clearly reported If the histopathologist is described as 'blinded' with no further detail as to whether the blinding applies to both index test or to clinical information (prior testing), we will assume that blinding is to the index test result only, unless further detail is provided For studies comparing teledermatology against a face‐to‐face expert diagnosis Yes – if the face‐to‐face reference standard diagnosis was described as interpreted without knowledge of the teledermatology diagnosis (e.g. the remote and face‐to‐face diagnosis was made by 2 different dermatologists) No – if the face‐to‐face reference standard diagnosis was made with knowledge of the teledermatology diagnosis or was made by the same dermatologist within a month of the remote image‐based diagnosis Unclear – if it is not possible to tell whether knowledge of the teledermatology diagnosis could have influenced the reference standard diagnosis
Could the reference standard, its conduct, or its interpretation have introduced bias? For comparisons against a histological reference standard: If answer to question 1. 'Yes' If answer to question 1. 'No' If answer to question 1. 'Unclear' For comparisons against a face‐to‐face reference standard: If answers to questions 1. AND 2. 'Yes' If answers to questions 1. OR 2. 'No' If answers to questions 1. OR 2. 'Unclear'	For comparisons against a histological reference standard: Risk is low Risk is high Risk is unclear For comparisons against a face‐to‐face reference standard: Risk is low Risk is high Risk is unclear
Reference standard (3) –concern about applicability
1. For studies comparing teledermatology/face‐to‐face clinical diagnosis to histology, was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes – if histology interpretation was reported to be carried out by an experienced histopathologist or dermatopathologist No – if histology interpretation was reported to be carried out by a less‐experienced histopathologist Unclear – if the experience/qualifications of the pathologist were not reported
2. For studies comparing teledermatology to face‐to‐face diagnosis, was the clinical diagnosis carried out by an experienced observer?	Yes – if face‐to‐face interpretation was reported to be carried out by an experienced dermatologist No – if face‐to‐face interpretation was reported to be carried out by a less‐experienced dermatologist Unclear – if the experience/qualifications of the face‐to‐face clinician were not reported
Is there concern that the target condition as defined by the reference standard does not match the review question? If answer to either questions 1. or 2. 'Yes' If answer to either questions 1. OR 2. 'No' If answer to either questions 1. OR 2. 'Unclear'	Concern is low Concern is high Concern is unclear
Flow and timing (4): risk of bias
1. Was there an appropriate interval between index test and reference standard? a) For histopathological reference standard, was the interval between index test and reference standard ≤ 1month? b) If the reference standard includes clinical follow‐up of borderline/benign appearing lesions, was there a minimum follow‐up following application of index test(s) of: ≥ 3 months for melanoma or cSCC ≥ 6 months for BCC?	a) Yes – if study reports ≤ 1 month between index and reference standard No – if study reports > 1 month between index and reference standard Unclear – if study does not report interval between index and reference standard b) Yes – if study reports ≥ 3 (or 6) months follow‐up No – if study reports < 3 (or 6) months follow‐up Unclear – if study does not report length of clinical follow‐up
2. Did all participants receive the same reference standard?	Yes – if all participants underwent the same reference standard No – if > 1 reference standard was used Unclear – if not clearly reported
3. Were all participants included in the analysis?	Yes – if all participants were included in the analysis No – if some participants were excluded from the analysis Unclear – if not clearly reported
Could the participant flow have introduced bias? For NC and BPC studies If answers to questions 1. AND 2. AND 3. 'Yes' If answers to any 1 of questions 1. OR 2. OR 3. 'No' If answers to any 1 of questions 1. OR 2. OR 3. 'Unclear'	Risk is low Risk is high Risk is unclear
Comparative domain: for BPC or WPC (or both) of index tests or testing strategies (i.e. > 1 index test applied per participant)
Index tests
1. Was each index test result interpreted without knowledge of the results of other index tests or testing strategies?	Yes – if all index tests were described as interpreted without knowledge of the results of the others No – if the index tests were described as interpreted in the knowledge of the results of the others Unclear – if it is not possible to tell whether knowledge of other index tests could have influenced test interpretation N/A – if only 1 index test was evaluated
2. Was the interval between application of index tests ≤ 1 month?	Yes – if study reports ≤ 1 month between index tests No – if study reports > 1 month between index tests Unclear – if study does not report interval between index tests
Clinical applicability of comparison 1. Were both tests applied and interpreted in a clinically applicable manner?	—