Item	Response (delete as required)
PARTICIPANT SELECTION (1) ‐ RISK OF BIAS
1) Was a consecutive or random sample of participants or images enrolled?	Yes – if paper states consecutive or random No – if paper describes other method of sampling Unclear – if participant sampling not described
2) Was a case‐control design avoided?	Yes – if consecutive or random or case‐control design clearly not used No – if study described as case‐control or describes sampling specific numbers of participants with particular diagnoses Unclear – if not described
3) Did the study avoid inappropriate exclusions, e.g. 'difficult to diagnose' lesions not excluded lesions not excluded on basis of disagreement between evaluators	Yes ‐ if inappropriate exclusions were avoided No – if lesions were excluded that might affect test accuracy, e.g. 'difficult to diagnose' lesions, or where disagreement between evaluators was observed Unclear – if not clearly reported but there is suspicion that difficult to diagnose lesions may have been excluded
4) For between‐person comparative studies only (i.e. allocating different tests to different study participants): A) were the same participant selection criteria used for those allocated to each test? B) was the potential for biased allocation between tests avoided through adequate generation of a randomised sequence? C) was the potential for biased allocation between tests avoided through concealment of allocation prior to assignment?	For A) Yes – if same selection criteria were used for each index test, No – if different selection criteria were used for each index test, Unclear – if selection criteria per test were not described, NA – if only 1 index test was evaluated or all participants received all tests For B) Yes – if adequate randomisation procedures are described, No – if inadequate randomisation procedures are described, Unclear – if the method of allocation to groups is not described (a description of 'random' or 'randomised' is insufficient), NA – if only 1 index test was evaluated or all participants received all tests For C) Yes – if appropriate methods of allocation concealment are described, No – if appropriate methods of allocation concealment are not described, Unclear – if the method of allocation concealment is not described (sufficient detail to allow a definite judgement is required), NA – if only 1 index test was evaluated
Could the selection of participants have introduced bias? For non‐comparative and within‐person‐comparative studies If answers to all of questions 1), 2), and 3) 'Yes' If answers to any 1 of questions 1), 2), or 3) 'No' If answers to any 1 of questions 1), 2), or 3) 'Unclear' For between‐person comparative studies If answers to all of questions 1), 2), 3), and 4) 'Yes' If answers to any 1 of questions 1), 2), 3), or 4) 'No' If answers to any 1 of questions 1), 2), 3), or 4) 'Unclear'	For non‐comparative and within‐person‐comparative studies Risk is low Risk is high Risk unclear For between‐person comparative studies Risk is low Risk is high Risk unclear
PARTICIPANT SELECTION (1) ‐ CONCERNS REGARDING APPLICABILITY
1) Are the included participants and chosen study setting appropriate to answer the review question, i.e. are the study results generalisable? This item is not asking whether exclusion of certain participant groups might bias the study's results (as in Risk of Bias above), but is asking whether the chosen study participants and setting are appropriate to answer our review question. Because we are looking to establish test accuracy in both primary presentation and referred participants, a study could be appropriate for 1 setting and not for the other, or it could be unclear as to whether the study can appropriately answer either question For each study assessed, please consider whether it is more relevant for A) participants with a primary presentation of a skin lesion or B) referred participants, and respond to the questions in either A) or B) accordingly. If the study gives insufficient details, please respond Unclear to both parts of the question	A) For studies that will contribute to the analysis of participants with a primary presentation of a skin lesion (i.e. test naive) Yes – if participants included in the study appear to be generally representative of those who might present in a usual practice setting No – if study participants appear to be unrepresentative of usual practice, e.g. in terms of severity of disease, demographic features, presence of differential diagnosis or comorbidity, setting of the study, and previous testing protocols Unclear – if insufficient details are provided to determine the generalisability of study participants B) For studies that will contribute to the analysis of referred participants (i.e. who have already undergone some form of testing) Yes – if study participants appear to be representative of those who might be referred for further investigation. If the study focuses only on those with equivocal lesions, for example, we would suggest that this is not representative of the wider referred population No – if study participants appear to be unrepresentative of usual practice, e.g. if a particularly high proportion of participants have been self‐referred or referred for cosmetic reasons. Other factors to consider include severity of disease, demographic features, presence of differential diagnosis or comorbidity, setting of the study, and previous testing protocols Unclear – if insufficient details are provided to determine the generalisability of study participants
2) Did the study avoid including participants with multiple lesions?	Yes – if the difference between the number of included lesions and number of included participants is less than 5% No – if the difference between the number of included lesions and number of included participants is greater than 5% Unclear – if it is not possible to assess
Is there concern that the included participants do not match the review question? If the answer to question 1) or 2) 'Yes' If the answer to question 1) or 2) 'No' If the answer to question 1) or 2) 'Unclear'	Concern is low Concern is high Concern is unclear
INDEX TEST (2) ‐ RISK OF BIAS (to be completed per test evaluated)
1) Was the index test or testing strategy result interpreted without knowledge of the results of the reference standard?	Yes – if index test described as interpreted without knowledge of reference standard result or, for prospective studies, if index test is always conducted and interpreted prior to the reference standard No – if index test described as interpreted in knowledge of reference standard result Unclear – if index test blinding is not described
2) Was the diagnostic threshold at which the test was considered positive (i.e. melanoma present) prespecified?	Yes – if threshold was prespecified (i.e. prior to analysing study results) No – if threshold was not prespecified Unclear – if not possible to tell whether or not diagnostic threshold was prespecified
3) For within‐person comparisons of index tests or testing strategies (i.e. > 1 index test applied per participant): was each index test result interpreted without knowledge of the results of other index tests or testing strategies?	Yes – if all index tests were described as interpreted without knowledge of the results of the others No – if the index tests were described as interpreted in the knowledge of the results of the others Unclear – if it is not possible to tell whether knowledge of other index tests could have influenced test interpretation NA – if only 1 index test was evaluated
Could the conduct or interpretation of the index test have introduced bias? For non‐comparative and between‐person comparison studies If answers to questions 1) and 2) 'Yes' If answers to either questions 1) or 2) 'No' If answers to either questions 1) or 2) 'Unclear' For within‐person comparative studies If answers to all questions 1), 2), for any index test and 3) 'Yes' If answers to any 1 of questions 1) or 2) for any index test or 3) 'No' If answers to any 1 of questions 1) or 2) for any index test or 3) 'Unclear'	For non‐comparative and between‐person comparison studies Risk is low Risk is high Risk is unclear For within‐person comparative studies Risk is low Risk is high Risk is unclear
INDEX TEST (2) ‐ CONCERN ABOUT APPLICABILITY
1) Was the diagnostic threshold to determine presence or absence of disease established in a previously published study? e.g. previously evaluated/established algorithm/checklist used lesion characteristics indicative of melanoma used objective (usually numerical) threshold used	Yes – if a previously evaluated/established tool to aid diagnosis of melanoma was used or if the diagnostic threshold used was established in a previously published study No – if an unfamiliar/new tool to aid diagnosis of melanoma was used, if no particular algorithm was used, or if the objective threshold reported was chosen based on results in the current study Unclear – if insufficient information was reported
2) Were thresholds or criteria for diagnosis reported in sufficient detail to allow replication? Study results can only be reproduced if the diagnostic threshold is described in sufficient detail. This item applies equally to studies using pattern recognition and those using checklists or algorithms to aid test interpretation	Yes – if the criteria for diagnosis of melanoma were reported in sufficient detail to allow replication No – if the criteria for diagnosis of melanoma were not reported in sufficient detail to allow replication Unclear – if some but not sufficient information on criteria for diagnosis to allow replication were provided
3) Was the test interpretation carried out by an experienced examiner?	Yes – if the test was interpreted by 1 or more speciality‐accredited dermatologists, or by examiners of any clinical background with special interest in dermatology and with any formal training in the use of the test No – if the test was not interpreted by an experienced examiner (see above) Unclear – if the experience of the examiner(s) was not reported in sufficient detail to judge or if examiners were described as 'Expert' with no further detail given NA – if artificial intelligence‐based diagnosis, i.e. no observer interpretation
Is there concern that the index test, its conduct, or interpretation differ from the review question? If answers to questions 1), 2), and 3) 'Yes' If answers to questions 1), 2), or 3) 'No' If answers to questions 1), 2), or 3) 'Unclear'	Concern is low Concern is high Concern is unclear
REFERENCE STANDARD (3) ‐ RISK OF BIAS
1) Is the reference standard likely to correctly classify the target condition? A) Disease‐positive – 1 or more of the following: histological confirmation of melanoma following biopsy or lesion excision clinical follow‐up of benign‐appearing lesions for at least 3 months following the application of the index test, leading to a histological diagnosis of melanoma B) Disease‐negative – 1 or more of the following: histological confirmation of absence of melanoma following biopsy or lesion excision in at least 80% of disease‐negative participants clinical follow‐up of benign‐appearing lesions for a minimum of 3 months following the index test in up to 20% of disease‐negative participants	A) Disease‐positive Yes – if all participants with a final diagnosis of melanoma underwent 1 of the listed reference standards No – If a final diagnosis of melanoma for any participant was reached without histopathology Unclear – if the method of final diagnosis was not reported for any participant with a final diagnosis of melanoma or if the length of clinical follow‐up used was not clear or if a clinical follow‐up reference standard was reported in combination with a participant‐based analysis and it was not possible to determine whether the detection of a malignant lesion during follow‐up is the same lesion that originally tested negative on the index test B) Disease‐negative Yes – If at least 80% of benign diagnoses were reached by histology and up to 20% were reached by clinical follow‐up for a minimum of 3 months following the index test No – if more than 20% of benign diagnoses were reached by clinical follow‐up for a minimum of 3 months following the index test or if clinical follow‐up period was less than 3 months Unclear – if the method of final diagnosis was not reported for any participant with benign or non‐melanoma diagnosis
2) Were the reference standard results interpreted without knowledge of the results of the index test? Please score this item for all studies even though histopathology interpretation is usually conducted with knowledge of the clinical diagnosis (from visual inspection or dermoscopy or both). We will deal with this by not including the response to this item in the 'Risk of bias' assessment for these tests. For reviews of all other tests, this item will be retained	Yes – if the reference standard diagnosis was reached blinded to the index test result No – if the reference standard diagnosis was reached with knowledge of the index test result Unclear – if blinded reference test interpretation was not clearly reported
Could the reference standard, its conduct, or its interpretation have introduced bias? For visual inspection/dermoscopy evaluations If answer to question 1) 'Yes' If answer to question 1) 'No' If answer to question 1) 'Unclear' For all other tests If answers to questions 1) and 2) 'Yes' If answers to questions 1) or 2) 'No' If answers to questions 1) or 2) 'Unclear'	For visual inspection/dermoscopy evaluations Risk is low Risk is high Risk is unclear For all other tests Risk is low Risk is high Risk is unclear
REFERENCE STANDARD (3) ‐ CONCERN ABOUT APPLICABILITY
1) Are index test results presented separately for each component of the target condition (i.e. separate results presented for those with invasive melanoma, melanoma in situ, lentigo maligna, severe dysplasia, BCC, and cSCC)?	Yes – if index test results for each component of the target condition can be disaggregated No – if index test results for the different components of the target condition cannot be disaggregated Unclear – if not clearly reported
2) Expert opinion (with no histological confirmation) was not used as a reference standard 'Expert opinion' means diagnosis based on the standard clinical examination, with no histology or lesion follow‐up ***do not complete this item for teledermatology studies	Yes – if expert opinion was not used as a reference standard for any participant No – if expert opinion was used as a reference standard for any participant Unclear – if not clearly reported
3) Was histology interpretation carried out by an experienced histopathologist or by a dermatopathologist?	Yes – if histology interpretation was reported to be carried out by an experienced histopathologist or dermatopathologist No – if histology interpretation was reported to be carried out by a less experienced histopathologist Unclear – if the experience/qualifications of the pathologist were not reported
Is there concern that the target condition as defined by the reference standard does not match the review question? If answers to all questions 1), 2), and 3) 'Yes' If answers to any 1 of questions 1), 2), or 3) 'No' If answers to any 1 of questions 1), 2), or 3) 'Unclear' ***For teledermatology studies only If answers to all questions 1) and 3) 'Yes' If answers to questions 1) or 3) 'No' If answers to questions 1) or 3) 'Unclear'	Concern is low Concern is high Concern is unclear ***For teledermatology studies only Concern is low Concern is high Concern is unclear
FLOW AND TIMING (4):RISK OF BIAS
1) Was there an appropriate interval between index test and reference standard? A) For histopathological reference standard, was the interval between index test and reference standard ≤ 1 month? B) If the reference standard includes clinical follow‐up of borderline/benign‐appearing lesions, was there at least 3 months' follow‐up following application of index test(s)?	A) Yes – if study reports ≤ 1 month between index and reference standard No – if study reports > 1 month between index and reference standard Unclear – if study does not report interval between index and reference standard B) Yes – if study reports ≥ 3 months' follow‐up No – if study reports < 3 months' follow‐up Unclear – if study does not report the length of clinical follow‐up
2) Did all participants receive the same reference standard?	Yes – if all participants underwent the same reference standard No – if more than 1 reference standard was used Unclear – if not clearly reported
3) Were all participants included in the analysis?	Yes – if all participants were included in the analysis No – if some participants were excluded from the analysis Unclear– if not clearly reported
4) For within‐person comparisons of index tests Was the interval between application of index tests ≤ 1 month?	Yes – if study reports ≤ 1 month between index tests No – if study reports > 1 month between index tests Unclear – if study does not report the interval between index tests
Could the participant flow have introduced bias? For non‐comparative and between‐person comparison studies If answers to questions 1), 2), and 3) 'Yes' If answers to any 1 of questions 1), 2), or 3) 'No' If answers to any 1 of questions 1), 2), or 3) 'Unclear' For within‐person comparative studies If answers to all questions 1), 2), 3), and 4) 'Yes' If answers to any 1 of questions 1), 2), 3), or 4) 'No' If answers to any 1 of questions 1), 2), 3), or 4) 'Unclear'	For non‐comparative and between‐person comparison studies Risk is low Risk is high Risk is unclear For within‐person comparative studies Risk is low Risk is high Risk is unclear
BCC = basal cell carcinoma; cSCC = cutaneous squamous cell carcinoma.