| DOMAIN | Low risk/concern | Unclear | High risk/concern |
| PATIENT SELECTION | Describe methods of patient selection; describe included patients (prior testing, presentation, intended use of index test and setting): | ||
| Was a consecutive or random sample of patients enroled? | Consecutive sampling or random sampling seeking refractive error correction or refractive surgery in eye services. | Unclear whether consecutive or random sampling used. | Selection of non‐consecutive patients. |
| Was a case‐control design avoided? | No selective recruitment of people with or without keratoconus. | Unclear selection mechanism. | Selection of either cases or control in a predetermined, non‐random fashion; or enrichment of the cases from a selected population. |
| Did the study avoid inappropriate exclusions? | Exclusions are detailed and felt to be appropriate (e.g. people already diagnosed with keratoconus or with other corneal diseases). | Exclusions are not detailed (pending contact with study authors). | Inappropriate exclusions are reported (e.g. of people with borderline index test results). |
| Risk of bias: could the selection of patients have introduced bias? | 'No' for any of the above | ||
| Concerns regarding applicability: are there concerns that the included patients do not match the review question? | Inclusion of patients seeking refractive error correction or refractive surgery in primary or secondary care eye services. | Unclear inclusion criteria. | Inclusion of patients attending cornea services for known disease, population‐based studies, registry‐based studies. |
| INDEX TEST | Describe the index test and how it was conducted and interpreted: | ||
| Were the index test results interpreted without knowledge of the results of the reference standard? | Test performed "blind" or "independently and without knowledge of" reference standard results are sufficient and full details of the blinding procedure are not required; or clear temporal pattern to the order of testing that precludes the need for formal blinding. | Unclear whether results are interpreted independently. | Reference standard results available to those who conducted or interpreted the index test. |
| If a threshold was used, was it prespecified? | The study authors declare that the selected cut‐off used to dichotomize data was specified a priori, or a protocol is available with this information. | No information on preselection of index test cut‐off values. | A study is classified at higher risk of bias if the authors define the optimal cut‐off post hoc based on their own study data. |
| Risk of bias: could the conduct or interpretation of the index test have introduced bias? | 'No' for any of the above. | ||
| Concerns regarding applicability: are there concerns that the index test, its conduct, or interpretation differ from the review question? | Tests used and testing procedure clearly reported and tests executed by personnel with sufficient training. | Unclear execution of the tests or unclear study personnel profile, background, and training. | Tests used are not validated, or study personnel is insufficiently trained. |
| REFERENCE STANDARD | Describe the reference standard and how it was conducted and interpreted: | ||
| Is the reference standard likely to correctly classify the target condition? | Topography and/or tomography interpreted independently by 2 or more cornea specialists. | Topography and/or tomography interpreted by cornea specialists, but not enough details to adjudicate 'yes' or 'no'. | Topography and/or tomography interpreted by only one cornea specialist. |
| Were the reference standard results interpreted without knowledge of the results of the index test? | Reference standard performed "blind" or "independently and without knowledge of" index test results are sufficient and full details of the blinding procedure are not required; or clear temporal pattern to the order of testing that precludes the need for formal blinding. | Unclear whether results are interpreted independently. | Index test results available to those who conducted the reference standard. |
| Risk of bias: could the reference standard, its conduct, or its interpretation have introduced bias? | 'No' for any of the above. | ||
| Concerns regarding applicability: are there concerns that the target condition as defined by the reference standard does not match the review question? | Same or similar definition of the target condition as described in the protocol. | Unclear definition of the target disease diagnosed by the reference standard. | Different definition of the target condition as defined in the protocol. |
| FLOW AND TIMING | Describe any patients who did not receive the index test(s) and/or reference standard or who were excluded from the 2 × 2 table (refer to flow diagram): describe the time interval and any interventions between index test(s) and reference standard. | ||
| Was there an appropriate interval between index test(s) and reference standard? | No more than three months between index and reference test execution. | — | More than three months between index and reference test execution. |
| Did all patients receive a reference standard? | All participants receiving the index test are verified with the reference standard. | — | Not all participants receiving the index test are verified with the reference standard. |
| Did all patients receive the same reference standard? | Not applicable for this review. | ||
| Were all patients included in the analysis? | The number of participants included in the study matches the number in analyses, or participants with undefined or borderline test results are excluded. | — | The number of participants included in the study does not match the number in analyses, or participants with undefined or borderline test results are excluded. |
| Risk of bias: could the patient flow have introduced bias? | 'No' for any of the above, | ||
| ADDITIONAL QUESTIONS | These questions concern the direct comparisons between AI tests, | ||
| Were different AI tests developed and interpreted without knowledge of each other? | Different AI tests were developed and interpreted "blind" or "independently and without knowledge of" the results of each other. | — | Different AI tests were developed or their results interpreted with knowledge of the results of each other. |
| Are the proportions and reasons for missing data similar for all index tests? | Missing data and their causes were similar for each AI test. | — | The amount of missing data or their causes differed between AI tests. |