DOMAIN |
Low risk/concern |
Unclear |
High risk/concern |
PATIENT SELECTION |
Describe methods of patient selection; describe included patients (prior testing, presentation, intended use of index test and setting): |
Was a consecutive or random sample of patients enroled? |
Consecutive sampling or random sampling seeking refractive error correction or refractive surgery in eye services. |
Unclear whether consecutive or random sampling used. |
Selection of non‐consecutive patients. |
Was a case‐control design avoided? |
No selective recruitment of people with or without keratoconus. |
Unclear selection mechanism. |
Selection of either cases or control in a predetermined, non‐random fashion; or enrichment of the cases from a selected population. |
Did the study avoid inappropriate exclusions? |
Exclusions are detailed and felt to be appropriate (e.g. people already diagnosed with keratoconus or with other corneal diseases). |
Exclusions are not detailed (pending contact with study authors). |
Inappropriate exclusions are reported (e.g. of people with borderline index test results). |
Risk of bias: could the selection of patients have introduced bias? |
'No' for any of the above |
Concerns regarding applicability: are there concerns that the included patients do not match the review question? |
Inclusion of patients seeking refractive error correction or refractive surgery in primary or secondary care eye services. |
Unclear inclusion criteria. |
Inclusion of patients attending cornea services for known disease, population‐based studies, registry‐based studies. |
INDEX TEST |
Describe the index test and how it was conducted and interpreted: |
Were the index test results interpreted without knowledge of the results of the reference standard? |
Test performed "blind" or "independently and without knowledge of" reference standard results are sufficient and full details of the blinding procedure are not required; or clear temporal pattern to the order of testing that precludes the need for formal blinding. |
Unclear whether results are interpreted independently. |
Reference standard results available to those who conducted or interpreted the index test. |
If a threshold was used, was it prespecified? |
The study authors declare that the selected cut‐off used to dichotomize data was specified a priori, or a protocol is available with this information. |
No information on preselection of index test cut‐off values. |
A study is classified at higher risk of bias if the authors define the optimal cut‐off post hoc based on their own study data. |
Risk of bias: could the conduct or interpretation of the index test have introduced bias? |
'No' for any of the above. |
Concerns regarding applicability: are there concerns that the index test, its conduct, or interpretation differ from the review question? |
Tests used and testing procedure clearly reported and tests executed by personnel with sufficient training. |
Unclear execution of the tests or unclear study personnel profile, background, and training. |
Tests used are not validated, or study personnel is insufficiently trained. |
REFERENCE STANDARD |
Describe the reference standard and how it was conducted and interpreted: |
Is the reference standard likely to correctly classify the target condition? |
Topography and/or tomography interpreted independently by 2 or more cornea specialists. |
Topography and/or tomography interpreted by cornea specialists, but not enough details to adjudicate 'yes' or 'no'. |
Topography and/or tomography interpreted by only one cornea specialist. |
Were the reference standard results interpreted without knowledge of the results of the index test? |
Reference standard performed "blind" or "independently and without knowledge of" index test results are sufficient and full details of the blinding procedure are not required; or clear temporal pattern to the order of testing that precludes the need for formal blinding. |
Unclear whether results are interpreted independently. |
Index test results available to those who conducted the reference standard. |
Risk of bias: could the reference standard, its conduct, or its interpretation have introduced bias? |
'No' for any of the above. |
Concerns regarding applicability: are there concerns that the target condition as defined by the reference standard does not match the review question? |
Same or similar definition of the target condition as described in the protocol. |
Unclear definition of the target disease diagnosed by the reference standard. |
Different definition of the target condition as defined in the protocol. |
FLOW AND TIMING |
Describe any patients who did not receive the index test(s) and/or reference standard or who were excluded from the 2 × 2 table (refer to flow diagram): describe the time interval and any interventions between index test(s) and reference standard. |
Was there an appropriate interval between index test(s) and reference standard? |
No more than three months between index and reference test execution. |
— |
More than three months between index and reference test execution. |
Did all patients receive a reference standard? |
All participants receiving the index test are verified with the reference standard. |
— |
Not all participants receiving the index test are verified with the reference standard. |
Did all patients receive the same reference standard? |
Not applicable for this review. |
Were all patients included in the analysis? |
The number of participants included in the study matches the number in analyses, or participants with undefined or borderline test results are excluded. |
— |
The number of participants included in the study does not match the number in analyses, or participants with undefined or borderline test results are excluded. |
Risk of bias: could the patient flow have introduced bias? |
'No' for any of the above, |
ADDITIONAL QUESTIONS |
These questions concern the direct comparisons between AI tests, |
Were different AI tests developed and interpreted without knowledge of each other? |
Different AI tests were developed and interpreted "blind" or "independently and without knowledge of" the results of each other. |
— |
Different AI tests were developed or their results interpreted with knowledge of the results of each other. |
Are the proportions and reasons for missing data similar for all index tests? |
Missing data and their causes were similar for each AI test. |
— |
The amount of missing data or their causes differed between AI tests. |