DOMAIN: Participant selection | |
Was a consecutive or random sample of patients enrolled? | This will be similar for all index tests, target conditions, and populations. Yes: if a study explicitly stated that all participants within a certain time frame were included; that this was done consecutively; or that a random selection was done. No: if it was clear that a different selection procedure was employed; for example, selection based on clinician's preference, or based on institutions, or based on result of RT‐PCR Unclear: if the selection procedure was not clear or not reported |
Was a case‐control design avoided? | This will be similar for all index tests, target conditions, and populations. Yes: if a study explicitly stated that all participants came from the same group of (suspected) patients. No: if it was clear that a different selection procedure was employed for the participants depending on their COVID‐19 status or SARS‐CoV‐2 infection status; or if only participants with SARS‐CoV‐2 infection were included Unclear: if the selection procedure was not clear or not reported. |
Did the study avoid inappropriate exclusions? | Studies may have excluded patients, or selected patients in such a way that they avoided including those who were difficult to diagnose or likely to be borderline. Although the inclusion and exclusion criteria will be different for the different index tests, inappropriate exclusions and inclusions will be similar for all index tests: for example, only elderly patients excluded, or children (as sampling may be more difficult). This needs to be addressed on a case‐by‐case basis. Yes: if a high proportion of eligible patients was included without clear selection. No: if a high proportion of eligible patients was excluded without providing a reason; if, in a retrospective study, participants without index test or reference standard results were excluded. Unclear: if the exclusion criteria were not reported. |
Did the study avoid inappropriate inclusions? | Some laboratory studies may have intentionally included groups of patients in whom the accuracy was likely to differ, such as those with particularly low or high viral loads, or who had other diseases, such that the sample over‐represented these groups. This needs to be addressed on a case‐by‐case basis. Yes: if samples included were likely to be representative of the spectrum of disease. No: if the study oversampled patients with particular characteristics likely to affect estimates of accuracy. Unclear: if the exclusion criteria were not reported. |
Could the selection of patients have introduced bias? | High: if one or more signalling questions were answered with no, as any deviation from the selection process may lead to bias. Low: if all signalling questions were answered with yes. Unclear: all other instances |
Is there concern that the included participants do not match the review question? | High: for two‐group studies that included healthy or other disease controls, whether pre‐pandemic or contemporaneous; studies that only included people with COVID‐19 (whether RT‐PCR‐confirmed only, participants meeting official guideline criteria); Low: for single‐group studies recruiting participants with signs and symptoms of COVID‐19; or for two‐group studies where control groups suspected of COVID‐19 were separately recruited. Unclear: if a description about the participants was lacking. |
DOMAIN: Index tests | |
Were the index test results interpreted without knowledge of the results of the reference standard? | Yes: if blinding was explicitly stated or index test was recorded before the results from the reference standard were available. No: if it was explicitly stated that the index test results were interpreted with knowledge of the results of the reference standard. Unclear: if blinding was unclearly reported. |
If a threshold was used, was it prespecified? | Yes: if the test was dichotomous by nature, or if the threshold was stated in the methods section, or if study authors stated that the threshold as recommended by the manufacturer was used. No: if a receiver operating characteristic curve was drawn or multiple threshold reported in the results section; and the final result was based on one of these thresholds. Unclear: if threshold selection was not clearly reported. |
Could the conduct or interpretation of the index test have introduced bias? | High: if one or more signalling questions were answered with no, as even in a laboratory situation knowledge of the reference standard may lead to bias. Low: if all signalling questions were answered with yes. Unclear: all other instances |
Is there concern that the index test, its conduct, or interpretation differ from the review question? | For all test types, if index test is 'in‐house' or not commercially available, then state 'High'. If any test procedures used in the study diverged from IFU ((use of VTM, or testing outwith stated time limit), also state High If testing carried out in centralised laboratory and not near patient then state High. Evaluations that withheld the name of the test, or that used mixed sample types or did not report the evaluation setting, state Unclear If samples used and any sample processing steps are in accordance with test IFU, or if study describes conducting the test according to the manufacturer's protocol, state Low |
DOMAIN: Reference standard | |
Is the reference standard likely to correctly classify the target condition? | We will define acceptable reference standards using a consensus process once the list of reference standards that have been used has been obtained from the eligible studies. For COVID‐19 cases Yes: RT‐PCR; confirmed or suspected case using official criteria (WHO, CDC) or a clearly set out combination of signs/symptoms/exposure No: RT‐PCR not used, or if inadequate combination of clinical characteristics used in PCR‐negatives, e.g. computed tomography alone Unclear: if definition of COVID‐19 was not reported For absence of COVID‐19 Yes: if at least 2 negative RT‐PCR results reported if suspected COVID‐19 based on signs/symptoms; single negative RT‐PCR test for asymptomatic contacts or contemporaneous controls with no clinical suspicion of COVID‐19; only pre‐pandemic sources of control samples used. No: single RT‐PCR or number of negative RT‐PCRs not reported for COVID‐19 suspects; no RT‐PCR reported (untested) for asymptomatic contacts or contemporaneous controls Unclear: if timing of control samples (pre‐pandemic or contemporaneous) was not reported |
Were the reference standard results interpreted without knowledge of the results of the index test? | Yes: if it was explicitly stated that the reference standard results were interpreted without knowledge of the results of the index test, or if the result of the index test was obtained after the reference standard. No: if it was explicitly stated that the reference standard results were interpreted with knowledge of the results of the index test or if the index test was used to make the final diagnosis. Unclear: if blinding was unclearly reported. |
Did the definition of the reference standard incorporate results from the index test(s)? | Yes: if results from the index test were a component of the reference standard definition. No: if the reference standard did not incorporate the index standard test. Unclear: if it was unclear whether the results of the index test formed part of the reference standard. |
Could the conduct or interpretation of the reference standard have introduced bias? | High: if one or more signalling questions were answered with no. Low: if all signalling questions were answered with yes. Unclear: all other instances |
Is there concern that the target condition as defined by the reference standard does not match the review question? | Applicability was judged primarily on the definition of disease‐positive. High: if RT‐PCR alone used to define cases Low: if clinical criteria, including RT‐PCR, were used to define cases, regardless of whether official criteria were used, as long as the criteria were explicitly described. Unclear: if definition of COVID‐19 cases was not provided, including if some clinically diagnosed cases were included but the clinical criteria used were not described. |
DOMAIN: Flow and timing | |
Was there an appropriate interval between index test and reference standard? | Yes: if same swab used, or swabs obtained at same time regardless of freezing (which is covered under index applicability) No: if different samples used with more than 24 hours between collection times Unclear: if can't tell |
Did all participants receive the same reference standard? | Yes: if all participants received the same reference standard (clearly no differential verification). No: if (part of) the index test‐positives or index test‐negatives received a different reference standard. Unclear: if it was not reported |
Were all participants included in the analysis? | Yes: if it is clear that all eligible participants were included in the analyses. No: if after the inclusion/exclusion process, participants were removed from the analyses for different reasons: no reference standard done, no index test done, intermediate results of both index test or reference standard, indeterminate results of both index test or reference standard, samples unusable. Unclear: if it is not possible to determine whether all participants were included (e.g. from a STARD‐style participant flow diagram) |
Did all participants receive a reference standard? | Yes: if all participants received a reference standard (clearly no partial verification). No: if only (part of) the index test positives or index test negatives received the complete reference standard. Unclear: if it was not reported |
Were results presented per participant? | Yes: if either only one sample per participant (regardless of disaggregation of results over time), or if multiple samples per participant but results are disaggregated by time period (at least week by week) No: if multiple samples per participant and results are not disaggregated by time period Unclear: if it is not possible to tell whether results presented are per participant or per sample |
Could the participantflow have introduced bias? | High: if one or more signalling questions were answered with no. Low: if all signalling questions were answered with yes. Unclear: all other instances |
CDC: Centers for Disease Control; ICU: intensive care unit; IFU: instructions for use; RT‐PCR: real‐time polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2; VTM: viral transport medium; WHO: World Health Organization |