DOMAIN: PARTICIPANT SELECTION | |
Was a consecutive or random sample of patients enrolled? | This will be similar for all index tests, target conditions, and populations. YES: if a study explicitly stated that all participants within a certain time frame were included; that this was done consecutively; or that a random selection was done. NO: if it was clear that a different selection procedure was employed; for example, selection based on clinician's preference, or based on institutions. UNCLEAR: if the selection procedure was not clear or not reported. |
Was a case‐control design avoided? | This will be similar for all index tests, target conditions, and populations. YES: if a study explicitly stated that all participants came from the same group of (suspected) patients. NO: if it was clear that a different selection procedure was employed for the participants depending on their COVID‐19 status or SARS‐CoV‐2 infection status; or if only participants with SARS‐CoV‐2 infection were included UNCLEAR: if the selection procedure was not clear or not reported. |
Did the study avoid inappropriate exclusions? | Studies may have excluded patients, or selected patients in such a way that they avoided including those who were difficult to diagnose or likely to be borderline. Although the inclusion and exclusion criteria will be different for the different index tests, inappropriate exclusions and inclusions will be similar for all index tests: for example, only elderly patients excluded, or children (as sampling may be more difficult). This needs to be addressed on a case‐to‐case basis. YES: if a high proportion of eligible patients was included without clear selection. NO: if a high proportion of eligible patients was excluded without providing a reason; if, in a retrospective study, participants without index test or reference standard results were excluded. UNCLEAR: if the exclusion criteria were not reported. |
Did the study avoid inappropriate inclusions? | Some laboratory studies may have intentionally included groups of patients in whom the accuracy was likely to differ, such as those with particularly low or high viral loads, or who had other diseases, such that the sample over‐represented these groups. This needs to be addressed on a case‐to‐case basis. Artificial spiked samples are a clear example. YES: if samples included were likely to be representative of the spectrum of disease. NO: if the study oversampled patients with particular characteristics likely to affect estimates of accuracy. UNCLEAR: if the exclusion criteria were not reported. |
Could the selection of patients have introduced bias? | HIGH: if one or more signalling questions were answered with NO, as any deviation from the selection process may lead to bias. LOW: if all signalling questions were answered with YES. UNCLEAR: all other instances. |
Is there concern that the included participant s do not match the review question? | HIGH: for two‐group studies that included healthy or other disease controls, whether pre‐pandemic or contemporaneous; studies that only included people with COVID‐19 (whether reverse transcription polymerase chain reaction (RT‐PCR)‐confirmed only, participants meeting official guideline criteria). LOW: for single‐group studies recruiting participants with signs and symptoms of COVID‐19; or for two‐group studies where control groups suspected of COVID‐19 were separately recruited. UNCLEAR: if a description about the participants was lacking. |
DOMAIN: INDEX TESTS | |
Were the index test results interpreted without knowledge of the results of the reference standard? | YES: if blinding was explicitly stated or index test was recorded before the results from the reference standard were available. NO: if it was explicitly stated that the index test results were interpreted with knowledge of the results of the reference standard. UNCLEAR: if blinding was unclearly reported. |
If a threshold was used, was it prespecified? | YES: if the test was dichotomous by nature, or if the threshold was stated in the methods section, or if study authors stated that the threshold as recommended by the manufacturer was used. NO: if a receiver‐operating‐characteristic curve was drawn or multiple threshold reported in the results section; and the final result was based on one of these thresholds. UNCLEAR: if threshold selection was not clearly reported. |
Could the conduct or interpretation of the index test have introduced bias? | HIGH: if one or more signalling questions were answered with NO, as even in a laboratory situation knowledge of the reference standard may lead to bias. LOW: if all signalling questions were answered with YES. UNCLEAR: all other instances. |
Is there concern that the index test, its conduct, or interpretation differ from the review question? | For evaluations of laboratory‐based tests, HIGH: if tests were built inhouse, or if commercially available tests using SARS‐Cov antigens instead of SARS‐CoV‐2‐specific antigens. LOW: most other laboratory evaluations. UNCLEAR: name of the test was withheld. For evaluations of lateral flow assays, HIGH: if tests were built inhouse; if only serum or plasma instead of fingerprick or whole blood samples were used; if test evaluated in laboratory settings rather than at the point of care. LOW: commercially available tests, using whole blood or fingerprick samples, and that were conducted in the intended setting for the test (i.e. point‐of‐care). UNCLEAR: name of the test was withheld; mixed sample types; or did not report the evaluation setting. |
DOMAIN: REFERENCE STANDARD | |
Is the reference standard likely to correctly classify the target condition? | We will define acceptable reference standards using a consensus process once the list of reference standards that have been used has been obtained from the eligible studies. For COVID‐19 cases: YES: RT‐PCR; confirmed or suspected case using official criteria (WHO, CDC) or a clearly set out combination of signs/symptoms/exposure. NO: RT‐PCR not used, or if inadequate combination of clinical characteristics used in PCR negatives, e.g. computed tomography alone. UNCLEAR: if definition of COVID‐19 was not reported. For absence of COVID‐19: YES: if at least 2 negative RT‐PCR results reported if suspected COVID‐19 based on signs/symptoms; single‐negative RT‐PCR test for asymptomatic contacts or contemporaneous controls with no clinical suspicion of COVID‐19; only pre‐pandemic sources of control samples used. NO: single RT‐PCR or number of negative RT‐PCRs not reported for COVID‐19 suspects; no RT‐PCR reported (untested) for asymptomatic contacts or contemporaneous controls. UNCLEAR: if timing of control samples (pre‐pandemic or contemporaneous) was not reported. |
Were the reference standard results interpreted without knowledge of the results of the index test? | YES: if it was explicitly stated that the reference standard results were interpreted without knowledge of the results of the index test, or if the result of the index test was obtained after the reference standard. NO: if it was explicitly stated that the reference standard results were interpreted with knowledge of the results of the index test or if the index test was used to make the final diagnosis. UNCLEAR: if blinding was unclearly reported. |
Did the definition of the reference standard incorporate results from the index test(s)? | YES: if results from the index test were a component of the reference standard definition. NO: if the reference standard did not incorporate the index standard test. UNCLEAR: if it was unclear whether the results of the index test formed part of the reference standard. |
Could the conduct or interpretation of the reference standard have introduced bias? | HIGH: if one or more signalling questions were answered with NO. LOW: if all signalling questions were answered with YES. UNCLEAR: all other instances. |
Is there concern that the target condition as defined by the reference standard does not match the review question? | Applicability was judged primarily on the definition of disease‐positive. HIGH: if RT‐PCR alone used to define cases. LOW: if clinical criteria, including RT‐PCR, were used to define cases, regardless of whether official criteria were used, as long as the criteria were explicitly described. UNCLEAR: if definition of COVID‐19 cases was not provided, including if some clinically diagnosed cases were included but the clinical criteria used were not described. |
DOMAIN: FLOW AND TIMING | |
Did all participants receive the same reference standard? | YES: if all participants received the same reference standard (clearly no differential verification). NO: if (part of) the index test‐positives or index test‐negatives received a different reference standard. UNCLEAR: if it was not reported. |
Were all participants included in the analysis? | YES: if it is clear that all eligible participants were included in the analyses. NO: if after the inclusion/exclusion process, participants were removed from the analyses for different reasons: no reference standard done, no index test done, intermediate results of both index test or reference standard, indeterminate results of both index test or reference standard, samples unusable. UNCLEAR: if it is not possible to determine whether all participants were included (e.g. from a STARD style participant flow diagram) |
Did all participants receive a reference standard? | YES: if all participants received a reference standard (clearly no partial verification). NO: if only (part of) the index test‐positives or index test‐negatives received the complete reference standard. UNCLEAR: if it was not reported. |
Were results presented per participant? | YES: if either only one sample per participant (regardless of disaggregation of results over time), or if multiple samples per participant but results are disaggregated by time period (at least week by week). NO: if multiple samples per participant and results are not disaggregated by time period. UNCLEAR: if it is not possible to tell whether results presented are per participant or per sample. |
Could the participant flow have introduced bias? | HIGH: if one or more signalling questions were answered with NO. LOW: if all signalling questions were answered with YES. UNCLEAR: all other instances. |