Summary of findings 1. Summary of findings of the available filters.
| Range of measurements of accuracy from a) internal validation, b) external validation and c) independent evaluations | |||||||
| Study | Database (interface) |
Year of development/ Year of external evaluation |
Sensitivitya | Specificitya | Precisiona | NNRa | Comments |
| Filters with external evaluations and acceptable sensitivity and precisionb | |||||||
| Shojania 2001 | MEDLINE (PubMed) | 1999 to 2000/2007, 2012 and 2021 | b) 93 to 97% c) 62 to 90% |
c) 97.2% to 99.1% | c) 1.7 to 33.2% | c) 3.01 to 57.8 | Filter with independent evaluations with lower sensitivity. |
| Boynton 1998 | MEDLINE (Ovid) | 1992 to 1995/2001 and 2012 | a) 39 to 98% c) 47.8 to 99.5% |
c) 75.6 to 99.6% | a) 12 to 79% c) 0.1 to 2.1% |
a) 2.04 to 8.33 c) 46.7 to 1395 |
Filter developed in an old dataset but with recent positive independent evaluations. High sensitivity but low precision. |
| Wilczynski 2007 | Embase (Ovid) | 2000/2012 | a) 61.4 to 94.6% c) 63.4 to 96.3% |
a) 63.7 to 99.3% c) 72.3 to 99.5% |
a) 2 to 40.9% b) 0 to 0.9% |
a) 2.44 to 50 b) 117.8 to 2709.5 |
Four filters with different sensitivity and specificity profiles with consistent independent evaluations. |
| MEDLINE (Ovid) | 2000/2012 | a) 75.2 to 100% b) 71.2 to 99.9% c) 81.6 to 99.0% |
a) 63.5 to 99.4% b) 52 to 99.2% c) 62 to 99.3% |
a) 3.41 to 60.2% b) 3.14 to 57.1% c) 0 to 2% |
a) 1.66 to 29.33 b) 1.75 to 31.84 c) 49.4 to 2191.2 |
Three filters with different sensitivity and specificity profiles with consistent independent evaluations. | |
| Filters focus on specific topics | |||||||
| Boluyt 2008 | MEDLINE (PubMed) | 1994 to 2004 | b) 68 to 96% | N/A | b) 2 to 45% | 2.22 to 50 | Adaptation of existing search filters (Boynton 1998; Shojania 2001; White 2001; Wilczynski 2007) for systematic reviews on child health. |
| Lee 2012 | MEDLINE (Ovid) | 2004 to 2005 | a) 86.8% b) 89.9% |
a) 99.2% b) 98.9 % |
a) 1.1% b) 1.4% |
a) 91.6 b) 71.4 |
Filter with a focus on public health topics without independent evaluations with suboptimal sensitivity. |
| Embase (Ovid) | a) 72.7% b) 87.9% |
a) 99.1% b) 98.2% |
a) 0.6% b) 0.5% |
a) 171.6 b) 186.0 |
|||
| Avau 2021 | MEDLINE (PubMed) | 2019‐2020 | N/A | b) 97% | b) 9.7% | b) 10 | Filter with a focus on fist aid without independent evaluations. |
| Embase (Elsevier) | N/A | b) 96% | b) 5.4% | b) 19 | |||
| Others | |||||||
| White 2001 | MEDLINE (Ovid) | 1995 to 1997 | a) 67.1 to 87.1% b) 84.2% |
a) 89.2 to 99.4% b) 93% |
N/A | N/A | Filter without independent evaluations with suboptimal sensitivity. |
| Salvador‐Oliván 2021 | MEDLINE (PubMEd) | 2020 | N/A | N/A | b) 83.8% | b) 1.19 | Filter without independent evaluation assessed to retrieve "possible systematic reviews" |
Definition of outcome measures:
- Sensitivity: Proportion of systematic reviews that are correctly identified using the methodological filter.
- Specificity: Proportion of records that are not systematic reviews not identified using the methodological filter.
- Precision: Proportion of systematic reviews that are identified from all records retrieved using the methodological filter.
- Number needed to read (NNR): 1/precision
N/A: Not available. NR: not reported.
a Range of point estimates across different reports, evaluations and subsets of filters.
b >90% for sensitivity and >10% for precision
See Table 2 for a more detailed summary of the number of filters developed and evaluated in each database.