4. InterTASC ‐ Boluyt 2008.
| A. Information | |
| A.1 State the author’s objective | To determine the sensitivity and precision of existing search strategies for retrieving child health systematic reviews in MEDLINE using PubMed. |
| A.2 State the focus of the research. | Sensitivity‐maximising Precision‐maximising |
| A.3 Database(s) and search interface(s) . | MEDLINE(PubMed) |
| A.4 Describe the methodological focus of the filter (e.g. RCTs). | Systematic reviews and meta‐analyses. |
| A.5 Describe any other topic that forms an additional focus of the filter (e.g. clinical topics such as breast cancer, geographic location such as Asia or population grouping such as paediatrics). | Child health |
| A.6 Other observations. | NO |
| B. Identification of a gold standard (GS) of known relevant records | |
| B.1 Did the authors identify one or more gold standards (GSs)? | 1 GSs. To measure search sensitivity strategies, a reference standard set of RS was established by search for RS of children's health in DARE and by manually searching for various magazines for RS. |
| B.2 How did the authors identify the records in each GS? | All titles and abstracts in DARE (Cochrane Library, Issue 2, 2004) were searched for SRs of child health also indexed in MEDLINE. We hand‐searched 7 MEDLINE‐indexed pediatric journals with a variety of impact factors and for which full‐text electronic copies were available in our medical library. All issues of each journal were searched for the following 5 years: 1994, 1997, 2000, 2002 and 2004 |
| B.3 Report the dates of the records in each GS. | Yes, 1994, 1997, 2000, 2002, and 2004 |
| B.4 What are the inclusion criteria for each GS? | “Any literature review, meta‐analysis, or other article that explicitly indicates the use of a strategy for locating evidence by mentioning at least the databases that were searched and reviewing the empirical evidence on children” |
| B.5 Describe the size of each GS and theauthors’ justification, if provided (for example the size of the gold standard may have been determined by a power calculation) | Total Reference Standard: 387 (298 by DARE + 115 found by hand search) ‐ 26 overlap= 387 |
| B.6 Are there limitations to the gold standard(s)? | Subset of the MEDLINE database |
| B.7 How was each gold standard used? | To test external validity |
| B.8 Other observations. | |
| C. How did the researchers identify the search terms in their filter(s) (select all that apply)? | |
| C.1 Adapted a published search strategy. | To identify articles that report on the development and validation of systems review the search filters in MEDLINE, searched MEDLINE from January 1995 to January 2006 with the following MeSH terms: MEDLINE, information storage and Retrieval/Methods, and Review, Literature. In addition, reference lists of relevant articles were reviewed and content experts were contacted to find further studies. To improve accuracy, © InterTASC Information Specialist Subgroup (ISSG) March 2008 We combined the systematic review filters with a sensitive child filter developed by the Cochrane Field of Child Health to retrieve only studies in children. |
| C.2 Asked experts for suggestions of relevant terms. | Contact with experts |
| C.3 Used a database thesaurus. | MeSH terms |
| C.4 Statistical analysis of terms in a gold standard set of records (see B above). | Not reported |
| C.5 Extracted terms from the gold standard set of records (see B above). | Not reported |
| C.6 Extracted terms from some relevant records (but not a gold standard). | Not reported |
| C.7 Tick all types of search terms tested. | Subject headings Text words (e.g. in title, abstract) Publication types |
| C.8 Include the citation of any adapted strategies. | Yes: Shojania, Boyton, White, Montori, PubMed plus child |
| C.9 How were the (final) combination(s) of search terms selected? | Not applicable. |
| C.10 Were the search terms combined (using Boolean logic) in a way that is likely to retrieve the studies of interest? | Not reported. |
| C.11 Other observations. | |
| D. Internal validity testing (This type of testing is possible when the search filter terms were developed from a known gold standard set of records). | |
| D.1 How many filters were tested for internal validity? | Not applicable. |
| For each filter report the following information | |
| D.2 Was the performance of the search filter tested on the gold standard from which it was derived? | Not applicable. |
| D.3 Report sensitivity data (a single value, a range, ‘Unclear’* or ‘not reported’, as appropriate). *Please describe. | Not applicable. |
| D.4 Report precision data (a single value, a range, ‘Unclear’* or ‘not reported’ as appropriate). *Please describe. | Not applicable. |
| D.5 Report specificity data (a single value, a range, ‘Unclear’* or ‘not reported’ as appropriate). *Please describe. | Not applicable. |
| D.6 Other performance measures reported. | Not applicable. |
| D.7 Other observations. | |
| E. External validity testing (This section relates to testing the search filter on records that are different from the records used to identify the search terms) | |
| E.1 How many filters were tested for external validity on records different from those used to identify the search terms? | 1. Shojania Plus Child 2. Boynton plus child 3. White 1 plus child 4. White 2 plus child 5. Montori 1 plus child 6. Montori 2 plus child 7. Montori 3 plus child 8. Montori 4 plus child 9. PubMed plus child |
| E.2 Describe the validation set(s) of records, including the interface. | Total Reference Standard: 387 True child health systematic reviews found in MEDLINE (limits 1990‐2006). |
| E.3 On which validation set(s) was the filter tested? | Reference standard all Pubmed records |
| E.4 Report sensitivity data for each validation set (a single value, a range or ‘Unclear’ or ‘not reported’, as appropriate). | Shojania+child 74 (95%CI 69‐78) Boynton+child 95 (95%CI 92‐97) White 1+child 93 (95%CI 91‐96) White 2+child 94 (95%CI 91‐96) Montori 1+child 96 (95%CI 93‐97) Montori 2+child 68 (95%CI 64‐73) Montori 3+child 94 (95%CI 91‐96) Montori 4+child 72 (95%CI 67‐76) Pubmed +child 76 (95%CI 72‐80) |
| E.5 Report precision data for each validation set (report a single value, a range or ‘Unclear’ or ‘not reported’, as appropriate). | Shojania+child 45 (95%CI 36‐55) Boynton+child 3 (95%CI 1‐9) White 2+child 2 (95%CI 1‐7) Montori 2+child 45 (95%CI 36‐55) Montori 3+child 3 (95%CI 1‐9) Pubmed +child 32 (95%CI 24‐42) |
| E.6 Report specificity data for each validation set (a single value, a range or ‘Unclear’ or ‘not reported’, as appropriate). | Not reported |
| E.6 Other performance measures reported. | Not reported |
| E.7 Other observations. | |
| F. Limitations and comparisons. | |
| F.1 Did the authors discuss any limitations to their research? | Broad definition of systematic reviews |
| F.2 Are there other potential limitations to this research that you have noticed? | Not reported |
| F.3 Report any comparisons of the performance of the filter against other relevant published filters (sensitivity, precision, specificity or other measures). | Not reported |
| F.4 Include the citations of any compared filters. | Not reported |
| F.5 Other observations and / or comments. | Not reported |
| G. Other comments. This section can be used to provide any other comments. Selected prompts for issues to bear in mind are given below. | |
| G.1 Have you noticed any errors in the document that might impact on the usability of the filter? | Not reported |
| G.2 Are there any published errata or comments (for example in the MEDLINE record)? | Not reported |
| G.3 Is there public access to pre‐publication history and / or correspondence? | Not reported |
| G.4 Are further data available on a linked site or from the authors? | Not reported |
| G.5 Include references to related papers and/or other relevant material. | Not reported |
| G.6 Other comments. | |