Table 2.
Adapted meta-tool for quality assessment of public health evidence (Meta QAT) from Public Health Ontario [25]
| Item | Criteria | Assessment |
|---|---|---|
| Validity | a) Are findings presented objectively? • Clear rationale and justification • Findings presented and discussed within appropriate context • Similar to existing literature and if not, reasons explained • Conflict of interest statement |
Yes No Unclear N/A |
| b) Are the authors conclusions justified? • Transparency • Results consistent with those described in discussion • Conclusions based on empirical findings |
Yes No Unclear N/A |
|
| Reliability | a) Is the research methodology clearly described? • Study population and surveillance system described • Can identify the research design • Methods carried out in a rigorous and transparent manner |
Yes No Unclear N/A |
| b) Is methodology appropriate for the scope of research?a
• Comparison with an appropriate reference standard (i.e. a data source other than notification data)? o Is the sensitivity of the reference standard described? Are cases verified? o Is the population base for notifications and supplementary data sources drawn from same catchment area? o Is there an adequate description of case ascertainment? o Are cases ascertained for the same time period in both datasets? • Are the data sources to be compared clearly identified and described/referenced? • What case definition is used? Is the same definition applied to all data sources? • Are the methods for linkage described? o Were the identifiers unique and available? o How complete was the link? (% linked) o How accurate was the link? (i.e. were individuals linked who were not supposed to be or vice versa) o Were duplicate cases identified and removed? • How precise are results? (confidence interval width) • Were any statistical measures of agreement used (did authors express level of agreement using a statistic)? • For capture-recapture methods: o If dependency between data sources is suspected, are ≥3 or more data sources used, along with log linear modelling methods, to account for this dependency? |
Yes No Unclear N/A |
|
| c) Is the research methodology free from bias? • Are there major sources of bias? • Can the study be reproduced? |
Yes No Unclear N/A |
|
| d) Are ethics procedures described? • Approval from ethics review board • Individual consent |
Yes No Unclear N/A |
|
| e) Can I be confident about findings? • Were sources of information quality assessed? • Any major methodological flaws? |
Yes No Unclear N/A |
|
| Applicability | Can results be applied within the scope of public health? • Do the results of the study apply to the issue under consideration (i.e. are surveillance systems and supplementary data sources similar to Ontario?)? |
Yes No Unclear N/A |