Table 2.
Strength and quality of research evidence rating scheme for individual studies
| LEVEL | STRENGTH OF EVIDENCE/DESIGN | YES OR NO | |
|---|---|---|---|
| I | Evidence obtained from an experimental study/randomized controlled trial. Includes factorial or crossover designs where individuals were randomly assigned to groups that each had a different order of treatment. | — | — |
| II | Evidence obtained from a pre-experimental or quasi-experimental study that lacked either randomization or control. | — | — |
| III | Evidence obtained from a non-experimental study, qualitative study, or meta-synthesis. | — | — |
| GRADE | QUALITY OF RESEARCH EVIDENCE | ||
| High | • Sample: power analysis reported | — | — |
| • Control: confounders controlled, equivalent groups, equivalent attention and diversion in treatment and control conditions | — | — | |
| • Methods: description permits replication | — | — | |
| • Conclusion validity: | |||
| • Attrition: ≤10% or use of intent-to-treat or other appropriate methods for analyzing missing data | — | — | |
| • Use of at least one validated and reliable outcome measurement tool | — | — | |
| • Investigators blinded to participant group allocation | — | — | |
| Moderate | • Sample: clear criteria for how sample was selected | — | — |
| • Control: non-equivalent groups or unequal attention and diversion in comparison condition(s) | — | — | |
| • Methods: major details are described | — | — | |
| • Conclusion validity: | |||
| • Attrition: 11–20% or analysis of attrition rates and group equivalency | — | — | |
| • Use of at least one validated and reliable outcome measurement tool | — | — | |
| • Some attempt to limit potential investigator/rater bias | — | — | |
| Low/major flaw(s) | • Sample size: no explanation, small convenience sample (n ≤ 10) | — | — |
| • Control: no attempt made to control relevant confounders | — | — | |
| • Methods: inadequate description | — | — | |
| • Conclusion validity: | |||
| • Attrition >20%, not analyzed, or not reported | — | — | |
| • Data collection did not use validated measures (of NPS) | — | — | |
| • Potential investigator/rater bias | — | — | |
Note: Adapted from Johns Hopkins Nursing Evidence Based Practice (JHNEBP: Newhouse et al., 2007) and the Validity Rating Tool (Forbes, 1998). A criterion of consistency of findings was also included when evaluating intervention evidence across related studies.