Table 3.
Certainty of Evidence Ratings
| Outcome | Number of studies (N) |
Range of effects | Certainty of evidence (rationale) |
|---|---|---|---|
| Adoption of targeted process of care activities | |||
| Composite process of care outcomes |
9 randomized trials (677 practices and health service organizations) |
6 of 9 trials (75%; 95% CI 35 to 97%) with at least 1 outcome favoring the intervention; 5 trials with statistically significant findings |
Moderate certainty that coaching probably has a beneficial effect on composite process of care outcomes (rated down for serious risk of bias) |
| Organizational processes of care |
5 randomized trials (471 practices) |
4 of 5 trials (80%; 95% CI 28 to 99%) with at least 1 outcome favoring the intervention; 3 trials with statistically significant findings |
Very low certainty that coaching possibly has a beneficial effect on organizational processes of care (rated down for serious risk of bias, inconsistency, indirectness, and imprecision) |
| Appropriate documentation |
5 randomized trialsb (168 practices) |
3 of 5trials (60%; 95% CI 15 to 95%) with at least 1 outcome favoring the intervention; 3 trials with statistically significant findings |
Very low certainty that coaching possibly has a beneficial effect on appropriate documentation (rated down for very serious risk of bias, serious inconsistency, and imprecision) |
| Appropriate medication prescription |
7 randomized trialsb (452 practices) |
4 of 6 trials (66%; 95% CI 22 to 96%) with at least 2 outcomes favoring the intervention; none statistically significant | Very low certainty that coaching probably does not have a beneficial effect on appropriate medication prescription (rated down for very serious risk of bias, serious inconsistency, and imprecision) |
| Appropriate counseling |
3randomized trials (412 practices) |
3 of 3 trials (100%; 95% CI 29 to 100%); all statistically significant | Very low certainty that coaching possibly has a beneficial effect on appropriate counseling (rated down for serious risk of bias, indirectness, and imprecision) |
| Appropriate provider exams and procedures |
4 randomized trials (255 practices) |
3 of 4 trials (75%; 95% CI 19 to 99%) with at least 1 outcome favoring the intervention; 2 trials with statistically significant findings | Very low certainty of uncertain effect of coaching on improvement of provider exams/procedures (rated down for serious risk of bias, inconsistency, and imprecision) |
| Ordering of lab tests and vital signs |
6 randomized trials (146 practices) |
5 of 6 trials (83%; 95% CI 36 to 100%); 4 trials with statistically significant findings | Very low certainty that coaching probably has a beneficial effect on ordering of labs/vitals (rated down for serious risk of bias, inconsistency, and very serious imprecision) |
| QI process goal attainment (e.g., the number of QI projects reaching completion) | |||
| Mean # of QI projects initiated |
1 randomized trial (49 practices) |
3.9 QI projects per practice (intervention) vs 2.6 (comparator);p < 0.001 |
Low certainty that coaching possibly has a beneficial effect on number of the projects initiated (rated down for serious inconsistency and imprecision) |
| % mean indicators at target |
1 randomized trial (23 practices) |
Not significanta | Very low certainty that coaching has no effect on the number of indicators at target (rated down for serious risk of bias, inconsistency, and imprecision) |
| Improved team member knowledge | |||
| No trials addressed this outcome | – | – | – |
| Improved team member self-efficacy | |||
| Confidence in management |
1 randomized trial (26 practices) |
Mean confidence (pre) = 3.36 (SD 0.82); (post) = 3.89 (SD0.79); p value 0.000 |
Low certainty that coaching possibly has a beneficial effect on team member self-efficacy (rated down for serious risk of bias, and inconsistency) |
aAuthors only reported not significant results for comparison of relevance
bOnly 3 trials provided valid information on direction of effect
Abbreviations: CI confidence interval; QI quality improvement