Table 2.
Methodological status of controlled studies
| Study | Design | Methodological status | Commentary on potential bias |
|---|---|---|---|
| Horbar et al. [84] (2004) | Randomized controlled | Randomization (computer generated), allocation concealment (investigators, prior to intervention), baseline (13 of 14 measures similar, no statistical testing), blinding (statistician), ITT (done), follow-up (100%) | Voluntary participation in collaborative: 114/178 hospitals eligible participated. |
| Curley et al. [102] (1998) |
Randomized controlled | Randomization (blocked), allocation concealment (NS), baseline (18 of 19 similar), blinding (NS), ITT (NS), follow-up (NS) | Used a convenience sample for one measure; controlled for potential covariates in analyses; questionable construct validity for provider satisfaction. |
| Carlhed et al. [26] (2006) | Controlled before | Allocation (matched then randomized), allocation concealment (controls), baseline (7 of 7 similar), blinding (controls), ITT (NS), follow-up (NS) | Intervention group hospitals self-selected, whereas control hospitals were hospitals that did not self-select; no group differences at baseline; registry had continuous monitoring; no reason to believe proposition of patients with contraindications systematically differed. |
| Doran et al. [56] (2002) | Controlled before | Allocation (participant preference, attempts to randomize), allocation concealment (NS), baseline (NS), blinding (external reviewers), ITT (NS), follow-up (time 1: 85%, time 2: 74%; higher control group attrition) | Selection: sample may be biased towards those who responded most quickly; measurement: unlikely, external reviewers blinded to group allocation and not part of study, reported methods to avoid bias; attrition/exclusion: differences between intervention group and those who withdrew, greater drop-out in the control group; gave description of sample, but did not compare group characteristics; performance: unlikely, analyses at team level. |
| Hermida and Robalino [19] (2002) | Controlled before | Allocation (matched then randomized), allocation concealment (NS), baseline (higher outcomes in intervention group), blinding (NS), ITT (NS), follow-up (NS) | |
| Howard et al. [21] (2007) | Controlled before | Allocation (matched, wait-list control), allocation concealment (NS), baseline (2 of 6 similar - controls, 5 of 6 similar - delayed comparison), blinding (NS), ITT (NS), follow-up (NS) | Provided information on non-responders; selection: self-selection, 43/58 participated, group differences at baseline; provide evidence against regression to the mean and selection bias in the wait-list controls; no information on quality of the data source. |
| Bookbinder et al. [64] (2005) | Controlled before | Allocation (location - unit type), allocation concealment (NS), baseline (3 of 21 similar), blinding (NS), ITT (NS), follow-up (NS) | Measurement: no baseline data; developed tools with interrater reliability; attrition bias: short survival of patients on the oncology unit; one tool could not completed: use was limited to 50 patients on intervention unit; selection: loss to follow up on comparison unit; performance: not possible to control for extraneous variables; referral to consultation team, exposure of staff to other educational offerings, cultural and leadership styles. |
| Brickman et al. [27] (1998) | Controlled before | Allocation (location - hospital, unclear if 'randomization' occurred), allocation concealment (NS), baseline (NS), blinding (NS), ITT (NS), follow-up (NS) | Performance: changing processes. |
| Horbar et al. [23] (2001) | Controlled before | Allocation (project participation), allocation concealment (NS), baseline (9 of 9 similar), blinding (NS), ITT (NS), follow-up (attrition in control) | Selection: self-selection of institutions. |
| Wang et al. [113] (2003) | Controlled before | Allocation (location - unit type), allocation concealment (NS), baseline (10 of 12 similar), blinding (NS), analyses (covariates), ITT (NS), follow-up (NS) | Selection: allocated by unit type, differences between groups on baseline characteristics and outcome measures, controlled for characteristics in analyses; clinical significance of differences in question; no attrition bias; performance: likely with different unit types being compared; source of inventory data quality is not known. |
| Isouard [112] (1999) | Controlled before | Allocation (location - hospital), allocation concealment (NS), baseline (3 of 3 similar), blinding (NS), analyses (no covariates), ITT (NS), follow-up (NS) | Selection: well defined criteria for selection for AMI. |
| Cable [37] (2001) | Interrupted time series | Data points (pre - 42-47 months/data points, post 22 to 27 months/data points), blinding (NS), analyses (ARIMA, switching replication), ITT (NS), follow-up (100%) | Measurement: change in catheterization tray, which affected catheterization events. |
| Berriel-Cass et al. [20] (2006) | Interrupted time series | Baseline (retrospective, NS case mix; pre - 7/8 months/data points, post - 23/24 months/data points), blinding (NS), analyses (pre-post comparisons), ITT (NS), follow-up (NS) | |
| Carey and Teeters [75] (1995) | Interrupted time series | Baseline (pre - 6 months/data points, post - 15 months/data points), blinding (NS), analyses (np charts, no inferential statistics), ITT (NS), follow-up (NS) | Selection/attrition: NA; performance/measurement: nurses may have increased reporting after training program, rather than the intervention being efficacious; unclear as to whether there was a change in intervention midway or after training program. |
| Harris et al. [38] (2000) | Interrupted time series | Baseline (pre - 3 years/6 data points, post - 3 years/6 data points), blinding (NS), analyses (no inferential statistics), ITT (NS), follow-up (NS) | Performance: physicians were already beginning to establish criteria before implementation; selection: no information about the sample. |
| Bartlett et al. [85] (2002) | Interrupted time series | Baseline (1. pre - 20 weeks/data points, post - 20 weeks/data points; 2. pre - 10 weeks/6 data points, post - 25 weeks/14 data points), blinding (NS), analyses (no inferential statistics), ITT (NS), follow-up (100%) | Selection/attrition: unlikely; measurement/performance: team-self and director-reported 'significant improvements', attempts to blind director to team identity. |
| Fox et al. [24] (2006) | Interrupted time series | Baseline (pre - 15 months/5 data points, post - 27 months/9 data points), blinding (NS), analyses (no inferential statistics), ITT (NS), follow-up (100%) | Time series controls for selection, but does not for history, instrumentation, and testing; no testing and instruments using review of charts; difficult to determine if there were any historical events that may have influenced results. |
| Allison and Toy [82] (1996) | Interrupted time series | Baseline (pre - 6 years/data points, post - 5 years/data points), blinding (NS), analyses (no inferential statistics), ITT (NS), follow-up (NS) | Measurement/instrumentation: unclear as to how some of the data was collected. |
| Halm et al. [40] (2004) | Cohort | Cohort (matched, separate pre- post cohorts, 30 of 37 similar), blinding (NS), ITT (NS), follow-up (NS) | Selection: acknowledges pre-post comparison of separate groupings of patients who met criteria of CAP; samples matched for age, race, sex, severity of diseases, co-morbidities, etc. |
| Berenholtz et al. [39] (2004) | Cohort | Cohort (different ICU types, baseline NS), blinding (NS), ITT (NS), follow-up (NS) | Selection: no description of population; may not have accounted for other confounding factors such as antibiotic use and location of catheter insertion. |
| Brown et al. [42] (2006) | Case-control | Cohort (prospective, case mix 3 of 4 similar, before-after comparisons), blinding (NS), analyses (regression) | Participants matched on post-data; performance: defined eras and care; selection bias: no loss to follow up, matched on most confounding variables; no masking regarding exposure and outcome. |
| Houston et al. [43] (2003) | Case-control | Cohort (matched - chart review, NS case mix), blinding (NS), analyses (no inferential statistics) | |
| Bromenshenkel et al. [41] (2000) | Case-control | Cohort (chart review, NS case mix; pre-post comparisons), blinding (NS), analyses (no inferential statistics) | No information on comparability of cases and controls for confounding variables, or if data collection was masked with regard to disease status of participant. |
Abbreviations: NS = not specified, ITT = intention to treat, ARIMA = Autoregressive integrated moving average, ICU = intensive care unit.