Table 1.
Observer Ratings of Implementation Adherence and Quality of Facilitation
| Program | Implementation variable | Na | M | SD | Range | Interrater reliability (r)b |
|---|---|---|---|---|---|---|
| SFP 10–14 | Adherence | 411 | 90.9% | 9.2% | 48%–100% | .86 (n = 131) |
| 350 | 90.8% | 10.9% | 24%–100% | .76 (n = 106) | ||
| Group participation | 328 | 3.64 | .53 | 1.50–4.00 | .66 (n = 101) | |
| 283 | 3.58 | .56 | 2.00–4.00 | .63 (n = 90) | ||
| Facilitation quality | 894 | 3.76 | .34 | 1.89–4.00 | .71 (n = 271) | |
| 754 | 3.71 | .40 | 1.25–4.00 | .66 (n = 229) | ||
| All Stars | Adherence | 67 | 94.4% | 9.3% | 47%–100% | .94 (n = 22) |
| 64 | 87.4% | 22.5% | 0%–100% | .67 (n = 22) | ||
| Student engagement | 67 | 3.49 | .59 | 1.75–4.00 | .46 (n = 19) | |
| 64 | 3.60 | .53 | 1.50–4.00 | .68 (n = 21) | ||
| Life Skills Training | Adherence | 41 | 88.8% | 14.6% | 33%–100% | .84 (n = 11) |
| 48 | 88.5% | 16.4% | 27%–100% | .94 (n = 6) | ||
| Student engagement | 41 | 3.37 | .51 | 1.75–4.00 | .71 (n = 9) | |
| 48 | 3.49 | .63 | 1.75–4.00 | .90 (n = 6) | ||
| Project Alert | Adherence | 67 | 88.2% | 14.6% | 25%–100% | .24 (n = 25) |
| 50 | 89.3% | 10.2% | 64%–100% | .70 (n = 14) | ||
| Student engagement | 67 | 3.39 | .50 | 1.75–4.00 | .43 (n = 25) | |
| 50 | 3.31 | .53 | 2.00–4.00 | .23 (n = 13) | ||
| All school-based programsc | Adherence | 175 | 90.7% | 13.1% | 25%–100% | .81 (n = 58) |
| 162 | 88.3% | 17.6% | 0%–100% | .69 (n = 42) | ||
| Student engagement | 175 | 3.42 | .54 | 1.75–4.00 | .55 (n = 53) | |
| 162 | 3.48 | .57 | 1.50–4.00 | .55 (n = 40) |
Note. Values for Cohort 1 are placed above values for Cohort 2. SFP 10–14 = Strengthening Families Program: For Parents and Youth 10–14.
Values of N in this column correspond to the number of ratings on which the mean, standard deviation, and range are based for each variable. The ratings used for these statistics were those made in sessions observed by a single observer. For SFP 10–14, there was one rating of adherence per session observed. The SFP 10–14 observation forms did not include group participation items for all sessions, resulting in an average of less than one rating per observed session. Observation of SFP 10–14 sessions resulted in facilitation quality items being rated once for parent sessions (because they had a single facilitator), twice for youth sessions (because they had two facilitators), and three times for family sessions (because they had one parent and two youth facilitators). The school-based programs yielded one rating of adherence and one rating of student engagement for each session observed.
Interrater reliability values reflect bivariate correlations between ratings provided by two observers making ratings of the same session. The values of n in this column correspond to the number of rating pairs on which each correlation is based. See previous footnote for information regarding the number of ratings produced per session for each of the variables from the SFP 10–14 and school-based programs.
The school-based programs included All Stars, Life Skills Training, and Project Alert.