Table 1.
Reliability comparisons for phone-based assessment |
Experienced Rater | Second Phone Rater | Mean Absolute Difference |
Range of Absolute Differences |
Intraclass Correlation Coefficient |
||
---|---|---|---|---|---|---|---|
Mean | SD | Mean | SD | ||||
Total DACTS | |||||||
Experienced vs. Second Rater (n=23) |
4.30 | .17 | 4.32 | .18 | .07 | .00 – 0.25 | .92 |
Experienced vs. Consultant (n=11) |
4.38 | .12 | 4.41 | .14 | .07 | .00 – 0.25 | .92 |
Experienced vs. Naïve (n=12) |
4.23 | .18 | 4.23 | .17 | .06 | .00 – 0.14 | .91 |
Organizational Boundaries Subscale1 |
4.73 | .15 | 4.71 | .17 | .08 | .00 – 0.29 | .78 |
Human Resources Subscale1 |
4.34 | .23 | 4.38) | .26 | .11 | .00 – 0.27 | .93 |
Nature of Services Subscale1 |
3.96 | .31 | 3.98 | .30 | .18 | .00 – 0.50 | .91 |
Validity Comparisons | Consensus Phone | Onsite | Mean Absolute Difference |
Range of Absolute Differences |
Intraclass Correlation Coefficient |
||
Mean | SD | Mean | SD | ||||
Total DACTS | |||||||
Consensus ratings (n=23) |
4.29 | .18 | 4.30 | .14 | .07 | .00 – 0.32 | .87 |
Consultant rater (n=11) | 4.41 | .14 | 4.36 | .11 | .06 | .00 – 0.32 | .92 |
Experienced rater (n=23) |
4.30 | .17 | 4.30 | .14 | .07 | .00 – 0.25 | .86 |
Naïve rater (n=12) | 4.23 | .17 | 4.25 | .14 | .08 | .00 – 0.29 | .79 |
Organizational Boundaries Subscale2 |
4.71 | .17 | 4.72 | .17 | .09 | .00 – 0.29 | .69 |
Human Resources Subscale2 |
4.35 | .24 | 4.34 | .26 | .11 | .00 – 0.27 | .88 |
Nature of Services Subscale2 |
3.93 | .29 | 3.96 | .23 | .14 | .00 – 0.50 | .87 |
Note: DACTS scores range from 1 to 5, 5=full implementation.
Note 1: experienced vs. second rater;
Note 2: consensus ratings