Table 2.
Reliability assessment of Web Resource Rating criteria measured by intraclass correlation coefficient, July 2015. n=107 resources/214 ratings.
| Criteria | Intraclass Correlation Coefficient (95% CI) | |||
|
|
|
Single measures | Average measures | |
| Evidence Base |
|
|
|
|
|
|
1. Is the Web resource informed by published single studies? | 0.933 (0.904-0.954) | 0.965 (0.949-0.976) | |
|
|
2. Is the Web resource informed by published randomized controlled trials? | 1 | 1 | |
|
|
3. Is the Web resource informed by published systematic reviews/meta-analyses? | 1 | 1 | |
|
|
4. Is the Web resource informed by best practice guidelines? | 1 | 1 | |
|
|
5. Is the quality of the evidence reported? | 0.945 (0.921-0.962) | 0.972 (0.959-0.981) | |
|
|
6. Is the strength of recommendations provided? | 0.660 (0.538-0.755) | 0.795 (0.700-0.860) | |
| Transparency |
|
|
|
|
|
|
7. Are peer-reviewed sources provided for each claim/recommendation? | 0.740 (0.641-0.815) | 0.851 (0.781-0.898) | |
|
|
8. Is the author’s or editor’s name and affiliations disclosed? | 0.942 (0.917-0.960) | 0.970 (0.957-0.980) | |
|
|
9. Is the advertising clearly labelled (or is there no advertising)? | 1 | 1 | |
|
|
10. Has the Web resource been created or updated within the last 3 years? | 0.926 (0.893-0.949) | 0.961 (0.943-0.974) | |
|
|
11. Is there a feedback mechanism? | 1 | 1 | |
| Usability |
|
|
|
|
|
|
12. Logical flow: is the information easy to follow? | 1 | 1 | |
|
|
13. Accessibility: does the Web resource offer options to access the information? Can text be resized or is there a screen reader? For nontext content, is a transcription or subtitle option available? | 0.944 (0.920-0.962) | 0.971 (0.958-0.980) | |
| Total Score | 0.988 (0.982-0.992) | 0.994 (0.991-0.996) | ||