Table 3.
Quality assessment of eligible studies
| Author | Item 1 | Item 2 | Item 3 | Item 4 | Item 5 | Item 6 | Item 7 | Item 8 | Item 9 | Item 10 | Item 11 | Item 12 | Item 13 | Item 14 | Item 15 | Item 16 | Total score | % | Rating |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Aiken et al. [32] | 0 | 3 | 3 | 0 | 3 | 3 | 2 | 2 | 1 | 3 | N/A | 3 | 3 | N/A | 0 | 1 | 27 | 64 | High |
| Aryal et al. [39] | 0 | 2 | 3 | 0 | 1 | 3 | 3 | 1 | 0 | 2 | N/A | 2 | 2 | N/A | 0 | 2 | 21 | 50 | Low |
| Coombe et al. [46] | 0 | 3 | 2 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | N/A | 2 | 2 | N/A | 0 | 3 | 17 | 40 | Low |
| Endler et al. [38] | 0 | 2 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 2 | N/A | 1 | 1 | N/A | 1 | 1 | 13 | 31 | Low |
| Fuchs et al. [53] | 1 | 2 | 2 | 0 | 3 | 2 | 3 | 1 | 0 | 1 | N/A | 2 | 1 | N/A | 1 | 2 | 21 | 50 | Low |
| Leight et al. [42] | 0 | 3 | 3 | 1 | 3 | 3 | 3 | 1 | 0 | 3 | N/A | 3 | 3 | N/A | 0 | 3 | 29 | 69 | High |
| Li et al. [45] | 1 | 3 | 1 | 0 | 3 | 1 | 3 | 1 | 0 | 1 | N/A | 2 | 2 | N/A | 0 | 3 | 21 | 50 | Low |
| Luetke et al. [54] | 1 | 3 | 3 | 0 | 3 | 3 | 3 | 3 | 0 | 3 | N/A | 3 | 3 | N/A | 0 | 3 | 31 | 74 | High |
| Nagendra et al. [49] | 0 | 3 | 3 | 0 | 2 | 3 | 2 | 3 | 0 | 3 | N/A | 2 | 1 | N/A | 1 | 1 | 24 | 57 | Low |
| Phelan et al. [48] | 0 | 2 | 1 | 0 | 1 | 2 | 1 | 1 | 0 | 1 | N/A | 2 | 2 | N/A | 0 | 3 | 16 | 38 | Low |
| Rimmer et al. [51] | 0 | 3 | 3 | 0 | 1 | 3 | 3 | 3 | 0 | 3 | N/A | 3 | 2 | N/A | 2 | 3 | 29 | 69 | High |
| Roberts et al. [35] | 0 | 3 | 3 | 0 | 2 | 2 | 2 | 2 | 0 | 2 | N/A | 3 | 2 | N/A | 0 | 3 | 24 | 57 | Low |
| Roland et al. [41] | 0 | 3 | 3 | 3 | 3 | 3 | 3 | 1 | 0 | 3 | N/A | 3 | 3 | N/A | 0 | 0 | 28 | 67 | High |
| Stifani et al. [40] | 0 | 3 | 2 | 0 | 1 | 2 | 2 | 2 | 0 | 1 | N/A | 2 | 2 | N/A | 0 | 3 | 20 | 48 | Low |
| Tao et al. [52] | 0 | 3 | 3 | 0 | 3 | 2 | 3 | 1 | 2 | 3 | N/A | 2 | 2 | N/A | 0 | 3 | 27 | 64 | High |
| Tschann et al. [33] | 0 | 3 | 1 | 0 | 1 | 2 | 2 | 1 | 0 | 2 | N/A | 2 | 2 | N/A | 0 | 2 | 18 | 43 | Low |
| White et al. [36] | 1 | 3 | 2 | 2 | 3 | 2 | 3 | 1 | 2 | 3 | N/A | 3 | 3 | N/A | 0 | 2 | 30 | 71 | High |
| Yuksel et al. [44] | 1 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | N/A | 3 | 3 | N/A | 0 | 3 | 37 | 88 | High |
| Aiken et al. [32] | 0 | 3 | 3 | 0 | 3 | 3 | 2 | 2 | 1 | 3 | N/A | 3 | 3 | N/A | 0 | 1 | 27 | 64 | High |
| Aryal et al. [39] | 0 | 2 | 3 | 0 | 1 | 3 | 3 | 1 | 0 | 2 | N/A | 2 | 2 | N/A | 0 | 2 | 21 | 50 | Low |
| Caruso et al. [43] | 1 | 3 | 3 | 0 | 3 | 1 | 1 | 3 | 0 | 2 | N/A | 1 | 0 | N/A | 0 | 1 | 19 | 45 | Low |
| Coombe et al. [46] | 0 | 3 | 2 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | N/A | 2 | 2 | N/A | 0 | 3 | 17 | 40 | Low |
| Dell'Utri et al. [50] | 1 | 2 | 3 | 0 | 3 | 3 | 3 | 3 | 0 | 3 | N/A | 2 | 1 | N/A | 1 | 1 | 26 | 62 | High |
| Endler et al. [38] | 0 | 2 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 2 | N/A | 1 | 1 | N/A | 1 | 1 | 13 | 31 | Low |
Quality Assessment Tool for Studies with Diverse Designs [29]
Item 1: Explicit theoretical framework
Item 2: Statement of aims/objectives in main report
Item 3: Clear description of research setting
Item 4: Evidence of sample size considered in terms of analysis
Item 5: Representative sample of target group of a reasonable size
Item 6: Description of procedure for data collection
Item 7: Rationale for choice of data collection tool(s)
Item 8: Detailed recruitment data
Item 9: Statistical assessment of reliability and validity of measurement tool(s) (Quantitative studies only)*
Item 10: Fit between research question and method of data collection (Quantitative studies only)*
Item 11: Fit between research question and format and content of data collection tool e.g., interview schedule (Qualitative studies only)*
Item 12: Fit between research question and method of analysis
Item 13: Good justification for analytic method selected
Item 14: Assessment of reliability of analytic process (Qualitative studies only)*
Item 15: Evidence of user involvement in design
Item 16: Strengths and limitations critically discussed
Scores: 0 = not at all; 1 = very slightly; 2 = moderately; 3 = complete
Total scores > 60% = High quality; scores ≤ 60% = Low quality [31]