Table 3.
Summary of scores for measuring quality of the publications.
| Study first author, year | Type of publication | Points per question (Q)a | Sum of scores for Q2-Q4 | Overall score for Q1-Q4 (%), mean (SD) | Quality of paperb | ||||||
| Q1 | Q2 | Q3 | Q4 |
|
|
|
|||||
| Maslove, 2018 [14] | Journal | 4 | 3 | 3 | 4 | 10 | 90.0 (0.5) | High | |||
| Benchoufi, 2017 [15] | Journal | 4 | 3 | 2 | 3 | 8 | 80.0 (0.7) | Moderate | |||
| Ichikawa, 2017 [16] | Conference proceeding | 4 | 4 | 2 | 3 | 9 | 85.0 (0.8) | Moderate | |||
| Azaria, 2016 [17] | Journal | 4 | 3 | 2 | 3 | 8 | 80.0 (0.7) | Moderate | |||
| Cunningham, 2017 [18] | Journal | 4 | 3 | 2 | 4 | 9 | 85.0 (0.8) | Moderate | |||
| Fan, 2018 [19] | Journal | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Li, 2018 [20] | Conference proceeding | 4 | 3 | 2 | 4 | 9 | 85.0 (0.8) | Moderate | |||
| Liu, 2018 [21] | Journal | 4 | 3 | 2 | 2 | 7 | 75.0 (0.8) | Low | |||
| Mendes, 2018 [22] | Journal | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Nagasubramanian, 2018 [23] | Journal | 4 | 3 | 2 | 3 | 8 | 80.0 (0.7) | Moderate | |||
| Zhang, 2018 [24] | Journal | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Cichosz, 2018 [25] | Journal | 4 | 4 | 2 | 3 | 9 | 85.0 (0.8) | Moderate | |||
| Omar, 2019 [26] | Conference proceeding | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Angeletti, 2017 [27] | Journal | 4 | 4 | 2 | 4 | 10 | 90.0 (0.9) | High | |||
| Kleinaki, 2018 [28] | Journal | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Nugent, 2016 [29] | Journal | 4 | 4 | 2 | 4 | 10 | 90.0 (0.9) | High | |||
| Liang, 2017 [30] | Conference proceeding | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Saravanan, 2017 [31] | Journal | 4 | 3 | 1 | 3 | 7 | 75.0 (1.1) | Low | |||
| Bocek, 2017 [32] | Journal | 4 | 4 | 1 | 4 | 9 | 85.0 (1.3) | Moderate | |||
| Zhou, 2018 [33] | Conference proceeding | 4 | 4 | 1 | 3 | 8 | 80.0 (1.2) | Moderate | |||
| Kotsiuba, 2018 [34] | Conference proceeding | 4 | 3 | 1 | 2 | 6 | 70.0 (1.1) | Low | |||
| Talukder, 2018 [35] | Conference proceeding | 4 | 3 | 0 | 3 | 6 | 70.0 (1.5) | Low | |||
aTwo reviewers assessed each query from Q1 to Q4, based on a 5-point ordinal scale ranging from 0 to 4, where 0 indicates the lowest level (criterion was addressed very poorly or not at all) and 4 indicates the highest level (criterion was exceptional).
bAn overall score of ≥90% indicates a high-quality paper; an overall score of 80%-89% indicates a moderate-quality paper; an overall score ≤79% indicates a low-quality paper.