Table 7.
Q-SSP assessments by item, domain and total score for included studies (n = 48 studies, 1 companion report)
| First author, year | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Q11 | Q12 | Q13 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Butterfoss, 1996 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Kegler, 1998 | 1 | NC | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| Chan, 2000 | 1 | 1 | 1 | 1 | 1 | 1 | NC | 0 | 0 | 1 | 1 | 1 | 0 |
| Shortell, 2002 | 1 | 1 | 1 | 1 | 0 | 0 | NC | 0 | 0 | 1 | 1 | 1 | 0 |
| Weiss, 2002 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| El Ansari, 2004 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| El Ansari, 2004b | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Metzger, 2005 | 1 | NC | 1 | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 0 |
| Kegler, 2005 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Cramer, 2006 | 1 | NC | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| Feinberg, 2008 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 |
| Feinberg, 2008b | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
| Orr Brawer, 2008 | 1 | 1 | 1 | 1 | 1 | 1 | NC | NC | 1 | 1 | 1 | 0 | 0 |
| King, 2009 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| Tolma, 2009 | 1 | NC | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NC | 0 | 0 | 1 |
| Wagemakers, 2010 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 |
| King, 2010 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| Ziff, 2010 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Jones, 2011 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| Jones, 2011b | 1 | 1 | 0 | 1 | 1 | 1 | NC | 0 | NC | 1 | 1 | 1 | 0 |
| Perkins, 2011 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 |
| Bilodeau, 2011 and 2019b | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | NC | 1 | 1 | 1 |
| Curro, 2012 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| El Ansari, 2012 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 |
| Brown, 2012 | 1 | 1 | 0 | 1 | 1 | 1 | NC | 0 | 0 | 1 | 0 | 1 | 0 |
| Nargiso, 2013 | 1 | NC | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 |
| Perkins, 2014 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 |
| Chang, 2014 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 |
| Brown, 2015 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 |
| Bornstein, 2015 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 0 |
| Oetzel, 2015 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 |
| Oetzel, 2015b | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 0 |
| Stocks, 2015 | 1 | NC | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 |
| Brown, 2016 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 |
| Goodman, 2017 | 1 | 1 | 0 | 1 | 1 | NC | 0 | 0 | 0 | 1 | 1 | 0 | 0 |
| Okazaki, 2017 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 |
| Jones, 2018 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| West, 2018 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 |
| Oetzel, 2018 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 |
| Duran, 2019 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 |
| Soobiah, 2019 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |
| Dickson, 2020 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | NC | 1 | 0 | 1 | 0 |
| Rodriguez Espinosa, 2020 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| Lucero, 2020 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 |
| Van Schelven, 2020 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 0 |
| Hamilton, 2021 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | NC | NC | 1 | 1 | 1 | 0 |
| Boursaw, 2021 | 1 | 1 | 1 | 1 | 1 | 1 | NC | 0 | 1 | 1 | 1 | 1 | 0 |
| Loban, 2021 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 |
| First author, year | Q14 | Q15 | Q16 | Q17 | Q18 | Q19 | Q20 | D1 | D2 | D3 | D4 | %a | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Butterfoss, 1996 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 3 | 1 | 6 | 1 | 55 | |
| Kegler, 1998 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 2 | 1 | 4 | 1 | 40 | |
| Chan, 2000 | NC | NC | 0 | 1 | 0 | 0 | 1 | 4 | 2 | 4 | 1 | 55 | |
| Shortell, 2002 | NC | 0 | 0 | 1 | 0 | 0 | 1 | 4 | 0 | 4 | 1 | 45 | |
| Weiss, 2002 | NC | 0 | 1 | 1 | NC | 1 | 1 | 4 | 2 | 5 | 2 | 65 | |
| El Ansari, 2004 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 4 | 0 | 5 | 1 | 50 | |
| El Ansari, 2004b | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 2 | 2 | 6 | 1 | 55 | |
| Metzger, 2005 | 1 | NC | 0 | 1 | 0 | 0 | 1 | 2 | 2 | 6 | 1 | 55 | |
| Kegler, 2005 | 1 | 1 | 1 | 1 | NC | 0 | 1 | 3 | 2 | 6 | 1 | 60 | |
| Cramer, 2006 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 2 | 1 | 4 | 2 | 45 | |
| Feinberg, 2008 | 1 | NC | 0 | 1 | 0 | NC | 1 | 3 | 2 | 5 | 1 | 55 | |
| Feinberg, 2008b | 1 | 1 | 0 | 0 | NC | 1 | 1 | 4 | 1 | 6 | 2 | 65 | |
| Orr Brawer, 2008 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 4 | 2 | 7 | 3 | 80 | |
| King, 2009 | NC | 0 | 0 | 1 | 1 | NC | 1 | 3 | 2 | 4 | 2 | 55 | |
| Tolma, 2009 | NC | 0 | 0 | 1 | NC | 1 | 1 | 1 | 0 | 2 | 2 | 25 | |
| Wagemakers, 2010 | 0 | NC | 1 | 1 | 0 | NC | 1 | 2 | 3 | 5 | 1 | 55 | |
| King, 2010 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 3 | 2 | 7 | 2 | 70 | |
| Ziff, 2010 | 0 | NC | 0 | 1 | 0 | 0 | 1 | 3 | 2 | 3 | 1 | 45 | |
| Jones, 2011 | NC | NC | 0 | NC | 1 | 0 | 1 | 4 | 1 | 3 | 2 | 50 | |
| Jones, 2011b | 0 | 1 | 0 | 1 | 1 | 0 | 1 | 3 | 2 | 5 | 2 | 60 | |
| Perkins, 2011 | NC | 1 | 1 | 1 | 1 | 1 | 1 | 4 | 2 | 6 | 3 | 75 | |
| Bilodeau, 2011 and 2019b | NC | NC | 0 | 1 | 1 | 1 | 1 | 3 | 1 | 4 | 3 | 55 | |
| Curro, 2012 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 2 | 2 | 3 | 1 | 40 | |
| El Ansari, 2012 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 3 | 2 | 4 | 1 | 50 | |
| Brown, 2012 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 3 | 2 | 4 | 1 | 50 | |
| Nargiso, 2013 | 1 | NC | 0 | 1 | NC | NC | 0 | 3 | 0 | 4 | 0 | 35 | |
| Perkins, 2014 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 4 | 3 | 6 | 1 | 70 | |
| Chang, 2014 | 0 | NC | 1 | 0 | NC | NC | 1 | 2 | 2 | 3 | 1 | 40 | |
| Brown, 2015 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 4 | 2 | 7 | 1 | 70 | |
| Bornstein, 2015 | 1 | 0 | 1 | 1 | NC | 0 | 1 | 3 | 2 | 7 | 1 | 65 | |
| Oetzel, 2015 | 0 | NC | 0 | 1 | 1 | 0 | 1 | 2 | 3 | 5 | 2 | 60 | |
| Oetzel, 2015b | 1 | NC | 1 | 1 | 1 | NC | 1 | 4 | 2 | 6 | 2 | 70 | |
| Stocks, 2015 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 2 | 1 | 5 | 3 | 55 | |
| Brown, 2016 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 3 | 2 | 7 | 1 | 65 | |
| Goodman, 2017 | NC | 0 | 1 | 1 | 0 | NC | 1 | 3 | 1 | 4 | 1 | 45 | |
| Okazaki, 2017 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 4 | 1 | 7 | 2 | 70 | |
| Jones, 2018 | 0 | NC | 0 | 1 | 0 | 0 | 1 | 3 | 2 | 4 | 1 | 50 | |
| West, 2018 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 2 | 2 | 9 | 2 | 75 | |
| Oetzel, 2018 | 1 | 1 | 1 | 1 | 1 | NC | 1 | 4 | 2 | 8 | 2 | 80 | |
| Duran, 2019 | NC | 1 | 1 | 1 | 1 | 0 | 1 | 3 | 2 | 8 | 2 | 75 | |
| Soobiah, 2019 | 0 | 0 | 1 | 1 | NC | 0 | 1 | 2 | 2 | 5 | 1 | 50 | |
| Dickson, 2020 | 1 | 1 | 1 | 1 | NC | 0 | 1 | 3 | 2 | 6 | 1 | 60 | |
| Rodriguez Espinosa, 2020 | NC | NC | 0 | 1 | 1 | 1 | 1 | 4 | 2 | 2 | 3 | 55 | |
| Lucero, 2020 | 0 | 0 | 1 | 1 | NC | 0 | 1 | 4 | 2 | 6 | 1 | 65 | |
| Van Schelven, 2020 | 1 | NC | 1 | 1 | NC | 0 | 1 | 3 | 3 | 5 | 1 | 60 | |
| Hamilton, 2021 | 1 | NC | 1 | 1 | 1 | 0 | 1 | 3 | 3 | 6 | 2 | 70 | |
| Boursaw, 2021 | 1 | NC | 1 | 1 | NC | 0 | 1 | 4 | 2 | 7 | 1 | 70 | |
| Loban, 2021 | 1 | 1 | 1 | 1 | 1 | NC | 1 | 2 | 3 | 8 | 2 | 75 | |
| All studies |
Mean SD |
3.04 0.82 |
1.77 0.78 |
5.27 1.62 |
1.52 0.71 |
58.02 12.32 |
Q-SSP questions: Q1 Problem or phenomenon defined, described and justified? Q2 Population defined, described and justified? Q3 Specific research questions and/or hypotheses stated? Q4 Operational definitions of all study variables provided? Q5 Participant* inclusion criteria stated? Q6 Participant* recruitment strategy described? Q7 Justification/rationale for the sample size provided? Q8 Attrition* rate provided? Q9 Method of treating attrition* provided? Q10 Data analysis techniques justified—link between hypotheses, aims, research questions and analyses explained? Q11 Tool and/or all measures provided in report or supplement? Q12 Evidence for validity of measures present? Q13 Characteristics of person(s) collecting data (e.g. training, expertise, other demographic characteristics) provided? Q14 Information about context of data collection provided? Q15 Information about duration of data collection provided? Q16 Description of key demographic characteristics of study sample* provided? Q17 Discussion confined to the population* from which the sample was drawn? Q18 Informed consent or assent requested? Q19 Participants* debriefed at the end of data collection? Q20 Funding sources and conflicts of interest disclosed? [*questions adapted to reflect individual, project and/or partnership based on study unit of analysis]
Scoring
D1 Domain 1: introduction/rationale/variables score (ratio of sum Q1–Q4/total applicable domain items)
D2 Domain 2: participants*/sampling/recruitment score (ratio of sum Q5–Q7/total applicable domain items)
D3 Domain 3: data collection/analyses/measures/results/discussion score (ratio of sum Q8–17/total applicable domain items)
D4 Domain 4: ethics score (ratio of sum Q18–Q20/total applicable domain items)
aTotal overall quality score (%) (sum D1–D4/total applicable domain items × 100)
bCompanion report