Table 1.

Summary of quality assessment tool characteristics in studies reviewed.

Study	Quality tool used	Discipline	Number of quality criteria	Scoring Strategy	Type of scoring	Guide or explanation of criteria provided?	Quality classification system
Cuijpers et al., 2010	Developed quality criteria from a review of empirically supported psychotherapies (Chambless and Hollon, 1998) and from methodological quality recommendations of the Cochrane Collaboration (Higgins and Green, 2006)	Clinical/counseling psychology	8	Checks of whether quality criteria were met	A sum of criteria met by the study	Explanation of criteria provided by authors	A study that met all quality criteria was classified as high quality, otherwise it was classified as lower quality
Faragher et al., 2005	Developed quality criteria based on guidelines on research procedures in organizational psychology and expert consensus	Organizational/ industrial/ occupational psychology.	10	Each criterion was given a 0 score (rating) for unacceptable rigor or 1 for acceptable rigor	A summated rigor score computed (range 0–10)	Not indicated	A study that met all 10 criteria was classified as of acceptable rigor, otherwise it was classed as of unacceptable rigor
Godfrey et al., 2015	Effective Public Health Practice Project Quality Assessment Tool (EPHPP; Jackson and Waters, 2005)	Clinical/counseling psychology; health psychology; applied psychology	6	Each criterion was given 1 point for a weak quality rating, 2 points for a moderate quality rating, and 3 points for a strong quality rating	Sum of scores divided by total number of applicable criteria	Tool is published with guide	Studies of weak quality had a rating of 3, while studies of moderate quality had a rating of 2, and studies of strong quality had a rating of 1.
Hagger et al., 2017	Quality criteria adapted from the National Institutes of Health Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies (National Institutes of Health, 2014), and from other quality checklists used in cross-sectional survey designs (Jack et al., 2010; Husebø et al., 2013; Oluka et al., 2014).	Health psychology; social psychology	16	A score of 1 was assigned for each criterion met and a score of zero 0 for each criterion not met or when there was insufficient information provided to evaluate the criterion	Three types of scoring: weighted checklist score out of 10; Tertile division of checklist scores; Average checklist score	Explanation of criteria provided by authors	Tertile division of scores on the quality checklist resulted in studies above the upper tertile classified as high quality and studies below the lower tertile classified as low quality. Also, studies scoring an average of ≥6 were classified as high quality and studies scoring an average score of < 6 were classified as low quality
Hoffmann et al., 2017	Criteria for correlational designs recommended in six previous studies (Gauthier, 2003, (Effective Public Health Practice Project [EPHPP], Jackson and Waters, 2005; Von Elm et al., 2007; Wong et al., 2008; Pace et al., 2012; National Heart, Lung, and Blood Institute, 2014)	Applied psychology; traffic psychology	5	A score of one (1) assigned for criteria met and a score of zero (0) assigned for criteria not met or with insufficient information provided.	Total mean score	Explanation of criteria provided by authors	Studies that received an overall score > 2 were rated as high quality, those receiving scores 1–2 were rated as medium quality, and those receiving a < 1 score were rated low quality
Pantelic et al., 2015	Adapted version of the Cambridge Quality Checklists (CQC; Murray et al., 2009)	Cultural psychology; health psychology	8	Each criterion was assigned a numerical score between 0 and 6	One hundred per cent score would indicate the maximum possible score across all correlations in a study	Tool is published with guide	Manuscript reported quality scores but did not formally classify studies according to quality
Protogerou et al., 2018	Adapted version of a generic quality appraisal tool Glynn, 2006	Health psychology; social psychology; applied psychology	23	Each quality criterion was checked as being present (yes = 1); absent (no = 2); unclear (3) or not applicable (4)	A ratio of the “yes” answers by the total applicable items, multiplied by 100	Tool is published with guide	In line with the tool's guidelines, studies receiving a total score of < 75% were classified as of questionable quality, whereas studies with a total score of ≥75% were classified of acceptable quality.
Quon and Mcgrath, 2014	Eight criteria to assess study quality	Health psychology	8	Not indicated in manuscript.	Not indicated	Not indicated in manuscript	High quality or low quality (cut-offs not indicated)
Santos et al., 2017	A short, adapted version of the Joanna Brigs Institute critical appraisal checklist for studies reporting prevalence data (Joanna Briggs Institute, 2014)	Health psychology; sports psychology.	5	Each quality criterion was scored as yes, no, unclear or not applicable No corresponds to a limitation in the respective methodological category	The tool does not allow for numerical summative scoring Quality was used in sensitivity analysis implying summative scoring but no details provided	Tool is published with guide	Not clearly indicated
Young et al., 2014	Checklist informed by the Strengthening of Reporting of Observational Studies in Epidemiology (STROBE: Von Elm et al., 2007) and Consolidated Standards for Reporting Trials (CONSORT: Moher et al., 2010) statements, augmented with items from two reviews (Rhodes et al., 2009; Plotnikoff et al., 2013); and a list of “strong model characteristics” (Noar and Zimmerman, 2005)	Health psychology; sports psychology	11	Each quality criterion was scored as present (Y), absent (N), unclear or inadequately described' (0) or not applicable (n/a)	Sum of scores of present quality criteria	Explanation of criteria provided by authors	Not clearly indicated