Table 3.
Mixed Methods Appraisal Tool (Version 2018) for critically appraising quantitative (n = 14), qualitative (n = 16), and mixed methods (n = 2) study reviews.
| First Author (Year) | Question | ||||||
|---|---|---|---|---|---|---|---|
| Qualitative | QS1 | QS2 | Q1.1 | Q1.2 | Q1.3 | Q1.4 | Q1.5 |
| Bezboruah 2013 [5] | Y | Y | Y | Y | Y | Y | Y |
| Black 2020 [36] | Y | Y | Y | Y | Y | Y | Y |
| Denham 1998 [11] | Y | Y | Y | Y | Y | Y | Y |
| Doherty 2006 [12] | Y | Y | Y | U | Y | Y | U |
| Douglas 2016 [13] | Y | Y | Y | Y | Y | Y | Y |
| Flick 1994 [15] | Y | Y | Y | Y | Y | Y | Y |
| Hatch 1978 [16] | Y | Y | Y | Y | Y | Y | Y |
| Hedley 2002 [37] | Y | Y | Y | Y | Y | Y | Y |
| Hilgendorf 2016 [18] | Y | Y | Y | Y | Y | Y | Y |
| Kang 2015 [19] | Y | Y | Y | Y | Y | Y | Y |
| Parker 2010 [21] | Y | Y | Y | Y | Y | Y | Y |
| Ross 2011 | Y | Y | Y | Y | U | Y | Y |
| Saxon 2021 [28] | Y | Y | Y | Y | Y | Y | Y |
| Tataw 2020 [30] | Y | Y | Y | N | Y | N | N |
| Wagoner 2010 [32] | Y | Y | Y | Y | Y | Y | Y |
| Zanoni 2011 [34] | Y | Y | Y | Y | Y | Y | Y |
| Quantitative (RCT) | QS1 | QS2 | Q2.1 | Q2.2 | Q2.3 | Q2.4 | Q2.5 |
| Livingstone 2018 | Y | Y | Y | Y | Y | N | U |
| Perry 2000 [22] | Y | Y | Y | Y | N | N | Y |
| Wagenaar 1999 [31] | Y | Y | U | Y | Y | N | Y |
| Quantitative (Non-randomised) | QS1 | QS2 | Q3.1 | Q3.2 | Q3.3 | Q3.4 | Q3.5 |
| Bosma 2005 [6] | Y | Y | Y | Y | Y | N | Y |
| Brookes 2010 [7] | Y | Y | Y | Y | Y | N | Y |
| Byrant 2010 | Y | Y | Y | Y | Y | Y | Y |
| Cheadle 2010 [10] | Y | Y | Y | Y | U | U | Y |
| Haseda 2019 [38] | Y | Y | Y | Y | Y | Y | Y |
| McKenzie 2004 [35] | Y | Y | Y | Y | Y | Y | N |
| Subica 2016 [29] | Y | Y | Y | U | Y | N | Y |
| Quantitative (Descriptive) | QS1 | QS2 | Q4.1 | Q4.2 | Q4.3 | Q4.4 | Q4.5 |
| Bauermeister 2017 [3] | Y | Y | Y | Y | Y | U | Y |
| Berman 2018 [4] | Y | Y | Y | Y | Y | N | Y |
| Poole 1997 [23] | Y | Y | Y | Y | Y | U | U |
| Salem 2005 [26] | Y | Y | Y | Y | Y | N | Y |
| Mixed Methods | QS1 | QS2 | Q5.1 | Q5.2 | Q5.3 | Q5.4 | Q5.5 |
| Agrusti 2020 [2] | Y | Y | Y | Y | Y | Y | Y |
| Weeks 2013 [33] | Y | Y | Y | Y | Y | Y | Y |
| Further appraisal is not feasible | |||||||
| Cheadle 2009 [9] | U | Y | |||||
| Fawcett 2018 [14] | Y | N | |||||
| Hays 2003 [17] | Y | N | |||||
| Hildebrandt 1994 [39] | Y | N | |||||
| Rask 2015 [24] | N | Y | |||||
| Santilli 2016 [27] | N | N | |||||
Index Abbreviations: Y = Yes, N = No, U = Unclear, MMAT = Mixed Methods Appraisal Tool. A. Mixed Methods Appraisal Tool (MMAT) screening questions for all studies—QS1: Are the research questions clear?, QS2: Do the collected data allow to address the research questions? B. Mixed Methods Appraisal Tool (MMAT) for qualitative studies—Q1.1: Is the qualitative approach appropriate to answer the research questions?, Q1.2: Are the qualitative data collection methods adequate to address the research question? Q1.3: Are the findings adequately derived from the data?, Q1.4: Is the interpretation of results sufficiently substantiated by data?, Q1.5: Is there coherence between qualitative data sources, collection, analysis and interpretation? C. Mixed Methods Appraisal Tool (MMAT) for quantitative Randomised Controlled Trail—2.1. Is randomization appropriately performed? 2.2. Are the groups comparable at baseline? 2.3. Are there complete outcome data? 2.4. Are outcome assessors blinded to the intervention provided? 2.5 Did the participants adhere to the assigned intervention? D. Mixed Methods Appraisal Tool (MMAT) for quantitative non-Randomised—3.1. Are the participants representative of the target population? 3.2. Are measurements appropriate regarding both the outcome and intervention (or exposure)? 3.3. Are there complete outcome data? 3.4. Are the confounders accounted for in the design and analysis? 3.5. During the study period, is the intervention administered (or exposure occurred) as intended? E. Mixed Methods Appraisal Tool (MMAT) for quantitative descriptive studies—Q4.1: Is the sampling strategy relevant to address the research question?, Q4.2: Is the sample representative of the target population?, Q4.3: Are the measurements appropriate?, Q4.4: Is the risk of nonresponse bias low?, Q4.5: Is the statistical analysis appropriate to answer the research question? F. Mixed Methods Appraisal Tool (MMAT) for mixed-method studies—Q5.1: Is there an adequate rationale for using a mixed methods design to address the research question?, Q5.2: Are the different components of the study effectively integrated to answer the research question?, Q5.3: Are the outputs of the integration of qualitative and quantitative components adequately interpreted?, Q5.4: Are divergences and inconsistencies between quantitative and qualitative results adequately addressed?, Q5.5: Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?