Table 2.
Mixed Methods Appraisal Tool (Hong et al., 2018)
| Quantitative Randomized Controlled Trails | Are there clear research questions? | Do the collected data allow to address the research questions? | Is randomization appropriately performed? | Are the groups comparable at baseline? | Are there complete outcome data? | Are outcome assessors blinded to the intervention provided? | Did the participants adhere to the assigned intervention? |
|---|---|---|---|---|---|---|---|
| Cheng (2015) | Yes | Yes | Can’t tell, not enough info | No | Can’t tell | Yes | Yes, 4 conditions |
| Quantitative Non-randomized | Are there clear research questions? | Do the collected data allow to address the research questions? | Are the participants representative of the target population? | Are measurements appropriate regarding both the outcome and intervention (or exposure)? | Are there complete outcome data? | Are the confounders accounted for the design and analysis? | During the study period, is the intervention administered (or exposure occurred) as intended? |
| No studies in this design | -- | -- | -- | -- | -- | -- | -- |
| Quantitative Descriptive | Are there clear research questions? | Do the collected data allow to address the research questions? | Is the sampling strategy relevant to address the research questions? | Is the sample representative of the target population? | Are the measurements appropriate? | Is the risk of nonresponse bias low? | Is the statistical analysis appropriate to answer the research questions? |
| Arora et al. (2016) | Yes | Yes | Yes | Yes, for South Asian college students in Southern US | Yes | Can’t tell | Yes |
| Atkinson & Gim (1989) | Yes | Yes | Yes | Yes, for Chinese, Japanese, and Korean American college students | Yes | Yes, overall usable response rate is 68.2% | Yes |
| Quantitative Descriptive | Are there clear research questions? | Do the collected data allow to address the research questions? | Is the sampling strategy relevant to address the research questions? | Is the sample representative of the target population? | Are the measurements appropriate? | Is the risk of nonresponse bias low? | Is the statistical analysis appropriate to answer the research questions? |
| Chang & Chang (2004) | Yes | Yes | Yes | Yes, AA and Asian international college students | Yes | No, return rate for questionnaire packets is 47% | Yes |
| Chang et al. (2013) | Yes | Yes, from the NLAAS | Yes, from the NLAAS | Yes, Latino and Asian Americans | Yes | Yes, nearly 76% for Latino Americans and 66% for Asian Americans |
Yes |
| Chen & Mak (2008) | Yes | Yes | Yes | Yes, European & Chinese Americans and Hong Kong & Mainland Chinese college students | Yes | Can’t tell | Yes |
| Chen et al. (2014) | Yes | Yes | Yes | Yes, API college students | Yes | No, response rate = 23.3%, 80.7% completed with less than 5.5% missing data | Yes |
| Cheng et al. (2013) | Yes | Yes | Yes | Yes, REM (African, Asian, Latino American) college students | Yes | Can’t tell | Yes |
| Cheng et al. (2017) | Yes | Yes | Yes | Yes, White, Asian, Latino, African American college students | Yes | No, 19% participated in survey | Yes |
| Cokley et al. (2012) | Yes | Yes | Yes | Yes, Asian, Latino, African, biracial college students | Yes | Can’t tell | Yes |
| De Luna & Kawabata (2020) | Yes | Yes | Yes, interested in Filipino students | Yes, Filipino university students | Yes | No, no mention of missing data | Yes |
| Quantitative Descriptive | Are there clear research questions? | Do the collected data allow to address the research questions? | Is the sampling strategy relevant to address the research questions? | Is the sample representative of the target population? | Are the measurements appropriate? | Is the risk of nonresponse bias low? | Is the statistical analysis appropriate to answer the research questions? |
| Eisenberg et al. (2009) | Yes | Yes | Yes | Yes, college students in general | Yes | No, 44% participation rate | Yes |
| Gee et al. (2020) | Yes | Yes | Yes, from a larger study of Asian and European American college students | Yes, Asian and European American college students | Yes | Can’t tell | Yes |
| Goodwill & Zhou (2020) | Yes | Yes | Yes, from a large national college student dataset | Yes, multiracial college students including Asian | Yes | Can’t tell | Yes |
| Hampton & Sharp (2014) | Yes | Yes | Yes | Yes, with Southwestern US | Yes | Can’t tell | Yes |
| Han & Pong (2015) | Yes | Yes | Yes | Yes, Asian college students in California region, US | Yes | Can’t tell | Yes, however data not fully presented |
| Kam et al. (2019) | Yes | Yes | Yes | Yes, with Southeastern region, US | Yes | Can’t tell | Yes |
| Kim & Kendall (2015) | Yes | Yes | Yes | Yes, with Pacific Northwest region, US | Yes | Can’t tell | Yes |
| Kim et al. (2016) | Yes | Yes | Yes | Yes, with Northwest region, US | Yes | Can’t tell | Yes |
| Kim & Lee (2014) | Yes | Yes | Yes | Yes, with Northwest region, US | Yes | Can’t tell | Yes |
| Kim & Omizo (2003) | Yes | Yes | Yes | Yes, with mid-Atlantic and Hawaii, US | Yes | Can’t tell | Yes |
| Kim & Park (2009) | Yes | Yes | Yes | Yes, with Midwestern, US | Yes | Can’t tell | Yes |
| Quantitative Descriptive | Are there clear research questions? | Do the collected data allow to address the research questions? | Is the sampling strategy relevant to address the research questions? | Is the sample representative of the target population? | Are the measurements appropriate? | Is the risk of nonresponse bias low? | Is the statistical analysis appropriate to answer the research questions? |
| Kim & Zane (2016) | Yes | Yes | Yes | Yes, with West coast region, US | Yes | Can’t tell | Yes |
| Leong et al. (2011) | Yes | Yes | Yes | Yes, with West coast region, US | Yes | Can’t tell | Yes |
| Lispon et al. (2018) | Yes | Yes | Yes, random sample in national universities | Yes, across the US | Yes | Yes | Yes |
| Loya et al. (2010) | Yes | Yes | Yes | Yes, with Northern California, US | Yes | Can’t tell | Yes |
| Luna & Kawabata (2020) | Yes | Yes | Yes | Yes, with Guam, US | Yes | Can’t tell | Yes |
| Masuda & Boone (2011) | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Masua et al. (2009) | Yes | Yes | Yes, students comparing AA to African American and European American | Yes | Yes | Yes | Yes |
| Miller et al. (2011) | Yes | Yes | Yes | Yes, AA students. Born in US and foreign born | Yes | Yes | Yes |
| Mokkarala et al. (2015) | Yes | Yes | Yes | Yes, compare South Asian and White students | Shame measure low alpha | Can’t tell | Yes |
| Pedersen & Paves (2014) | Yes | Yes, students comparing white and AA | Yes | Yes, students comparing white and AA | Yes | Yes | Yes |
| Quantitative Descriptive | Are there clear research questions? | Do the collected data allow to address the research questions? | Is the sampling strategy relevant to address the research questions? | Is the sample representative of the target population? | Are the measurements appropriate? | Is the risk of nonresponse bias low? | Is the statistical analysis appropriate to answer the research questions? |
| Shahid et al. (2021) | Yes | Yes | Yes | Yes, AA students | Yes | Can’t tell | Yes |
| Stanley et al. (1976) | Yes | Yes | Yes | Yes, compared AA to white students but small sample | No, not enough info given on measure | Can’t tell | Not sure |
| Tang & Masicampo (2018) | Yes | Yes | Yes | Yes, compare AA to white students | No, psychometrics not presented from previous studies | Can’t tell | Yes |
| Ting & Hwang (2009) | Yes | Yes | Yes | Yes, AA students | Yes | Can’t tell | Yes |
| Tummala-Narra et al. (2018) | Yes | Yes | Yes | Yes, AA students | Yes | Can’t tell | Yes |
| Tummala-Narra et al. (2021) | Yes | Yes | Yes | Yes, AA students | Yes | Can’t tell | Yes |
| Wong et al. (2010) | Yes | Yes | Yes | Yes, AA students | Yes | Can’t tell | Yes |
| Wong et al. (2018) | Yes | Yes | Yes | Yes | No info about missing data | Yes | Yes |
| Yang et al. (2020) | Yes | Yes | Yes | Yes, Asian compared to white students | Not fully described and no psychometrics presented | Can’t tell | Yes |
| Yeh & Wang (2000) | Yes | Yes | Yes | Yes, AA students | No fully described | Can’t tell | Yes |
| Qualitative Methods study | Are there clear research questions? | Do the collected data allow addressing the research questions? | Is the qualitative appropriate to answer the research question? | Are the qualitative data collection methods adequate to address the research question? | Are the findings adequately derived from the data? | Is the interpretation of results sufficiently substantiated by data? | Is there coherence between data source, collection, analysis and interpretation? |
| Dong et al. (2020) | Yes | Yes | Yes | Yes | Yes, themes and subthemes | Yes, quotes fit the themes | Yes |
| Lee et al. (2009) | Yes | Yes | Yes | Yes | Yes, coding emergent themes | Yes, quotes fit the themes | Yes |
| Mixed Methods Studies | Are there clear research questions? | Do the collected data allow for addressing the research questions? | Is there an adequate rationale for using a mixed method design to address the research questions? | Are the different components of the study effectively integrated to answer the research question? | Are the results adequately brought together into overall interpretation? | Are divergences and inconsistencies between quantitative and qualitative results adequately addressed? | Do the difference components of the study adhere to the quality criteria of each tradition of the methods involved? |
| Augsberger et al. (2015) | Yes | Yes | Yes, qual results gave more insight on mental health care barriers | Yes | Yes | Yes, no stigma variable in quantitative study, but stigma was mentioned in qual results | Yes, received from AWSHIP 5-year study |