Table 2.
Quality Assessment Details of Included Studies
Question | Wang et al. (2014)49 | Yuan (2019)43 | Yeung et al. (2008)25 | Zhang et al. (2013)50 | Ye et al. (2020)44 | Chen et al. (2013)46 | Du et al. (2017)48 |
---|---|---|---|---|---|---|---|
Overall risk of bias | Low | Indeterminate: unknown interval between test and reference standard | Low | Low | Indeterminate: no description of exclusion criteria, unknown blinding protocol | Low | Low |
Was a consecutive or random sample of patients enrolled? | Yes: probability proportionate to size sampling and simple random sampling | Yes: consecutive | Yes: consecutive | Yes: consecutive | Yes: consecutive | Yes: random | University student email list—unclear if truly random |
Was a case/control design avoided? | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Did the study avoid inappropriate exclusions? | Yes | Yes: illness preventing completion, previous history of mental illness or psychotherapy, unwilling | Yes | Yes: disabling disease or reduced life expectancy, difficulty communicating | Information not found | Yes: patients with psychotic disorders | Yes |
Were the index testa results interpreted without knowledge of the results of the reference standardb? | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Are the specificity and sensitivity recorded for multiple cutoff scores? | Yes | Yes | No: only one (15) | Yes | Yes | Yes | Yes |
Was the index test administered in a standardized fashion? | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Was an appropriate version of the index test used? | Information not found | Information not found | Yes: translated by investigators | Yes: translation on Hong Kong government website | Information not found | Information not found | Information not found |
Is the reference standard likely to correctly classify the target condition? | Yes: MINI | Yes: MINI | Yes: SCID | Yes: MINI | Yes: DSM-V | Yes: SCID | Yes: MINI |
Were the reference standard results interpreted without knowledge of the results of the index test? | Information not found | Yes | No: SCID conducted for all patients with PHQ-9≥15 and 8% of patients with PHQ-9<15 | Yes | Information not found | Yes | Yes |
Was the reference standard administered in a standardized fashion? | Yes: one interviewer | Yes: standardized training | Yes: one interviewer | Yes | Yes: one interviewer | Yes: two psychiatrists with interrater reliability 0.81 | Yes: two psychiatrists |
Was there an appropriate interval between index tests and reference standard? | Yes: <2 weeks | Information not found | No: “about 4 weeks” | No: “within 2–4 weeks” | Yes: immediate | Yes: 2 weeks | Yes: within 48 h |
Did all patients receive a reference standard? | Yes | Yes | No: subsample analysis | No: subsample analysis | Yes | No: subsample analysis | No: subsample analysis |
Did patients receive the same reference standard? | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Were all patients included in the analysis without a significant percentage of dropouts? | Yes: 0% dropout rate (1045/1045) | Yes: 0% dropout rate (782/782) | Yes: 0% dropout rate (1940/1940) | Yes: 0% dropout rate (586/586) | Yes: 0% dropout rate (148/148) | Yes: 7% dropout rate (280/300) | Yes: 0% dropout rate (230/230) |
Question | Peng et al. (2020)45 | Liu et al. (2016)40 | Xiong (2015)47 | Zhang et al. (2013)50 | Xia (2019)51 | Chen (2010)53 | Liu et al. (2011)41 |
---|---|---|---|---|---|---|---|
Overall risk of bias |
Indeterminate: unknown blinding protocol, interval between test and reference standard |
Low |
Indeterminate: unknown blinding protocol, interval between test and reference standard |
Low |
Low |
Low |
Low |
Was a consecutive or random sample of patients enrolled? |
Yes: consecutive |
Yes: study province → random towns → random villages → all individuals |
Yes: on randomly assigned days, patients consecutively invited |
Yes |
Yes: consecutive |
Yes: consecutive |
Yes: consecutive |
Was a case/control design avoided? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Did the study avoid inappropriate exclusions? |
Yes: patients with pre-existing mental illness, severe physical illness |
Yes: if not in area during study period or difficulty with communication due to illness |
Yes: wrong language, limited writing skills, cognitive impairment, psychosis, acute suicidality |
Yes |
Yes: severe psychiatric disorders or antisocial, schizotypal personality disorders |
Yes |
Yes |
Were the index testa results interpreted without knowledge of the results of the reference standardb? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Are the specificity and sensitivity recorded for multiple cutoff scores? |
No: only one (9) |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Was the index test administered in a standardized fashion? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Was an appropriate version of the index test used? |
Information not found |
Yes: cited Yeung et al. 2008 |
Information not found |
Information not found |
Information not found |
Information not found |
Yes: study team translated and back translated |
Is the reference standard likely to correctly classify the target condition? |
Yes: SCID |
Yes: SCID |
Yes: MINI |
Yes: SCID |
Yes: MINI |
Yes: SCID |
Yes: SCAN |
Were the reference standard results interpreted without knowledge of the results of the index test? |
Information not found |
No: SCID conducted immediately after PHQ-9 by same interviewer |
Information not found |
Yes |
Yes |
Yes |
Yes |
Was the reference standard administered in a standardized fashion? |
Yes: two psychiatrists |
Yes: standardized interview training |
Yes |
Yes: standardized training |
Yes: one interviewer |
Yes: two psychiatrists |
Yes: trained staff with 0.88 interrater reliability |
Was there an appropriate interval between index tests and reference standard? |
Information not found |
Yes: immediate |
Information not found |
Yes: <2 weeks |
Yes: <24h |
Information not found |
Information not found |
Did all patients receive a reference standard? |
Yes |
Yes |
Yes |
Yes |
Yes |
No: subsample analysis |
Yes |
Did patients receive the same reference standard? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Were all patients included in the analysis without a significant percentage of dropouts? | Yes: 8% dropout rate (275/300) | Yes: 0.7% dropout rate (839/845) | Yes: 0% dropout rate (491/491) | Significant: 13.8% dropout rate (959/1112) | Yes: 0% dropout rate (213/213) | Significant: 20% dropout rate (77/96) for subsample chosen to undergo reference standard | Significant: 21.6% dropout rate, however, sample was large (1532/1954) |
Question | Jin et al. (2011)39,c | Xu et al. (2007)38,c | Yang et al. (2015)37,c | Zhi et al. (2013)36,c | Chen (2015)35,c | Wang et al. (2015)33,c | |
---|---|---|---|---|---|---|---|
Overall risk of bias |
Indeterminate: unknown blinding protocol, interval between test and reference standard |
Low |
Low |
Low |
Low |
Low |
|
Was a consecutive or random sample of patients enrolled? |
Yes: random |
Yes: random |
Yes: consecutive |
Yes: consecutive |
Yes |
Yes: consecutive |
|
Was a case/control design avoided? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
|
Did the study avoid inappropriate exclusions? |
Yes: dementia or physically incapable of completing questionnaire |
Yes |
Yes: incapable of understanding and responding to questionnaire, severe mental health symptoms or physical disability, substance use disorders, currently receiving medication or therapy treatment for mental illness |
Yes: physically incapable of completing questionnaire, severe physical or mental illness |
Yes: those incapable of completing study or consent due to language or hearing |
Yes |
|
Were the index test results interpreted without knowledge of the results of the reference standard? |
Yes |
Yes |
Yes |
Yes |
Information not found |
Yes |
|
Are the specificity and sensitivity recorded for multiple cutoff scores? |
No: only one (10) |
No: only one (15) |
No: only one (10) |
Yes |
Yes |
Yes |
|
Was the index test administered in a standardized fashion? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
|
Was an appropriate version of the index test used? |
Information not found |
Information not found |
Information not found |
Information not found |
Information not found |
Yes: cited Liu 2011's translation |
|
Is the reference standard likely to correctly classify the target condition? |
Yes: SCID |
Yes: SCID |
Yes: SCID |
Yes: MINI |
Yes: MINI |
Yes: CIDI |
|
Were the reference standard results interpreted without knowledge of the results of the index test? |
Information not found |
Information not found |
Yes |
Information not found |
Information not found |
Yes |
|
Was the reference standard administered in a standardized fashion? |
Yes: standardized training |
Yes: two psychiatrists |
Yes: standardized training |
Yes: standardized training |
Yes |
Yes: one interviewer |
|
Was there an appropriate interval between index tests and reference standard? |
Information not found |
Yes: 1 week |
Information not found |
Yes: immediate |
Yes: immediate |
Yes: immediate |
|
Did all patients receive a reference standard? |
No: subsample analysis |
No: subsample analysis |
No: subsample analysis |
Yes |
Yes |
Yes |
|
Did patients receive the same reference standards? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
|
Were all patients included in the analysis without a significant percentage of dropouts? | Yes: 0% dropout rate (117/117) | Yes: 0% dropout rate (195/195) | Yes: 0% dropout rate (97/97) | 0.6% dropout rate (1997/2009) | Yes: 0% dropout rate (319/319) | Yes: 0% dropout rate (201/201) |
Index test=PHQ.
Reference standard=old standard clinical interview used by the particular study.
Study published in Chinese.