Skip to main content
. 2022 Aug 18;6(1):574–594. doi: 10.1089/heq.2022.0030

Table 2.

Quality Assessment Details of Included Studies

Question Wang et al. (2014)49 Yuan (2019)43 Yeung et al. (2008)25 Zhang et al. (2013)50 Ye et al. (2020)44 Chen et al. (2013)46 Du et al. (2017)48
Overall risk of bias Low Indeterminate: unknown interval between test and reference standard Low Low Indeterminate: no description of exclusion criteria, unknown blinding protocol Low Low
Was a consecutive or random sample of patients enrolled? Yes: probability proportionate to size sampling and simple random sampling Yes: consecutive Yes: consecutive Yes: consecutive Yes: consecutive Yes: random University student email list—unclear if truly random
Was a case/control design avoided? Yes Yes Yes Yes Yes Yes Yes
Did the study avoid inappropriate exclusions? Yes Yes: illness preventing completion, previous history of mental illness or psychotherapy, unwilling Yes Yes: disabling disease or reduced life expectancy, difficulty communicating Information not found Yes: patients with psychotic disorders Yes
Were the index testa results interpreted without knowledge of the results of the reference standardb? Yes Yes Yes Yes Yes Yes Yes
Are the specificity and sensitivity recorded for multiple cutoff scores? Yes Yes No: only one (15) Yes Yes Yes Yes
Was the index test administered in a standardized fashion? Yes Yes Yes Yes Yes Yes Yes
Was an appropriate version of the index test used? Information not found Information not found Yes: translated by investigators Yes: translation on Hong Kong government website Information not found Information not found Information not found
Is the reference standard likely to correctly classify the target condition? Yes: MINI Yes: MINI Yes: SCID Yes: MINI Yes: DSM-V Yes: SCID Yes: MINI
Were the reference standard results interpreted without knowledge of the results of the index test? Information not found Yes No: SCID conducted for all patients with PHQ-9≥15 and 8% of patients with PHQ-9<15 Yes Information not found Yes Yes
Was the reference standard administered in a standardized fashion? Yes: one interviewer Yes: standardized training Yes: one interviewer Yes Yes: one interviewer Yes: two psychiatrists with interrater reliability 0.81 Yes: two psychiatrists
Was there an appropriate interval between index tests and reference standard? Yes: <2 weeks Information not found No: “about 4 weeks” No: “within 2–4 weeks” Yes: immediate Yes: 2 weeks Yes: within 48 h
Did all patients receive a reference standard? Yes Yes No: subsample analysis No: subsample analysis Yes No: subsample analysis No: subsample analysis
Did patients receive the same reference standard? Yes Yes Yes Yes Yes Yes Yes
Were all patients included in the analysis without a significant percentage of dropouts? Yes: 0% dropout rate (1045/1045) Yes: 0% dropout rate (782/782) Yes: 0% dropout rate (1940/1940) Yes: 0% dropout rate (586/586) Yes: 0% dropout rate (148/148) Yes: 7% dropout rate (280/300) Yes: 0% dropout rate (230/230)
Question Peng et al. (2020)45 Liu et al. (2016)40 Xiong (2015)47 Zhang et al. (2013)50 Xia (2019)51 Chen (2010)53 Liu et al. (2011)41
Overall risk of bias
Indeterminate: unknown blinding protocol, interval between test and reference standard
Low
Indeterminate: unknown blinding protocol, interval between test and reference standard
Low
Low
Low
Low
Was a consecutive or random sample of patients enrolled?
Yes: consecutive
Yes: study province → random towns → random villages → all individuals
Yes: on randomly assigned days, patients consecutively invited
Yes
Yes: consecutive
Yes: consecutive
Yes: consecutive
Was a case/control design avoided?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Did the study avoid inappropriate exclusions?
Yes: patients with pre-existing mental illness, severe physical illness
Yes: if not in area during study period or difficulty with communication due to illness
Yes: wrong language, limited writing skills, cognitive impairment, psychosis, acute suicidality
Yes
Yes: severe psychiatric disorders or antisocial, schizotypal personality disorders
Yes
Yes
Were the index testa results interpreted without knowledge of the results of the reference standardb?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Are the specificity and sensitivity recorded for multiple cutoff scores?
No: only one (9)
Yes
Yes
Yes
Yes
Yes
Yes
Was the index test administered in a standardized fashion?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Was an appropriate version of the index test used?
Information not found
Yes: cited Yeung et al. 2008
Information not found
Information not found
Information not found
Information not found
Yes: study team translated and back translated
Is the reference standard likely to correctly classify the target condition?
Yes: SCID
Yes: SCID
Yes: MINI
Yes: SCID
Yes: MINI
Yes: SCID
Yes: SCAN
Were the reference standard results interpreted without knowledge of the results of the index test?
Information not found
No: SCID conducted immediately after PHQ-9 by same interviewer
Information not found
Yes
Yes
Yes
Yes
Was the reference standard administered in a standardized fashion?
Yes: two psychiatrists
Yes: standardized interview training
Yes
Yes: standardized training
Yes: one interviewer
Yes: two psychiatrists
Yes: trained staff with 0.88 interrater reliability
Was there an appropriate interval between index tests and reference standard?
Information not found
Yes: immediate
Information not found
Yes: <2 weeks
Yes: <24h
Information not found
Information not found
Did all patients receive a reference standard?
Yes
Yes
Yes
Yes
Yes
No: subsample analysis
Yes
Did patients receive the same reference standard?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Were all patients included in the analysis without a significant percentage of dropouts? Yes: 8% dropout rate (275/300) Yes: 0.7% dropout rate (839/845) Yes: 0% dropout rate (491/491) Significant: 13.8% dropout rate (959/1112) Yes: 0% dropout rate (213/213) Significant: 20% dropout rate (77/96) for subsample chosen to undergo reference standard Significant: 21.6% dropout rate, however, sample was large (1532/1954)
Question Jin et al. (2011)39,c Xu et al. (2007)38,c Yang et al. (2015)37,c Zhi et al. (2013)36,c Chen (2015)35,c Wang et al. (2015)33,c
Overall risk of bias
Indeterminate: unknown blinding protocol, interval between test and reference standard
Low
Low
Low
Low
Low
Was a consecutive or random sample of patients enrolled?
Yes: random
Yes: random
Yes: consecutive
Yes: consecutive
Yes
Yes: consecutive
Was a case/control design avoided?
Yes
Yes
Yes
Yes
Yes
Yes
Did the study avoid inappropriate exclusions?
Yes: dementia or physically incapable of completing questionnaire
Yes
Yes: incapable of understanding and responding to questionnaire, severe mental health symptoms or physical disability, substance use disorders, currently receiving medication or therapy treatment for mental illness
Yes: physically incapable of completing questionnaire, severe physical or mental illness
Yes: those incapable of completing study or consent due to language or hearing
Yes
Were the index test results interpreted without knowledge of the results of the reference standard?
Yes
Yes
Yes
Yes
Information not found
Yes
Are the specificity and sensitivity recorded for multiple cutoff scores?
No: only one (10)
No: only one (15)
No: only one (10)
Yes
Yes
Yes
Was the index test administered in a standardized fashion?
Yes
Yes
Yes
Yes
Yes
Yes
Was an appropriate version of the index test used?
Information not found
Information not found
Information not found
Information not found
Information not found
Yes: cited Liu 2011's translation
Is the reference standard likely to correctly classify the target condition?
Yes: SCID
Yes: SCID
Yes: SCID
Yes: MINI
Yes: MINI
Yes: CIDI
Were the reference standard results interpreted without knowledge of the results of the index test?
Information not found
Information not found
Yes
Information not found
Information not found
Yes
Was the reference standard administered in a standardized fashion?
Yes: standardized training
Yes: two psychiatrists
Yes: standardized training
Yes: standardized training
Yes
Yes: one interviewer
Was there an appropriate interval between index tests and reference standard?
Information not found
Yes: 1 week
Information not found
Yes: immediate
Yes: immediate
Yes: immediate
Did all patients receive a reference standard?
No: subsample analysis
No: subsample analysis
No: subsample analysis
Yes
Yes
Yes
Did patients receive the same reference standards?
Yes
Yes
Yes
Yes
Yes
Yes
Were all patients included in the analysis without a significant percentage of dropouts? Yes: 0% dropout rate (117/117) Yes: 0% dropout rate (195/195) Yes: 0% dropout rate (97/97) 0.6% dropout rate (1997/2009) Yes: 0% dropout rate (319/319) Yes: 0% dropout rate (201/201)
a

Index test=PHQ.

b

Reference standard=old standard clinical interview used by the particular study.

c

Study published in Chinese.