Abstract
Background
Depression in adolescents is a serious psychiatric illness. GPs play an important role in identifying adolescents with depression and those at risk of developing depression. Few validated tools are suitable for identifying adolescent depression in general practice.
Aim
To determine if three verbally asked key questions are valid for identifying depression in adolescents.
Design and setting
A cross-sectional, general practice multicentre, validation study was conducted in Oslo, Norway, and Aarhus, Denmark.
Method
A total of 294 adolescents answered three verbally asked key questions followed by a Composite International Diagnostic Interview (CIDI) for psychiatric diagnosis. Inclusion criteria were age (14–16 years) and fluency in the Norwegian or Danish language. The primary outcome was ROC curve statistics in terms of sensitivity and specificity, predictive values, and likelihood ratios of the three key questions. Secondary outcomes were Loevinger’s H, Cronbach’s α, and prevalence of depression.
Results
The three key questions met the criteria for construct and criterion validity for detecting depression among the adolescents. ROC curve statistics for the three key questions demonstrated an AUC of 0.79 for the answer ‘yes’ to either screening question and of 0.73 for the answer ‘yes’ to the help question. The positive predictive value was 31% and the negative predictive value was 97%.
Conclusion
The three key questions are useful for identifying depression in adolescents in primary health care.
Keywords: adolescents, depression, primary health care, validation
INTRODUCTION
The use of three key questions have so far been deemed sufficient to detect depression in adult patients. In a study by Arroll et al,1 three key questions (two ‘screening questions’ and one ‘help question’) were asked to consecutive patients in GP waiting rooms, and the GPs provided a possible diagnosis using the answers to the three questions.
Depression in adolescents is a serious psychiatric illness. The incidence of adolescents suffering from depressive disorders seems to be increasing,2,3 and early depression elevates the risk of developing depressive disorders later in life.4 Nevertheless, only one in five of the 13–16-year-olds consecutively attending general practice5 and half of all depressed adolescents are diagnosed before reaching adulthood.6 Depression in teenagers has negative consequences for education, relationships, general functioning, and health.7,8 Many adolescents with depression engage in self-harming behaviour, and 50% of victims of suicide had a medically recorded diagnosis of depression.9,10
Episode duration and recovery from depression can take months or even years,11 and the risk of recurrence is high.12 Evidence has shown that adolescents with previous depressive disorders, many depressive symptoms, and a genetic disposition to depression are at increased risk of developing depression.12–14 A few years ago, the World Health Organization (WHO) ranked ‘depression’ among the biggest health issues on the scale of leading causes of burden of disease (Disability Adjusted Life Year; DALY) for middle- and high-income countries.15 The prevalence of major depressive disorder (MDD) in adolescents is in the range of 2–6%,16–18 with up to 20% of adolescents experiencing symptoms of depression.19,20
GPs play an important role in identifying adolescents with depression and those at risk of developing depression. Mauerhofer et al21 reported that most adolescents chose to contact their GP when consulting for psychological problems. Of those who did not seek help for their psychological problems, 78% had seen their GP at least once during the previous year for other reasons; this suggests deficient diagnostic routines.
Thus, adolescents consulting their GP for physical complaints, unexplained somatic symptoms, irritability, functional impairment, and substance misuse often also suffer from psychological problems.8,22 The challenge for GPs is to recognise depression in this group of adolescents.23 Only a few validated screening or assessment tools are suitable for identifying adolescent depression in general practice. The National Institute for Health and Care Excellence (NICE) guidelines recommend the Mood and Feelings Questionnaire (MFQ),24 and various study groups have developed structured programmes for GPs to manage adolescent depression in primary care,25,26 but these programmes lack diagnostic instruments suited to the working conditions of GPs. A previous study, however, found the Hopkins Symptom Checklist-10 (HSCL-10) to be a valid instrument for identifying depression among adolescents in primary care.27
How this fits in
Depression in teenagers often has serious consequences for life and health. GPs play an important role in identifying adolescents with depression; however, there are only a few validated tools suitable for identifying adolescent depression in general practice. In this study, three short questions, suitable for use in general practice, are validated for identifying depression in adolescents.
Because GPs work to tight timetables, in their first meetings with adolescent patients they would benefit from having short and reliable assessment tools to hand. It is suggested that the three key questions validated by Arroll et al1 for identifying depression in adults could also be used in meetings with young patients. Arroll et al’s study showed good results compared with other tools by providing a more precise identification of depression.
The aim of the Danish-Norwegian Validation (DaNoVa) project was to test if these three questions could provide satisfactory construct and criterion validity for use in a general practice setting to identify depression in adolescents aged 14–16 years.
METHOD
The DaNoVa project was conducted as a general practice multicentre study in collaboration between the Research Unit for General Practice, Aarhus University, Denmark, and the Department of General Practice, University of Oslo, Norway. The protocol was published in 2007.28
A total of 43 GPs in Aarhus (n = 18) and Oslo (n = 25) were invited to participate in a cross-sectional validation study of adolescents. Included in the study were all patients aged 14–16 years listed with the GPs. Every adolescent was provided with a code for anonymisation. Included in the study were 1137 adolescents from Aarhus and 1222 from Oslo.
After the study group received the GPs’ lists, a standard letter of invitation to participate was sent to the adolescents. The invitation letter contained information about study objectives and procedures as well as a form with the individual key code containing two questionnaires. A letter was sent to each of the participating adolescents together with the questionnaires HSCL-6, HSCL-10,27 and WHO-5. Analyses of these results will not be treated further in this article.
The adolescents responded either by completing the hard-copy form sent by post or an online questionnaire. Hard-copy and online versions of the questionnaires were identical. The individual key code was used as a login ID to access the website and all responses were identified through the key code.
All participants stated an individual telephone contact number and a member of the study group called each adolescent and completed the Composite International Diagnostic Interview (CIDI) by telephone. The interviewers, who were all experienced GPs certified for CIDI, were blinded for questionnaire responses. The telephone interview began with three key questions followed by the depression and the demographic modules from the CIDI. The results were recorded electronically and pooled with the results from all questionnaires in the same database.
Key questions
The first two key questions (‘screening questions’) and the help question are presented in Box 1. Possible responses to the first two questions were ‘no’ or ‘yes’. Possible responses to the help question were ‘no’, ‘yes, but not today’, or ‘yes’. If the adolescent answered ‘yes’ to one or both of the first questions, the help question was added.
Box 1. Two key questions and a help question.
During the past month have you often been bothered by feeling down, depressed, or hopeless?
During the past month have you often been bothered by little interest or pleasure in doing things?
Is this something with which you would like help?
CIDI
CIDI (version 2.1) was used as a gold standard interview. The module for depressive disorders can provide detection of seven different depressive disorders based on the DSM-IV and ICD-10 (F32.0–F34.1). A diagnosis within the previous 2 weeks was defined as ongoing, whereas a diagnosis before that was defined as previous. The key questions were tested against the gold standard test (CIDI) on the same participant. This is similar to an individual test–retest method in which the individual has a greater effect on the outcome than the sample.
Analyses were performed on complete data only. Both construct and criterion validity were examined: construct validity by the use of Cronbach’s α and the Mokken analysis, and criterion validity through Receiver Operating Curves (ROC) with specific measurement for sensitivity, specificity, and likelihood ratios. The data files were analysed using STATA (version 11).
RESULTS
Participation invitation letters were sent to 2370 adolescents (1164 male and 1206 female). A total of 373 (15% of the invited males and 18% of the females) agreed to participate; 12% of these came from Norway and 21% from Denmark. A total of 294 of these adolescents were interviewed by telephone; 145 from Denmark and 149 from Norway. Among the CIDI-interviewed youth, 33 (11%) were diagnosed with a current depressive episode; 25 females and eight males. Thirteen (4%) interviewed participants met the ICD-10 criteria for moderate to severe depression (Table 1).
Table 1.
Depression diagnosis (ICD-10) by CIDI
| CIDI diagnoses | Frequency | % |
|---|---|---|
| No diagnoses | 261 | 88.78 |
| F32.0 Mild depression | 11 | 3.74 |
| F32.1 Moderate depression | 6 | 2.04 |
| F32.2 Severe depression | 4 | 1.36 |
| F33.1 Recurrent moderate depression | 1 | 0.34 |
| F33.2 Recurrent severe depression | 2 | 0.68 |
| F34.1 Dysthymia | 9 | 3.06 |
| Total | 294 | 100 |
CIDI = Composite International Diagnostic Interview.
Construct validity
The construct validity of the three questions was calculated using Loevinger’s H and Cronbach’s α (Table 2). Loevinger’s H computes the observed and expected errors for each pair of items between a given item and all other items of a scale, or among all possible pairs of items in a scale. Loevinger’s H >0.50 is interpreted as a strong scale.29 For the three key questions, Loevinger’s H was calculated at 0.71.
Table 2.
Construct validation of the key questions compared with the CIDI: Mokken scale analysis and Cronbach’s α
Cronbach’s α is used as a measure of the reliability of a psychometric instrument. It provides an expression of increased consistency, which increases as the correlation between items increases. A psychometric instrument is generally considered reliable if Cronbach’s α is >0.70;30 the α value for the three key questions was 0.74, which complies with this criterion. A comparison of Cronbach’s α values in the Feldt’s W-test showed no significant difference between Norway and Denmark (P = 0.90).
Criterion validity
For criterion validity, two different analyses were performed. First, the diagnostic accuracy of two screening questions was compared with two screening questions and the help question. Second, a calculation of sensitivity, specificity, and likelihood ratios was performed.
The diagnostic accuracy of the three key questions resulted in an AUC (Area Under Curve) for the ROC curve as presented in Figure 1. AUCs >70% indicate good diagnostic accuracy.31 For the total population, the AUC was 0.79 for two screening questions and 0.73 for two screening questions and the help question.
Figure 1.
ROC curves for key questions (keyq = answer ‘yes’ to both key questions) and key questions plus help question (keyqhelp = answer ‘yes’ to both key questions and the help question).
Criterion validity was assessed for two screening questions and for two screening questions and the help question. Results are presented in Tables 3 and 4. The answer ‘yes’ to two screening questions had a sensitivity of 82% and a specificity of 77%. Specificity increased to 98% (females constituting 97%) with 93% correctly classified for the answer ‘yes’ to both screening questions and the help question. For males, the specificity for two screening questions was 90%. No males had more than two positive answers.
Table 3.
Criterion validity for three key questions using CIDI as gold standard
| Key questions | Sensitivity % (95% CI) | Specificity % (95% CI) | Correctly classified % | Positive likelihood ratio |
|---|---|---|---|---|
| A | 82 (0.66 to 0.92) | 77 (0.71 to 0.81) | 78 | 3.6 |
| B | 48 (0.32 to 0.65) | 98 (0.96 to 0.99) | 93 | 25 |
A = answer ‘yes’ to either or both key questions. B = answer ‘yes’ to either or both key questions and ‘yes’ to help question. The values represent all depression diagnoses. Sensibility and specificity, both sexes.
Loevinger’s H > 0.50.29
Cronbach’s α >0.70.30 CIDI = Composite International Diagnostic Interview.
Table 4.
Criterion validity for both sexes using CIDI as gold standard for depression diagnosis
| Key questions | Sensitivity %, females/males | Specificity %, females/males | Correctly classified %, females/males | Positive likelihood ratio, females/males |
|---|---|---|---|---|
| A | 84/75 | 66/90 | 69/89 | 2.49/7.44 |
| B | 56/25 | 97/100 | 90/95.70 | 15.90/- |
A = answer ‘yes’ to either or both key questions. B = answer ‘yes’ to either or both key questions and ‘yes’ to help question.
The likelihood ratio increased from 3.6 to 25 with the answer ‘yes’ also to the help question.
The positive predictive value (PPV) for the three key questions (two screening questions and the help question) was 31%, whereas the negative predictive value (NPV) was 97%.
DISCUSSION
Summary
This study has confirmed that the three verbally asked key questions could be used for identifying depression in 14–16-year-olds. The first two questions can be used for identifying depression and the use of the help question seems to increase the accuracy of the assessment tool. The key questions demonstrated similar characteristics of measuring depression (construct and criterion validity) in Denmark and Norway.
When analysing the validity of the three key questions, it is important to keep in mind that the question about ‘wanting help’ is asked just after the two screening questions. These two questions contain several items that are relevant for identifying depression, and the two questions can be used in random order.
For every positive answer, more diagnoses of depression are correctly classified and the increased specificity implies that fewer participants are diagnosed false positive. A decrease in sensitivity enhances the risk of false negative diagnoses, but the risk is decreased when using the three questions successively. Also, the increase in likelihood ratio verifies that the use of all three questions is essential for completing the scale.
To the authors’ knowledge, there are no previous studies that have validated three key questions as an instrument to identify depression in young people, in line with Arroll’s key questions to identify depression in adults.1 The results of the validation of the three key questions match well, however, with results of a previous study in the same population using a 10-item questionnaire.27
Strengths and limitations
For females, specificity increased to 97% and the positive likelihood ratio more than doubled when the answer was ‘yes’ to all three questions. No males with a depression diagnosis answered ‘yes’ to all questions, but those who answered ‘yes’ to two questions already had a high specificity of 90%, a high likelihood ratio, and 87% were classified correctly.
The high accuracy for males could be related to the low prevalence of depression among this group compared with females. Also, males with depressive symptoms may present fewer false positive answers than females, and males may only declare depressive symptoms and need for help if they are obviously depressed to themselves or showing outward signs of depression, thus making this assessment tool even more effective for males. The 11% prevalence of depression in this study results from findings based on an unselected population. Case finding for adolescents who are suspected to be at risk of depression would probably present a higher prevalence in general practice. The adolescents in the study live in rather big cities and the prevalence of depression may differ from adolescents who live in other areas.
In terms of weaknesses, the low participation rate (21% in Denmark and 12% in Norway) may have caused a selection bias; the sensitive information required and the two-step procedure may influence the response rate; the study lacked information about non-responders, which may have influenced the sample representativeness; and there is little information about the sample participant characteristics.
Comparison with existing literature
It is not known whether a sufficiently representative sample of individuals was tested in this study. The validity of the results is strengthened, however, by the identified prevalence of depression being similar to the findings from other studies, and the findings being similar in Norway and Denmark despite different response rates.16,18,32 Moreover, representativeness is of minor importance as the main point in this study was to analyse the reliability of the three key questions measured against a gold standard (CIDI) in the same individual. The lower participation rate in Norway compared with Denmark is probably partly a result of the required parental consent for the 14- and 15-year-old participants in Norway. It is noteworthy that the results were similar between the two countries despite differences in response rates. This suggests that an adolescent’s inclination to participate is unaffected by mental state, which again suggests a generalisability of the findings for this age group.
Another limitation may be the sample size of 294 adolescents, which was slightly lower than required in the protocol calculation.28 During the study, however, the sensitivity of the instruments proved to be higher than first approximated, and the recruitment of participants was therefore stopped early. One final possible objection is that the gold standard instrument used, CIDI, is conducted as a telephone interview. There is good evidence, however, that telephone interviews can be used in psychiatric assessment,33,34 and in general this method may allow the participant increased anonymity and emotional distance from the interviewer in comparison with face-to-face interviewing. CIDI is a comprehensive and fully standardised diagnostic interview that has shown good inter-rater reliability with high κ values.35,36
Implications for research and practice
The three key questions asked verbally offer a new form of assessment tool for adolescents compared with the available written questionnaires. The challenge is to bring this tool into clinical practice by implementing the key questions in a conversation or interview with the patient. For this to be successful, the GP must be particularly aware of adolescents at risk of depression and must also be experienced in formulating and using these key questions. The threshold for using the three key questions must be low, for example, for young people who are regulars in primary care with somatic complaints.
The three key questions could be asked verbally at any time during the consultation process with an adolescent in whom the GP suspects depression. The questions are not designed to be used as a screening tool, but rather as a case-finding method, in which the strength of the tool is primarily that the three key questions can be asked in a short time, but also that the three questions can help to enhance the skills of the GP in detecting depression in daily practice. Positive responses must always be followed by a clinical assessment of depression before initiating a further plan of treatment. Further training for healthcare professionals in managing adolescent depression is recommended.26,37,38
Because young people with mental health issues in need of help often contact their GP for other health-related issues without addressing their mental health,21 the key questions may provide the clinician with an instrument that captures broad indications of depression.
The three key questions may also be suitable for use in other healthcare settings, but practical issues must be considered regarding the administration of the test: taking responsibility, using confidentiality, and carrying out any followup and treatment required for adolescent depression.
In conclusion, the validity of the three questions makes them attractive and viable for identifying depression in adolescents in primary health care. More research is needed to evaluate the effect of the key questions as used by primary health professionals in daily clinical practice.
Acknowledgments
The authors thank the participants in this study for their cooperation.
Funding
The study was funded by the Research Fund for Clinical Development in General Practice in Denmark and the Fund for General Practice Research in Norway.
Ethical approval
The study was approved by the Danish Data Protection Agency, the Danish Health Research Ethics Committee (2006-2.0/44), the Norwegian Social Science Data Services, and the National Committees for Medical and Health Research Ethics in Norway (S-06293a). If a response from an adolescent participating in the study indicated risk of major depression, the GP concerned was contacted and necessary treatment would be provided.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: bjgp.org/letters
REFERENCES
- 1.Arroll B, Goodyear-Smith F, Kerse N, et al. Effect of the addition of a ‘help’ question to two screening questions on specificity for diagnosis of depression in general practice: diagnostic validity study. BMJ. 2005;331(7521):884. doi: 10.1136/bmj.38607.464537.7C. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ballenger JC, Davidson JR, Lecrubier Y, et al. Consensus statement on the primary care management of depression from the International Consensus Group on Depression and Anxiety. J Clin Psychiatry. 1999;60( Suppl 7):54–61. [PubMed] [Google Scholar]
- 3.Burke KC, Burke JD, Jr, Rae DS, Regier DA. Comparing age at onset of major depression and other psychiatric disorders by birth cohorts in five US community populations. Arch Gen Psychiatry. 1991;48(9):789–795. doi: 10.1001/archpsyc.1991.01810330013002. [DOI] [PubMed] [Google Scholar]
- 4.Birmaher B, Ryan ND, Williamson DE, et al. Childhood and adolescent depression: a review of the past 10 years. Part II. J Am Acad Child Adolesc Psychiatry. 1996;35(12):1575–1583. doi: 10.1097/00004583-199612000-00008. [DOI] [PubMed] [Google Scholar]
- 5.Kramer T, Garralda ME. Psychiatric disorders in adolescents in primary care. Br J Psychiatry. 1998;173:508–513. doi: 10.1192/bjp.173.6.508. [DOI] [PubMed] [Google Scholar]
- 6.Kessler RC, Avenevoli S, Ries MK. Mood disorders in children and adolescents: an epidemiologic perspective. Biol Psychiatry. 2001;49(12):1002–1014. doi: 10.1016/s0006-3223(01)01129-5. [DOI] [PubMed] [Google Scholar]
- 7.Lewinsohn PM, Rohde P, Seeley JR. Major depressive disorder in older adolescents: prevalence, risk factors, and clinical implications. Clin Psychol Rev. 1998;18(7):765–794. doi: 10.1016/s0272-7358(98)00010-5. [DOI] [PubMed] [Google Scholar]
- 8.Yates P, Kramer T, Garralda E. Depressive symptoms amongst adolescent primary care attenders. Levels and associations. Soc Psychiatry Psychiatr Epidemiol. 2004;39(7):588–594. doi: 10.1007/s00127-004-0792-y. [DOI] [PubMed] [Google Scholar]
- 9.Portzky G, van Heeringen K. Deliberate self-harm in adolescents. Curr Opin Psychiatry. 2007;20(4):337–342. doi: 10.1097/YCO.0b013e3281c49ff1. [DOI] [PubMed] [Google Scholar]
- 10.Shaffer D, Fisher P, Lucas CP, et al. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry. 2000;39(1):28–38. doi: 10.1097/00004583-200001000-00014. [DOI] [PubMed] [Google Scholar]
- 11.Gledhill J, Garralda ME. The short-term outcome of depressive disorder in adolescents attending primary care: a cohort study. Soc Psychiatry Psychiatr Epidemiol. 2011;46(10):993–1002. doi: 10.1007/s00127-010-0271-6. [DOI] [PubMed] [Google Scholar]
- 12.Lewinsohn PM, Rohde P, Seeley JR, et al. Natural course of adolescent major depressive disorder in a community sample: predictors of recurrence in young adults. Am J Psychiatry. 2000;157(10):1584–1591. doi: 10.1176/appi.ajp.157.10.1584. [DOI] [PubMed] [Google Scholar]
- 13.Thapar A, Collishaw S, Potter R, Thapar AK. Managing and preventing depression in adolescents. BMJ. 2010;340:c209. doi: 10.1136/bmj.c209. [DOI] [PubMed] [Google Scholar]
- 14.Thapar A, Collishaw S, Pine DS, Thapar AK. Depression in adolescence. Lancet. 2012;379(9820):1056–1067. doi: 10.1016/S0140-6736(11)60871-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.World Health Organization . The global burden of disease: 2004 update. Geneva: WHO; 2008. http://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_full.pdf?ua=1 (accessed 16 Dec 2015). [Google Scholar]
- 16.Costello EJ, Erkanli A, Angold A. Is there an epidemic of child or adolescent depression? J Child Psychol Psychiatry. 2006;47(12):1263–1271. doi: 10.1111/j.1469-7610.2006.01682.x. [DOI] [PubMed] [Google Scholar]
- 17.Ford T, Goodman R, Meltzer H. The British Child and Adolescent Mental Health Survey 1999: the prevalence of DSM-IV disorders. J Am Acad Child Adolesc Psychiatry. 2003;42(10):1203–1211. doi: 10.1097/00004583-200310000-00011. [DOI] [PubMed] [Google Scholar]
- 18.Williams SB, O’Connor EA, Eder M, Whitlock EP. Screening for child and adolescent depression in primary care settings: a systematic evidence review for the US Preventive Services Task Force. Pediatrics. 2009;123(4):e716–e735. doi: 10.1542/peds.2008-2415. [DOI] [PubMed] [Google Scholar]
- 19.Haavet OR, Dalen I, Straand J. Depressive symptoms in adolescent pupils are heavily influenced by the school they go to. A study of 10th grade pupils in Oslo, Norway. Eur J Public Health. 2006;16(4):400–404. doi: 10.1093/eurpub/cki197. [DOI] [PubMed] [Google Scholar]
- 20.Saluja G, Iachan R, Scheidt PC, et al. Prevalence of and risk factors for depressive symptoms among young adolescents. Arch Pediatr Adolesc Med. 2004;158(8):760–765. doi: 10.1001/archpedi.158.8.760. [DOI] [PubMed] [Google Scholar]
- 21.Mauerhofer A, Berchtold A, Michaud PA, Suris JC. GPs’ role in the detection of psychological problems of young people: a population-based study. Br J Gen Pract. 2009 doi: 10.3399/bjgp09X454115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Ferrin M, Gledhill J, Kramer T, Elena GM. Factors influencing primary care attendance in adolescents with high levels of depressive symptoms. Soc Psychiatry Psychiatr Epidemiol. 2009;44(10):825–833. doi: 10.1007/s00127-009-0004-x. [DOI] [PubMed] [Google Scholar]
- 23.Iliffe S, Williams G, Fernandez V, et al. Treading a fine line: is diagnosing depression in young people just medicalising moodiness? Br J Gen Pract. 2009 doi: 10.3399/bjgp09X407081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.National Institute for Health and Care Excellence . Depression in children and young people: identification and management CG28. London: NICE; 2005. https://www.nice.org.uk/guidance/cg28 (accessed 9 Dec 2015). [PubMed] [Google Scholar]
- 25.Therapeutic identification of depression in young people: identification and treatment manual. London: Imperial College London & Londsdale Medical Centre; 2008. TIDY Project Academic Unit of Child and Adolescent Psychiatry. https://www1.imperial.ac.uk/resources/677CF992-2457-47A6-A8A2-9B440CA399AD/tidymanual.pdf (accessed 16 Dec 2015). [Google Scholar]
- 26.Zuckerbrot RA, Jensen PS. Improving recognition of adolescent depression in primary care. Arch Pediatr Adolesc Med. 2006;160(7):694–704. doi: 10.1001/archpedi.160.7.694. [DOI] [PubMed] [Google Scholar]
- 27.Haavet OR, Sirpal MK, Haugen W, Christensen KS. Diagnosis of depressed young people in primary health care — a validation of HSCL-10. Fam Pract. 2011;28(2):233–237. doi: 10.1093/fampra/cmq078. [DOI] [PubMed] [Google Scholar]
- 28.Haavet OR, Christensen KS, Sirpal MK, Haugen W. Diagnosis of depression among adolescents — a clinical validation study of key questions and questionnaire. BMC Fam Pract. 2007;8:41. doi: 10.1186/1471-2296-8-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Mokken RJ. A theory and procedure of scale analysis: methods and models in the social sciences. Reprint edition. Berlin: Mouton de Gruyter; 1971. [Google Scholar]
- 30.Bland JM, Altman DG. Cronbach’s alpha. BMJ. 1997;314:572. doi: 10.1136/bmj.314.7080.572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Henderson AR. Assessing test accuracy and its clinical consequences: a primer for receiver operating characteristic curve analysis. Ann Clin Biochem. 1993;30(Pt 6):521–539. doi: 10.1177/000456329303000601. [DOI] [PubMed] [Google Scholar]
- 32.Haavet OR, Straand J, Saugstad OD, Grunfeld B. Illness and exposure to negative life experiences in adolescence: two sides of the same coin? A study of 15-year-olds in Oslo, Norway. Acta Paediatr. 2004;93(3):405–411. [PubMed] [Google Scholar]
- 33.Evans M, Kessler D, Lewis G, et al. Assessing mental health in primary care research using standardized scales: can it be carried out over the telephone? Psychol Med. 2004;34(1):157–162. doi: 10.1017/s0033291703008055. [DOI] [PubMed] [Google Scholar]
- 34.Poutanen O, Koivisto AM, Joukamaa M, et al. The Depression Scale as a screening instrument for a subsequent depressive episode in primary healthcare patients. Br J Psychiatry. 2007;191:50–54. doi: 10.1192/bjp.bp.106.024752. [DOI] [PubMed] [Google Scholar]
- 35.Andrews G, Peters L, Guzman AM, Bird K. A comparison of two structured diagnostic interviews: CIDI and SCAN. Aust N Z J Psychiatry. 1995;29(1):124–132. doi: 10.3109/00048679509075901. [DOI] [PubMed] [Google Scholar]
- 36.Wittchen HU. Reliability and validity studies of the WHO — Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res. 1994;28(1):57–84. doi: 10.1016/0022-3956(94)90036-1. [DOI] [PubMed] [Google Scholar]
- 37.Cheung AH, Zuckerbrot RA, Jensen PS, et al. Guidelines for Adolescent Depression in Primary Care (GLAD-PC): II. Treatment and ongoing management. Pediatrics. 2007;120(5):e1313–e1326. doi: 10.1542/peds.2006-1395. [DOI] [PubMed] [Google Scholar]
- 38.Zuckerbrot RA, Cheung AH, Jensen PS, et al. Guidelines for Adolescent Depression in Primary Care (GLAD-PC): I. Identification, assessment, and initial management. Pediatrics. 2007;120(5):e1299–e1312. doi: 10.1542/peds.2007-1144. [DOI] [PubMed] [Google Scholar]