Skip to main content
NCBI home page
Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie logoLink to Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie
. 2015 Feb;60(2 Suppl 1):S55–S60.

Parent–Youth Agreement on Self-Reported Competencies of Youth With Depressive and Suicidal Symptoms

Valentin Mbekou 1, Sasha MacNeil 2, Martin Gignac 3, Johanne Renaud 4,✉
PMCID: PMC4345849  PMID: 25886673

Abstract

Objective:

A multi-informant approach is often used in child psychiatry. The Achenbach System of Empirically Based Assessment uses this approach, gathering parent reports on the Child Behaviour Checklist (CBCL) and youth reports on the Youth Self-Report (YSR), which contain scales assessing both the child’s problems and competencies. Agreement between parent and youth perceptions of their competencies on these forms has not been studied to date.

Method:

Our study examined the parent–youth agreement of competencies on the CBCL and YSR from a sample of 258 parent–youth dyads referred to a specialized outpatient clinic for depressive and suicidal disorders. Intraclass correlation coefficients were calculated for all competency scales (activity, social, and academic), with further examinations based on youth’s sex, age, and type of problem.

Results:

Weak-to-moderate parent–youth agreements were reported on the activities and social subscales. For the activities subscale, boys’ ratings had a strong correlation with parents’ ratings, while it was weak for girls. Also, agreement on activities and social subscales was stronger for dyads with the youth presenting externalizing instead of internalizing problems.

Conclusion:

Agreement on competencies between parents and adolescents varied based on competency and adolescent sex, age, and type of problem.

Keywords: adolescence, depression, psychopathology, inter-informant agreement, risk factors, protective factors, competencies

In child psychiatry, it is common practice to consult multiple informants’ ratings during assessment,1 thereby gathering information from the youth, their parents, and, in some cases, their teachers. Each informant additional to the child is believed to contribute a new perspective on the child’s behaviour depending on the amount of time spent with the child, the individual’s closeness to the child, the situation of the observation, and the purpose of the assessment.2 The study of cross-informant agreement has been of interest to provide effective interventions based on a more complete conceptualization of the youth’s functioning.3

An important tool in child psychiatry is the ASEBA,4 which contains various questionnaires using a multi-informant approach to assess the youth’s functioning. Among the questionnaires comprising the ASEBA, youths provide their own perception of their personal experience on the YSR form while parents complete a comparable form, the CBCL, regarding their perceptions of their children. Research evaluating agreement across multiple informants has repeatedly shown differences between self-reported and parent-reported behaviours and problems in youths. Among clinically referred samples, it has been found that parents report as many or more externalizing problems than do the youths, likely due to the easily observable and public nature of associated behaviours.1,2,5 It has been suggested that adolescents are not the most reliable source for indicating externalizing problems,6 possibly because of a tendency to minimize the importance of their problems on behaviour-rating checklists, or because of denial or lack of insight.7 Conversely, for internalizing problems parental reports tend to underestimate the severity, seeing as youth will report more internalizing problems.2,5 Internalizing problems, therefore, seem to be poorly recognized by parents due to their less observable and more private nature,1,8 and the youths are a better source of this kind of information. Cross-informant agreement is usually expected to be greater for externalizing problems than internalizing problems for this reason, yet mixed results have been found.5,8 Opposite patterns of cross-informant agreement have been observed among nonclinical youth samples; however, youths tend to self-report higher severity ratings and more problems of both kinds in comparison to their parents.1,5,8,9

These findings have been replicated across cultures,5 yet further examination within these patterns of agreement has shown differences among them based on youth characteristics, such as age and sex. Edelbrock et al10 found differences in parent–child agreement on a structured interview that were contingent with the age of the child, such that agreement was stronger among dyads with older youths (14 to 18 years) than those with younger youths (10 to 13 years). These results were also found using the ASEBA’s CBCL and YSR.11

Most research and clinical work to date in the field of adolescent mental health, including the use of the CBCL and YSR, has focused on the identification and treatment of various forms of problems and psychopathologies.6 More recently, there has been growing interest for the role of protective factors and well-being in mental health outcomes and their integration into etiological models of mental health to provide a more complete understanding of functioning.12 Studies6,12,13 have shown the relevance of including well-being as a dimension in mental health models. The ASEBA tools contain both these dimensions, whereby youths are scored on both problem and competencies subscales. The CBCL and YSR contain 2 super-ordinate problem subscales to assess internalizing and externalizing problems, while further assessing 3 competency subscales (activities, social competence, and academics) to obtain an overarching total competence score.14 The activities subscale explores the youth’s implications in sports and nonsports–related activities (how many and how implicated they are within these activities), as well as jobs and job performance. The social competence subscale examines involvement within groups and organizations; involvement with friend groups; behaviour with peers, siblings and family; and amount of time spent alone. The academic subscale evaluates school performance (failures, class rank), whether the youth has repeated grades, is in remedial class, or if other school problems are reported.14

Clinical Implications

  • Clinicians should consider risk and protective factors in the shaping of interventions.

  • Parental reports of youth competencies can be used with weak-to-moderate reliability.

Limitations

  • Our study is limited by the Berkson bias.

  • Competencies reported by parents on the CBCL may not reflect the youths’ actual competencies.

Research has shown that the competencies explored by the CBCL and YSR are related to positive outcomes in adolescent psychopathology. More involvement in activities (sports, social, and academic) during high school has been associated with lower levels of externalizing symptoms, along with less substance use and behaviour disorders.15 Difficulties with peers and (or) teachers, preparing for examinations, reading and writing tasks, self-reliant school performance, and other school performance issues have been associated with elevated depression among both boys and girls.16 Effective social competence has been linked with reduced likelihood of internalizing problems.17 Therefore, the assessment of such competencies may provide information about outcomes beyond the present risk factors.

The ASEBA assessment tools competency scales have not been studied as extensively as the problem scales. The few studies examining the competency scales have reported mixed results. Study of the competencies has found sex differences on the social competence and total competence scales18,19 and others have not.20,21 Age effects were also reported by some, whereby adolescents aged 17–18 years were found to be rated higher on the activity subscale and on total competence than younger adolescents,18 while an inverse relation has been reported by others.21 A few studies have examined cross-informant agreement (including reports from self, peers, parents, and teachers) of youth competencies.22–24 A meta-analysis conducted by Renk and Phares3 reported a small correlation (r = 0.21) between self-reports and parental reports of social competence, with greater agreement among dyads with younger children than with older ones. These studies of cross-informant agreement have not, however, used the CBCL or YSR from the ASEBA.

In light of the importance of well-being on mental health outcomes and the widespread use of the CBCL and YSR in child psychiatry, our study aimed to examine the agreement between parents and youths for youth competencies as reported on the CBCL and YSR. Only a few studies have examined this. Further investigation into youth characteristics that may affect agreement was also undertaken to evaluate whether the degree of agreement would change depending on the youth’s age, sex, and the types of problems they present.

Method

Participants

All youths who were included in the study were referred from first- or second-line health clinics to the Depressive and Suicidal Disorders Outpatient Clinic (youth section) at the Douglas Mental Health University Institute because of depressive symptoms. Permission to access medical files of patients followed from 2006 to 2013 was granted through the Directorate of Professional and Hospital Services of the Douglas Mental Health University Institute. A total of 258 parent–child dyads were included in our study. Upon entering the specialized outpatient clinic, the CBCL and YSR were administered to the parent(s) and the youth as part of the initial routine assessment procedure. The youth sample had a mean age of 15 years (SD 1.58) and was 84.1% female (n = 217). Youth participants were further categorized for analysis purposes based on the types of problems they displayed. Cutoff T-scores were used on the internalizing and externalizing scales of the CBCL parent reports, whereby T-scores of 70 or more on either scale was required to be categorized into the given problem group. The internalizing problems group represented 61.6% of the youth portion of the sample, while 31% were categorized into the externalizing problems group. The rest of the sample (7.4%) did not make the cut offs for either group and were excluded from concerned analyses. For those youth having CBCL forms completed by both parents, the mother’s version was retained for inclusion in the study, whom therefore represent 84.1% (n = 217) of the parent portion of the sample.

Measures

The ASEBA is a preliminary screening tool widely used to measure behaviour and to organize it according to appropriate norms based on age, sex, and informant, and is free from any form of clinician bias.25,26 Two forms from this set of assessment tools were used in our study: the YSR completed by youths, and the CBCL completed by parents. Both the YSR and CBCL contain 2 parts: 2 broadband problem subscales (internalizing and externalizing problems) evaluated on 8 narrowband subscales and 3 competency subscales (activities, social skills, and academic competence). The validity and reliability of both the YSR and CBCL have been established.4

Statistical Analyses

Raw scores for the subscales were used in all analyses because they are more representative of the distribution and are preferred over T-scores for analyses of the CBCL and YSR.9 ICCs were calculated to investigate the parent–child agreement across the activities, social skills, and academic subscales of both forms. Following Hallgreen’s27 descriptions of various common ICC variants, the 1-way random approach for ICC was used in our study. Specific guidelines for interpreting ICCs are not available, therefore Cohen’s conventions are used when other norms are unavailable.28 According to Cohen’s conventions29 for interpreting ICCs, below r = 0.40 are weak, between 0.40 and 0.60 are moderate, and above 0.60 are strong. ICC 95% confidence intervals were also calculated as suggested by Shrout and Fleiss30 when generalizing from a single rating to a mean rating of reliability, as is the case with our study’s methodology, to approximate the population value for these correlations.

Results

The means and standard deviations for the broadband problems and competencies scales are presented in Table 1 along with the sample’s demographic variables. Scores on the problem scales are above average, while scores on the competencies are below average.

Table 1.

Demographic variables and YSR and CBCL scales

Variable n (%) YSR youth-reported T-score (SD) CBCL parent-reported T-score (SD)
Sex
  Male 80 (31)
  Female 178 (69)
Age, years, mean (SD) 15.00 (1.58)
  11 to 14 89 (34.5)
  15 to 18 169 (65.5)
Ethnicity
  Caucasian 221 (85.7)
  Asian 8 (3.1)
  African American 3 (1.2)
  Latino 6 (2.3)
  Native American 2 (0.8)
  Other 10 (3.9)
Externalizing problems 80 (31) 62.92 (10.46) 65.15 (9.50)
Internalizing problems 159 (62) 66.57 (11.77) 71.22 (8.28)
Competencies
  Activities 41.13 (10.67) 40.99 (9.47)
  Social 38.96 (9.43) 38.12 (8.41)
  Academic — 42.27 (9.22)
Open in a new tab

— T-scores are not calculated for the YSR academic subscale

CBCL = Child Behaviour Checklist; YSR = Youth Self-Report

Table 2 describes the ICCs for each competency subscale. Only correlations for the activities and social subscales were significant at P < 0.001, with ICCs r = 0.46 and r = 0.34, respectively, indicating weak-to-moderate agreement between parents and youths on these competency subscales. Correlations for the academic subscale were nonsignificant.

Table 2.

ICC for competency scales

Competencies ICC 95% CI F df P
Activities 0.46 0.35 to 0.60 2.76 200/201 <0.001
Social 0.34 0.21 to 0.45 2.02 188/189 <0.001
Academic −0.05 −0.41 to 0.21 0.95 184/185 0.64
Open in a new tab

ICC = intraclass correlation coefficient

Sex differences were examined to evaluate whether ICCs would show the same degree and direction of agreement. Table 3 summarizes the parent–child ICCs based on sex across all 3 competencies subscales. Correlations for the activities and social subscales were significant, with rs indicating weak agreement for females on the activities subscales and strong agreement for males. Agreement between informants on the social subscale was weak for both males and females. Agreement on the academic subscale was nonsignificant.

Table 3.

ICC for competency scales based on sex

Competency ICC 95% CI F df P
Activities
  Male 0.71 0.55 to 0.81 5.78 61/62 <0.001
  Female 0.36 0.35 to 0.67 2.15 138/139 <0.001
Social
  Male 0.34 0.10 to 0.54 2.03 61/62 0.003
  Female 0.34 0.18 to 0.49 2.03 126/127 <0.001
Academic
  Male 0.18 −0.09 to 0.42 1.43 53/54 0.10
  Female −0.11 −0.28 to 0.06 0.80 130/131 0.90
Open in a new tab

ICC = intraclass correlation coefficient

Cross-informant agreement was further investigated according to age of the youth, whereby those aged 11 to 14 years were compared with adolescents aged 15 through 18 years. Table 4 describes the results for the ICCs separated by age groups. All ICCs on the activities and social subscales were statistically significant. Correlation coefficients indicate moderate agreement on the activities subscale and weak agreement on the social scale across age groups. Informant agreement on the academic subscale did not reach statistical significance.

Table 4.

ICC for competency scales based on age

Competency ICC 95% CI F df P
Activities
  11–14 years 0.42 0.20 to 0.60 2.42 65/66 <0.001
  15–18 years 0.48 0.33 to 0.60 2.81 134/135 <0.001
Social
  11–14 years 0.23 −0.02 to 0.45 1.60 62/63 0.03
  15–18 years 0.37 0.21 to 0.51 2.20 125/126 <0.001
Academic
  11–14 years −0.05 −0.29 to 0.19 0.90 65/66 0.66
  15–18 years −0.01 −0.19 to 0.17 0.98 118/119 0.54
Open in a new tab

ICC = intraclass correlation coefficient

Finally, differences in agreement based on the types of problems the youth displayed (internalizing, compared with externalizing) were examined. Table 5 contains the results from the ICCs separated by type of problem. Again, all correlations for the activities and social subscales were significant at P = 0.001 and indicated weak-to-moderate agreement. Adolescents with externalizing problems had slightly larger agreement with parents than did those with internalizing problems. Agreement on the academic subscale was nonsignificant.

Table 5.

ICC for competency scales based on type of problem

Comptency ICC 95% CI F df P
Activities
  Internalizing 0.43 0.29 to 0.56 2.53 141/142 <0.001
  Externalizing 0.56 0.35 to 0.71 3.50 58/59 <0.001
Social
  Internalizing 0.29 0.13 to 0.44 1.81 134/135 0.001
  Externalizing 0.43 0.19 to 0.62 2.50 53/54 0.001
Academic
  Internalizing −0.20 −0.36 to −0.02 0.672 125/126 0.99
  Externalizing 0.16 −0.09 to 0.40 1.39 58/59 0.10
Open in a new tab

ICC = intraclass correlation coefficient

Discussion

Our study sought to examine the degree of agreement between youths’ perceptions of their competencies and those of their parents. Overall, agreement between the YSR and CBCL was of weak-to-moderate strength for two of the 3 competencies subscales: activities and social. Results did not show informant agreement on the academic subscale across any analyses. Investigation into agreement differences based on age and types of problem (internalizing or externalizing) found weak-to-moderate agreement across both age groups and both problem categories for the activities and social subscales. However, sex differences in agreement were found in the activity subscale, such that there was strong agreement between boys and parents, and weak agreement with girls. These results indicate a stronger agreement between parents and youths than that which has been previously reported by Renk and Phares.3

These findings contribute to the growing literature on protective factors. The conceptualization of an individual’s mental health as a balance between their well-being and psychopathology symptoms can allow for interventions to be directed at people who have traditionally been neglected by health care systems due to their absence of psychopathological symptoms, seeing as the absence of symptoms does not guarantee complete mental health.12 Clinicians, therefore, should take into consideration both the person’s symptoms of psychopathology and his or her protective factors and well-being when assessing mental health. Assessment tools for both risk and protective factors exist. The ASEBA tools are a way of measuring these with the use of a single assessment method through available Diagnostic and Statistical Manual of Mental Disorders-oriented problem scales and competency scales.

Certain limitations of our study should be mentioned. Our study included a clinical sample from a specialized outpatient clinic with youths presenting more complex and severe problems, therefore fewer competencies and more problems were reported due to the nature of the sample. This phenomenon is known as the Berkson bias, whereby the hospital-setting sample used in our study may be inherently biased and provide a distorted perspective of the risks and competencies of similarly aged community populations not followed by this clinic.31 The level of agreement may therefore vary based on the type of sample, such that a community sample may offer a different pattern of agreement, as suggested by previous research comparing clinical and nonclinical youth–parent agreement.1,5,8,9 Future studies may further explore this agreement within various community and clinical samples. Further, the ASEBA measures the informant’s perceptions, which could be different than the youth’s actual competencies. Thus, it is important for future research to include concurrent tools to measure competencies.

Conclusion

Using a multi-informant approach in assessing both youth competencies and problems, we can obtain a more complete overview of the youth’s situation. Our research provides evidence that parental and youth reports on youth competencies can be used with some reliability seeing as parental perceptions moderately approximate those of youths. Results bring some methodological foundation for the interest on youth’s competencies in the ASEBA tools but should indeed be replicated in other clinical settings and in the general population.

Acknowledgments

This research was funded by Grant #20067 from the Réseau Québécois de Recherche sur le Suicide of the Fonds de Recherche du Québec—Santé. Dr Renaud also receives support from the Canadian Institute of Health Research, and is a Standard Life Senior Fellow in Teen Mental Health. Dr Renaud and Dr Mbekou are currently receiving funding from the Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention for a research pilot study. The other authors have no other affiliations to disclose.

Abbreviations

ASEBA

Achenbach System of Empirically Based Assessment

CBCL

Child Behaviour Checklist

ICC

intraclass correlation coefficient

YSR

Youth Self-Report

References

  • 1.van der Ende J, Verhulst FC, Tiemeier H. Agreement of informants on emotional and behavioral problems from childhood to adulthood. Psychol Assess. 2012;24:293–300. doi: 10.1037/a0025500. [DOI] [PubMed] [Google Scholar]
  • 2.Smith SR. Making sense of multiple informants in child and adolescent psychopathology: a guide for clinicians. J Psychoeduc Assess. 2007;25:139–149. [Google Scholar]
  • 3.Renk K, Phares V. Cross-informant ratings of social competence in children and adolescents. Clin Psychol Rev. 2004;24:239–254. doi: 10.1016/j.cpr.2004.01.004. [DOI] [PubMed] [Google Scholar]
  • 4.Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington (VT): University of Vermont, Research Center for Children, Youth, and Families; 2001. [Google Scholar]
  • 5.Rescorla LA, Ginzburg S, Achenbach TM, et al. Cross-informant agreement between parent-reported and adolescent self-reported problems in 25 societies. J Clin Child Adolesc Psychiatry. 2013;42:262–273. doi: 10.1080/15374416.2012.717870. [DOI] [PubMed] [Google Scholar]
  • 6.Suldo SM, Shaffer EJ. Looking beyond psychopathology: the dual-factor model of mental health in youth. School Psych Rev. 2008;37:52–68. [Google Scholar]
  • 7.Salbach-Andrae N, Lenz K, Lehmkuhl U. Patterns of agreement among parent, teacher and youth ratings in a referred sample. Eur Psychiatry. 2009;24:345–351. doi: 10.1016/j.eurpsy.2008.07.008. [DOI] [PubMed] [Google Scholar]
  • 8.Salbach-Andrae S, Klinkowski N, Lenz K, et al. Agreement between youth-reported and parent-reported psychopathology in a referred sample. Eur Child Adolesc Psychiatry. 2009;18:136–143. doi: 10.1007/s00787-008-0710-z. [DOI] [PubMed] [Google Scholar]
  • 9.Achenbach TM, Dumenci L, Rescorla LA. Ten-year comparisons of problems and competencies for national samples of youth: self, parent, and teacher reports. J Emot Behav Disord. 2002;10:194–203. [Google Scholar]
  • 10.Edelbrock C, Costello AJ, Dulcan MK, et al. Parent-child agreement on child psychiatric symptoms assessed via structured interview. J Child Psychol Psychiatry. 1986;27:181–190. [PubMed] [Google Scholar]
  • 11.Rey JM, Schrader E, Morris-Yates A. Parent-child agreement on children’s behaviours reported by the child behavior checklist (CBCL) J Adolesc. 1992;15:219–230. doi: 10.1016/0140-1971(92)90026-2. [DOI] [PubMed] [Google Scholar]
  • 12.Antaramian SP, Huebner S, Hills KJ, et al. A dual-factor model of mental health: toward a more comprehensive understanding of youth functioning. Am J Orthopsychiatry. 2010;80:462–472. doi: 10.1111/j.1939-0025.2010.01049.x. [DOI] [PubMed] [Google Scholar]
  • 13.Breton JJ, Labelle R, Berthiaume C, et al. Protective factors against depression and suicidal behaviour in adolescence. Can J Psychiatry. 2015;60(1 Suppl 1):S5–S15. [PMC free article] [PubMed] [Google Scholar]
  • 14.Achenbach System of Empirically Based Assessment. Manual for the assessment data manager program (ADM) for the CBCL/4–18, YSR, TRF, YASR, YABCL, CBCL/2–3, CBCL/1 2–5& C-TRF. Burlington (VT): University of Vermont College of Medicine; 2000. [Google Scholar]
  • 15.Bohnert AM, Garber J. Prospective relations between organized activity participation and psychopathology during adolescence. J Abnorm Child Psychol. 2007;35:1021–1033. doi: 10.1007/s10802-007-9152-1. [DOI] [PubMed] [Google Scholar]
  • 16.Fröjd SA, Nissinen ES, Pelkonen MUI, et al. Depression and school performance in middle adolescent boys and girls. J Adolesc. 2008;31:485–498. doi: 10.1016/j.adolescence.2007.08.006. [DOI] [PubMed] [Google Scholar]
  • 17.Masten AS, Burt KB, Coatsworth JD. Competence and psychopathology in development. In: Cicchetti D, Cohen DJ, editors. Developmental psychopathology. 2nd ed. Vol. 3. New York (NY): Wiley; 2006. pp. 696–738. [Google Scholar]
  • 18.Broberg AG, Ekeroth K, Gustafsson PA, et al. Self-reported competencies and problems among Swedish adolescents: a normative study of the YSR. Eur Child Adolesc Psychiatry. 2001;10:186–193. doi: 10.1007/s007870170025. [DOI] [PubMed] [Google Scholar]
  • 19.Fitzpatrick C, Deehan A. Competencies and problems of Irish children and adolescents. Eur Child Adolesc Psychiatry. 1999;8:17–23. doi: 10.1007/s007870050079. [DOI] [PubMed] [Google Scholar]
  • 20.Slobodskaya HR. Competence, emotional and behavioural problems in Russian adolescents. Eur Child Adolesc Psychiatry. 1999;8:173–180. doi: 10.1007/s007870050126. [DOI] [PubMed] [Google Scholar]
  • 21.Verhulst FC, Prince V, Vervuurt-Poot C, et al. Mental health in Dutch adolescents: self-reported competencies and problems for ages 11–18. Acta Psychiatr Scand Suppl. 1989;356:1–48. doi: 10.1111/j.1600-0447.1989.tb03050.x. [DOI] [PubMed] [Google Scholar]
  • 22.Coie JD, Dodge KA. Multiple sources of data on social behavior and social status in the school: a cross-age comparison. Child Dev. 1988;59:815–829. doi: 10.1111/j.1467-8624.1988.tb03237.x. [DOI] [PubMed] [Google Scholar]
  • 23.Cole DA, Maxwell SE, Martin JM. Reflected self-appraisals: strength and structure of the relation of teacher, peer, and parent ratings to children’s self-perceived competencies. J Educ Psychol. 1997;89:55–70. [Google Scholar]
  • 24.Ledingham JE, Younger A, Schwartzman A, et al. Agreement among teacher, peer, and self-ratings of children’s aggression, withdrawal, and likability. J Abnorm Child Psychol. 1982;10:363–372. doi: 10.1007/BF00912327. [DOI] [PubMed] [Google Scholar]
  • 25.Althoff RR, Ayer LA, Rettew D, et al. Assessment of dysregulated children using the Child Behavior Checklist: a receiver operating characteristic curve analysis. Psychol Assess. 2010;22:609–617. doi: 10.1037/a0019699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Biedermann J, Petty C, Monuteaux MC, et al. The CBCL-Pediatric Bipolar Disorder Profile predicts a subsequent diagnosis of bipolar disorder and associated impairments in ADHD youth growing up: a longitudinal analysis. J Clin Psychiatry. 2009;70:732–740. doi: 10.4088/JCP.08m04821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hallgreen KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8:23–34. doi: 10.20982/tqmp.08.1.p023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Hemphill JF. Interpreting the magnitudes of correlation coefficients. Am Psychol. 2003;58:78–80. doi: 10.1037/0003-066x.58.1.78. [DOI] [PubMed] [Google Scholar]
  • 29.Cohen J. Statistical power analysis for the behavioral sciences. Mahwah (NJ): Lawrence Erlbaum Associates; 1988. [Google Scholar]
  • 30.Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  • 31.Roberts RS, Spitzer WO, Delmore T, et al. An empirical demonstration of Berkson’s bias. J Chron Dis. 1978;31:119–128. doi: 10.1016/0021-9681(78)90097-8. [DOI] [PubMed] [Google Scholar]

Articles from Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie are provided here courtesy of SAGE Publications

RESOURCES