Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Nov 1.
Published in final edited form as: Compr Psychiatry. 2009 Mar 10;50(6):584–592. doi: 10.1016/j.comppsych.2009.01.005

Latent Class Analysis of the Child Behavior Checklist Obsessive-Compulsive Scale

Robert R Althoff 1, David C Rettew 1, Dorret I Boomsma 2, James J Hudziak 1,2
PMCID: PMC2806056  NIHMSID: NIHMS158102  PMID: 19840599

Abstract

The Obsessive Compulsive Scale (OCS) of the Child Behavior Checklist (CBCL) predicts Obsessive-Compulsive Disorder and is highly heritable. Latent Class Analysis (LCA) of the OCS was used to identify profiles within this 8-item scale and to examine heritability of those profiles. LCA was performed on maternal CBCL reports of their 6–18 year-old children from 2 U.S. nationally representative samples from 1989 (n=2475, 50% male) and 1999 (n=2029, 53% male) and from Dutch Twins in the Netherlands Twin Registry at ages 7 (n=10,194, 49.3% male), 10 (n=6448, 48.1% male), and 12 (n=3674, 48.6% male). The heritability of the resultant classes was estimated using odds ratios of twin membership across classes. A 4-class solution fit all samples best. The resulting classes were a “no or few symptoms” class, a “worries and has to be perfect” class, a “thought problems class, and an “OCS” class. Within class odds ratios were higher than across class odds ratios and were higher for MZ than DZ twins. We conclude that LCA identifies an OCS class and that class is highly heritable using across-twin comparisons.

Keywords: OCD, Latent Class Analysis, Genetics, Twins, CBCL

Introduction

Obsessive Compulsive Disorder (OCD) in childhood occurs at an estimated rate of 0.13–0.25 per 100 children with most adult cases beginning with symptoms before age 18 [1, 2]. Recently, there has been interest in using the Child Behavior Checklist (CBCL) [3, 4] to screen for OCD in general population and clinical samples. Nelson and colleagues first demonstrated that an 8-item scale from the CBCL could distinguish OCD clinical controls and the general population [5]. We expanded on that original work, demonstrating that the factor-analytically derived solution could be reliably applied in the clinic using a cutpoint approach [6]. We also demonstrated the heritability of this Obsessive Compulsive Scale (OCS) of the CBCL using twin samples [7] and demonstrated the stability of the OCS phenotype [8]. Our group has expanded these findings to demonstrate the heritability in adult samples [9]. Several other groups have tested various adaptations of the CBCL-OCS, including a 6-item version [10] a 3-item version [11], and a 2, 4, or 10-item version [12]. We sought here to determine whether latent variable modeling could shed light on the question of whether the original 8-items hold together as a scale or whether they represent simply a concatenation of items from the Anxious/Depressed (AD) and Thought Problems (TP) scales and whether refining the OCS using latent variable modeling would further improve heritability estimation. Latent variable models have been crucial tools in the study of psychopathology.

Latent Class Analysis (LCA) has been used successfully to advance the phenotypic understand of ADHD [13–17], eating disorders [18, 19], alcohol and drug dependence [20, 21], autism [22], temperament [23], tic disorders [24], juvenile bipolar disorder [25], and co-occurring disorders with OCD [26], among others. It offers the clinician and researcher the opportunity to place each individual into a statistically independent class with others who respond or behave in a like manner. This differentiates LCA from factor analysis which is performed at the variable level with items being placed together on the basis of how they load onto particular latent factors and has been used in identifying possible subgroups of OCD symptoms in children and adults [27]. These factor-analytically derived groupings of symptoms have enhanced genetic studies of OCD [28].

We performed LCA of the OCS in several samples to see whether classification into discrete classes could be obtained and then to see how this structure informed genetic models of the OCS. Because the OCS is derived from two factor-analytically defined subscales of the CBCL, we hypothesized that the latent classes would fall along 2 dimensions that measured those continuous latent constructs. Because of data demonstrating similar genetic structure of the OCS throughout childhood [8], we hypothesized that the same model would fit samples across age. Finally, given the heritability of the OCS, we hypothesized that monozygotic twins would have higher odds ratios of being placed into the same class than dizygotic twins as has been demonstrated in other heritable childhood disorders [25].

Methods and Materials

Participants

Data on children and adolescents were derived from three sources. First, for determining the model fitting a general population sample, we analyzed data from nonclinically referred participants taken from the CBCL 1989 national sample (CBCL-89) [3]. We verified this in a sample that contained clinically-referred and nonclinically referred participants taken from the CBCL 1999 national sample (CBCL-99) [4]. Briefly, in both of these samples, data were obtained from home interview surveys with the parents of participants chosen to be representative of the contiguous 48 states. These surveys included the CBCL and other questions regarding demographics and the participant’s mental health. The CBCL-89 consisted of 2475 children aged 6–20 (50% male). The mean age of boys was 13.02 (SD = 3.75) and girls was 12.97 (SD = 3.75). The CBCL-99 consisted of 2029 children aged 6–18 (53% male). The mean age of boys was 11.94 (SD = 3.56) and girls was 12.02 (SD = 3.50). Items from the 8-item version of the CBCL-OCS were selected. Data were analyzed with all participants included, and covariates were included for age and sex.

After running LCA on the CBCL-89 and CBCL-99, we applied the same analysis on maternal reports of twins at ages 7, 10, and 12 from the Netherlands Twin Registry (NTR7, NTR10, NTR12, respectively). The characteristics of this sample are described elsewhere [29–31]. The study is part of an ongoing longitudinal twin-family study of health-related characteristics, personality, and behavior in the Netherlands. Mothers returned the CBCL by mail. We used samples from the 1986–1994 period of data collection, including data from 10,194 (49.3% male) twins aged 7, 6448 (48.1% male) twins age 10, and 3674 (48.6% male) twins age 12. There was considerable overlap among these three samples, as they were taken from a combined cross-sectional/longitudinal study. 5107 (50.0%) of the NTR7 were also in the NTR10, 3029 (47.0%) of the NTR10 were also in the NTR12, and 2926 (28.7%) of the NTR7 were also in the NTR12.

All data collection and analysis was approved by human subjects review boards at either the University of Vermont, the VU University Amsterdam, or both. All subjects participated with informed voluntary consent.

Measures

The CBCL is a standardized questionnaire used for parents to respond to 118 problem behaviors exhibited by their child over the previous 6 months. The parent responds along a 3 point scale with 0 = “not true”, 1 = “somewhat or sometimes true”, and 2 = “very true or often true”. The characteristics and psychometric stability of the CBCL have been well established in American [3, 4] and Dutch [32] samples. The analyses performed here used the 2001 version of the CBCL for the American sample and the 1989 version for the Dutch sample. The items on the OCS are the same across the two versions.

The OCS was developed using factor analysis on 11 CBCL items thought to likely predict OCD [5, 6]. Using a 1 factor model, 8 items were retained and were shown to have good internal consistency (Cronbach’s alpha = 0.84). The items are shown in Table 1, along with their CBCL item number.

Table 1.

Items used for the Obsessive-Compulsive Scale (OCS)

CBCL Item Number CBCL Item CBCL Syndrome on which Item is Scored
66 Repeats certain acts over and over; compulsions Thought problems
84 Strange behavior Thought problems
85 Strange ideas Thought problems
9 Can’t get his/her mind off certain thoughts; obsessions Thought problems
31 Feels he/she might think or do something bad Anxious/depressed
52 Feels too guilty Anxious/depressed
112 Worries Anxious/depressed
32 Feels he/she has to be perfect Anxious/depressed
Open in a new tab

Latent Class Analysis (LCA)

LCA is a form of categorical data analysis which seeks to identify a number of mutually exclusive respondent classes (M) with similar endorsement profiles along a set of nominal or ordinal-measured items. LCA presupposes the existence of discrete categories or classes, distinguishing it from factor analysis which assumes continuous latent variables are present [33]. Local independence is assumed – i.e. that under an M-class solution, the conditional probabilities of endorsing a set of items are statistically independent for a given class [34]. As the number of latent classes estimate increases, it is assumed that homogenous classes or types will be defined such that individuals within a class will differ in symptom endorsement profiles only because of measurement error or stochastic factors. The resulting parameter estimates are class membership probabilities and symptom endorsement probabilities for each class.

Latent class models were computed using an Expectation Maximization (EM) algorithm [35], using the program Latent Gold 4.0 [36]. Models estimating 1-class through 5-class solutions were compared. To calculate the best fitting model, we compared an M class solution to an M+1 class solution. We used as a guideline the change in the Bayes Information Criterion (BIC), and the sample-size Adjusted Bayes Information Criterion (ABIC) goodness-of-fit indices that consider the rule of parsimony. Models were chosen if moving from the M to the M+1 solution led to a decrease in the BIC while retaining reasonable face validity. The ABIC was used if the differences between two models were questionable. For the U.S. samples, analyses were performed using sex and age as covariates and for the Dutch samples, analyses were initially performed using sex as a covariate (because samples were at ages 7, 10, and 12). The covariates were then dropped to determine if the fit worsened substantially. Given that the model was first fit to unrelated children in the CBCL national samples, and the fits with the Dutch twin data were nearly identical, we did not control for familiality in the NTR models.

Twin comparisons

Because simultaneously modeling the genetics of the probability of class membership and latent class membership has been demonstrably difficult, we estimated within-twin similarity with odds ratios using logistic regression in SPSS (version 15.0.1; [37]. The most likely class membership for both twins was calculated and a series of logistic regressions was run for each class separately with membership in a particular class coded as 1 or 0 for each twin. The odds ratio and 95% confidence interval around each estimate was calculated for twin type (MZ, DZ) and sex separately. Dizygotic twins who were of the opposite sex were not included in this analysis. This approach has been used by others to provide a window on heritability using a latent classes approach [14].

Results

Model fitting

Five latent class models were fitted to the data, representing a 1-class through a 5-class solution. As the number of classes increased from 1-class through 4-class models, the BIC and ABIC either decreased appreciably or the increase was minimal (Table 2). The 4-class model was considered the accepted model on the basis of the parsimony measures. The graphs for the 4-class solution are presented below. Dropping age as a covariate did not appreciably affect model fitting, but dropping sex as a covariate did. This is consistent with the model fits across the NTR data which showed essentially the same model, regardless of age.

Table 2.

Fit statistics for LCA models. Best model is indicated in bold.

Sample Class Solution Npar LL BIC ABIC
NTR7 1 Class, sex cov 16 −25123.6494 50394.97 50344.13
2 Class, sex cov 26 −23716.8184 47673.61 47590.98
3 Class, sex cov 36 −23586.3521 47504.97 47390.57
4 Class, sex cov 46 −23473.5711 47371.7 47225.52
5 Class, sex cov 56 −23442.2212 47401.3 47223.34
4 Class, drop sex cov 43 −23499.7894 47396.45 47259.8
NTR10 1 Class, sex cov 16 −17477.8734 35096.09 35045.25
2 Class, sex cov 26 −16221.9845 32672.03 32589.41
3 Class, sex cov 36 −16130.5097 32576.79 32462.4
4 Class, sex cov 46 −16052.8891 32509.27 32363.09
5 Class, sex cov 56 −16029.2822 32549.77 32371.82
4 Class, drop sex cov 43 −16069.3853 32515.95 32379.3
NTR12 1 Class, sex cov 16 −9119.9681 18371.28 18320.44
2 Class, sex cov 26 −8374.781 16963 16880.38
3 Class, sex cov 36 −8297.1347 16889.79 16775.4
4 Class, sex cov 46 −8241.1747 16859.97 16713.8
5 Class, sex cov 56 −8215.7246 16891.16 16713.21
4 Class, drop sex cov 43 −8260.7955 16874.58 16737.95
CBCL89 1 Class, sex and age cov 16 −9758.7347 19642.49 19588.61
2 Class, sex and age cov 27 −9141.6874 18494.35 18403.42
3 Class, sex and and cov 38 −9066.1205 18429.17 18301.19
4 Class, sex and age cov 49 −9005.713 18394.31 18229.28
5 Class, sex and age cov 60 −8980.1718 18429.18 18227.11
4 Class, drop sex cov 46 −9015.4726 18390.39 18235.46
4 Class drop age cov 46 −9007.2492 18373.94 18219.02
CBCL99 1 Class, sex and age cov 16 −7874.2422 15870.33 15819.5
2 Class, sex and age cov 27 −7459.1776 15123.97 15038.19
3 Class, sex and and cov 38 −7410.2618 15109.91 14989.18
4 Class, sex and age cov 49 −7371.3643 15115.88 14960.2
5 Class, sex and age cov 60 −7351.3761 15159.67 14969.05
4 Class, drop sex cov 46 −7425.9735 15202.25 15056.11
4 Class drop age cov 46 −7372.3365 15094.98 14948.83
Open in a new tab

Npar = number of parameters. LL = log-likelihood, BIC = Bayes Information Criterion, ABIC = sample size Adjusted Bayes Information Criterion, cov = covariate.

Class assignments

The latent classes for each sample, including prevalence of assignment of individuals to each class, are presented in Figures 1–4. The most common class was one with no or few symptoms (No Symptoms), with a probability ranging from .47–.82, with the differences appearing between US and Dutch samples. The lowest probability was in the CBCL-99 sample which included referred children. The next most common class demonstrated high responding primarily on the items from the anxious-depressed scale (Worries and Has to be Perfect) with a class membership probabilities ranging from .12–.41, with more children in the CBCL-99 placed into this category. For all samples, the third class consisted of relatively higher endorsement on the items from the Thought Problems scale (Thought Problems) with a class membership probabilities ranging from .04–.08 with more males than females being placed into this class across all samples (see Table 3). The final, and least common, class consisted of responses that endorsed high levels of all items (OCS) with class membership probabilities ranging from .01–.07. The classes were markedly similar, regardless of sample.

Figure 1.

Figure 1

Open in a new tab

Class 1: No symptoms class.

Figure 4.

Figure 4

Open in a new tab

Class 4: Obsessive-Compulsive Scale (OCS) class.

Table 3.

Class membership probabilities, average item scores, and average total OCS scores for the 4-class solution across samples. For class membership probabilities and proportion in class with OCS >= 5, proportions are given and absolute numbers are in parentheses. For item scores and OCS scores, mean score is given with SD in parentheses.

Sample Sex Class Proportion (n) in class Proportion (n) class with OCS >=5 Compulsions Strange Behavior Strange Ideas Obsessions Fear Do Bad Guilty Worries Be Perfect Total OCS
NTR 7 ♂ 1 .83 (4158) 0 (0) 0.01 (0.08) 0.02 (0.12) 0 (0) 0.06 (0.23) 0.01 (0.12) 0.01 (0.1) 0.02 (0.15) 0.24 (0.48) 0.37 (0.57)
2 .10 (477) .05 (25) 0.03 (0.16) 0.03 (0.17) 0.02 (0.14) 0.27 (0.55) 0.42 (0.55) 0.34 (0.5) 0.59 (0.53) 1.1 (0.63) 2.8 (0.9)
3 .07 (339) .12 (39) 0.24 (0.6) 0.73 (0.78) 0.31 (0.53) 0.80 (0.84) 0.04 (0.21) 0.06 (0.23) 0.3 (0.48) 0.34 (0.52) 2.82 (1.31)
4 .01 (56) .98 (55) 0.3 (0.66) 0.73 (0.8) 0.64 (0.7) 0.91 (0.82) 0.79 (0.62) 0.88 (0.69) 1.2 (0.55) 1.46 (0.66) 6.91 (2.03)
♀ 1 .79 (4105) 0 (0) 0 (0.05) 0.01 (0.12) 0 (0) 0.05 (0.22) 0.02 (0.13) 0 (0) 0.02 (0.16) 0.25 (0.44) 0.37 (0.52)
2 .16 (839) .06 (47) 0.02 (0.18) 0.03 (0.16) 0.03 (0.18) 0.29 (0.55) 0.35 (0.51) 0.33 (0.48) 0.45 (0.53) 1.22 (0.68) 2.72 (1)
3 .03 (164) .11 (18) 0.29 (0.64) 0.68 (0.73) 0.31 (0.48) 0.87 (0.84) 0.03 (0.17) 0.07 (0.26) 0.21 (0.44) 0.22 (0.46) 2.68 (1.21)
4 .01 (56) .98 (55) 0.38 (0.73) 0.46 (0.63) 0.71 (0.68) 0.98 (0.8) 0.77 (0.63) 0.91 (0.72) 1.14 (0.55) 1.38 (0.65) 6.73 (1.91)
NTR 10 ♂ 1 .80 (2487) 0 (0) 0 (0.06) 0.02 (0.12) 0 (0) 0.05 (0.22) 0.01 (0.11) 0.01 (0.11) 0.04 (0.19) 0.25 (0.48) 0.38 (0.56)
2 .11 (330) .13 (43) 0.02 (0.12) 0 (0.06) 0.01 (0.08) 0.37 (0.61) 0.42 (0.54) 0.41 (0.51) 0.68 (0.52) 1.1 (0.63) 3 (1.09)
3 .08 (255) .11 (27) 0.26 (0.59) 0.55 (0.69) 0.29 (0.51) 0.87 (0.84) 0.09 (0.29) 0.11 (0.31) 0.41 (0.52) 0.45 (0.59) 3.03 (1.33)
4 .01 (32) 1.0 (32) 0.31 (0.69) 0.72 (0.73) 0.69 (0.82) 1.38 (0.79) 0.75 (0.72) 1.06 (0.62) 1.31 (0.59) 1.63 (0.49) 7.84 (1.55)
♀ 1 .80 (2667) 0 (0) 0 (0.05) 0.01 (0.09) 0 (0) 0.05 (0.22) 0.01 (0.12) 0.01 (0.1) 0.04 (0.19) 0.3 (0.51) 0.43 (0.59)
2 .16 (495) .11 (56) 0.01 (0.12) 0.01 (0.08) 0.02 (0.13) 0.41 (0.65) 0.37 (0.52) 0.48 (0.55) 0.66 (0.54) 1.03 (0.62) 2.99 (1.1)
3 .04 (128) .13 (17) 0.37 (0.69) 0.51 (0.69) 0.4 (0.55) 0.83 (0.86) 0.03 (0.17) 0.13 (0.33) 0.43 (0.53) 0.43 (0.6) 3.12 (1.15)
4 .02 (54) .98 (54) 0.24 (0.58) 0.63 (0.76) 0.63 (0.71) 0.96 (0.78) 0.67 (0.58) 1.15 (0.56) 1.35 (0.55) 1.59 (0.57) 7.22 (1.93)
NTR 12 ♂ 1 .81 (1453) 0 (0) 0 (0) 0 (0) 0 (0) 0.04 (0.19) 0.01 (0.09) 0.01 (0.11) 0.04 (0.19) 0.24 (0.48) 0.34 (0.54)
2 .09 (162) .04 (7) 0.02 (0.16) 0.04 (0.19) 0.01 (0.08) 0.2 (0.41) 0.22 (0.41) 0.44 (0.53) 0.79 (0.45) 1.02 (0.54) 2.73 (0.83)
3 .07 (120) .07 (8) 0.34 (0.65) 0.63 (0.69) 0.28 (0.51) 0.68 (0.73) 0.05 (0.22) 0.05 (0.22) 0.31 (0.48) 0.29 (0.47) 2.64 (1.35)
4 .03 (52) 1.0 (52) 0.46 (0.8) 0.63 (0.74) 0.5 (0.67) 1.48 (0.73) 0.42 (0.57) 0.88 (0.68) 1.17 (0.68) 1.38 (0.69) 6.94 (1.86)
♀ 1 .82 (1547) 0 (0) 0 (0.03) 0 (0.07) 0 (0) 0.03 (0.18) 0.01 (0.1) 0.02 (0.13) 0.04 (0.19) 0.28 (0.5) 0.38 (0.56)
2 .15 (279) .05 (15) 0.03 (0.19) 0.02 (0.15) 0.02 (0.13) 0.27 (0.51) 0.24 (0.44) 0.48 (0.54) 0.75 (0.5) 0.99 (0.53) 2.8 (0.9)
3 .02 (40) .20 (8) 0.35 (0.66) 0.6 (0.71) 0.43 (0.64) 0.93 (0.89) 0 (0) 0.03 (0.16) 0.23 (0.42) 0.4 (0.67) 2.95 (1.41)
4 .01 (21) 1.0 (21) 0.29 (0.64) 0.62 (0.8) 0.67 (0.66) 1.05 (0.8) 0.52 (0.6) 0.95 (0.5) 1.14 (0.65) 1.33 (0.73) 6.57 (0.98)
CBCL89 ♂ 1 .58 (723) 0 (0) 0.02 (0.13) 0 (0) 0 (0) 0.13 (0.34) 0.03 (0.17) 0.01 (0.07) 0.09 (0.28) 0.2 (0.4) 0.46 (0.56)
2 .25 (304) .03 (9) 0.01 (0.08) 0 (0) 0 (0) 0.41 (0.55) 0.35 (0.5) 0.16 (0.37) 0.8 (0.51) 1.01 (0.68) 2.72 (0.84)
3 .10 (117) .14 (16) 0.43 (0.73) 0.15 (0.41) 0.5 (0.58) 0.99 (0.76) 0.18 (0.43) 0.07 (0.25) 0.34 (0.49) 0.38 (0.55) 3.04 (1.26)
4 .08 (93) .94 (87) 0.57 (0.76) 0.22 (0.51) 0.26 (0.53) 1.31 (0.67) 0.9 (0.65) 0.81 (0.58) 1.3 (0.55) 1.33 (0.63) 6.68 (1.69)
♀ 1 .54 (664) 0 (0) 0.01 (0.12) 0 (0) 0 (0) 0.09 (0.29) 0.03 (0.18) 0 (0) 0.13 (0.34) 0.18 (0.39) 0.45 (0.51)
2 .35 (434) .08 (33) 0.02 (0.13) 0 (0) 0 (0) 0.49 (0.61) 0.33 (0.49) 0.18 (0.38) 0.77 (0.52) 1.05 (0.67) 2.84 (0.97)
3 .06 (71) .17 (12) 0.49 (0.63) 0.37 (0.59) 0.51 (0.58) 0.92 (0.79) 0.24 (0.43) 0.03 (0.17) 0.27 (0.45) 0.34 (0.56) 3.15 (1.29)
4 .06 (69) .97 (67) 0.57 (0.76) 0.22 (0.51) 0.26 (0.53) 1.31 (0.67) 0.9 (0.65) 0.81 (0.58) 1.3 (0.55) 1.33 (0.63) 6.68 (1.69)
CBCL99 ♂ 1 .48 (515) 0 (0) 0.01 (0.12) 0.01 (0.09) 0.01 (0.12) 0.1 (0.31) 0 (0) 0 (0) 0 (0) 0.21 (0.41) 0.35 (0.48)
2 .38 (409) .02 (6) 0.03 (0.18) 0 (0) 0.05 (0.23) 0.43 (0.59) 0.2 (0.41) 0.09 (0.29) 0.75 (0.51) 0.76 (0.72) 2.33 (0.98)
3 .09 (98) .30 (24) 0.74 (0.79) 0.31 (0.53) 0.26 (0.46) 1.21 (0.69) 0.41 (0.59) 0.15 (0.39) 0.36 (0.5) 0.23 (0.47) 3.67 (1.34)
4 .05 (51) .98 (50) 0.39 (0.7) 0.04 (0.2) 0.25 (0.56) 1.06 (0.76) 0.96 (0.66) 0.96 (0.63) 1.33 (0.55) 1.41 (0.61) 6.41 (1.51)
♀ 1 .46 (445) 0 (0) 0.01 (0.09) 0.01 (0.09) 0.01 (0.11) 0.09 (0.29) 0 (0) 0 (0) 0 (0.05) 0.21 (0.41) 0.33 (0.48)
2 .45 (426) .02 (10) 0.05 (0.23) 0 (0.05) 0.05 (0.24) 0.5 (0.64) 0.21 (0.42) 0.09 (0.29) 0.75 (0.49) 0.72 (0.71) 2.37 (1)
3 .01 (7) .43 (3) 1.14 (0.9) 0.57 (0.79) 0.29 (0.49) 1.57 (0.53) 0.43 (0.79) 0.43 (0.53) 0.29 (0.49) 0 (0) 4.71 (1.8)
4 .08 (78) .87 (68) 0.38 (0.69) 0.12 (0.36) 0.21 (0.49) 0.97 (0.81) 0.79 (0.69) 0.76 (0.65) 1.41 (0.61) 1.41 (0.71) 6.05 (1.63)
Open in a new tab

Twin Cross-Class Odds Ratios

The OR’s across twins for each of the NTR samples are in Table 4. Significant OR’s are defined as those where the 95% CI does not cross 1. Because of low numbers within certain cells, not all OR’s were able to be calculated (and are listed as “n.c.” in the Table). For the remainder of the comparisons, it is clear that the majority of significant OR’s fall along the diagonal – representing within-class similarity across twins. In cases where significant odds ratios were found between different classes, they tended to be between group 4 (OCS) and either group 2 (Worries and Has to be Perfect) or group 3 (Thought Problems). It is also clear that the OR’s within each class are higher in the MZ twins than in the DZ twins for nearly every comparison. The OR’s were verified with Pearson correlations of the probability of class membership across twins, although non-independence of the measures makes this less acceptable. The pattern of correlation was the same (data available on request).

Table 4.

Odds ratios (and 95% confidence intervals) between classes for MZ and DZ twins across age and sex.

Monozygotic Dizygotic
Age class 1 2 3 4 1 2 3 4
7
♂ 1 15.54 (9.99–24.19) 0.12 (0.07–0.20) 0.11 (0.06–0.20) 0.03 (0.00–0.29) 6.46 (4.33–9.64) 0.22 (0.14–0.35) 0.17 (0.09–0.31) 0.35 (0.10–1.24)
2 0.13 (0.08–0.22) 12.34 (7.09–21.49) 0.43 (0.10–1.81) 11.63 (2.30–58.79) 0.31 (0.19–0.51) 4.31 (2.46–7.54) 1.52 (0.62–3.70) n.c.
3 0.08 (0.04–0.15) 1.83 (0.83–4.05) 25.71 (13.09–50.52) 3.22 (0.37–28.12) 0.17 (0.10–0.29) 2.49 (1.32–4.72) 7.68 (3.92–15.05) 5.04 (1.27–19.97)
4 0.03 (0.00–0.28) 3.65 (0.70–19.11) 11.80 (2.57–54.20) 27.17 (2.75–268.89) 0.13 (0.04–0.40) 3.78 (1.14–12.53) 5.20 (1.38–19.58) 7.21 (0.85–61.50)
♀ 1 12.50 (8.58–18.20) 0.11 (0.07–0.15) 0.17 (0.08–0.35) 0.06 (0.01–0.53) 3.79 (2.61–5.49) 0.32 (0.22–0.47) 0.32 (0.15–0.69) 0.35 (0.12–1.01)
2 0.09 (0.06–0.13) 12.00 (8.01–17.97) 1.69 (0.71–4.00) 3.66 (0.61–22.07) 0.26 (0.17–0.39) 4.00 (2.62–6.12) 1.23 (0.46–3.32) 2.19 (0.68–7.11)
3 0.23 (0.11–0.50) 0.84 (0.29–2.47) 21.05 (8.48–52.27) 8.66 (0.94–80.23) 0.54 (0.26–1.12) 0.76 (0.29–2.01) 9.60 (3.73–24.71) n.c.
4 0.15 (0.04–0.65) 2.95 (0.70–12.49) 4.41 (0.53–37.01) 32.86 (3.25–332.36) 0.31 (0.08–1.27) 1.45 (0.29–7.24) n.c 20.72 (3.79–113.31)
10
♂ 1 11.16 (6.70–18.58) 0.12 (0.06–0.21) 0.17 (0.10–0.34) 0.10 (0.01–1.15) 4.8 (3.02–7.64) 0.24 (0.14–0.40) 0.32 (0.16–0.65) 0.43 (0.07–2.58)
2 0.15 (0.08–0.27) 18.56 (9.59–35.93) 0.21 (0.03–1.56) n.c. 0.20 (0.11–0.35) 8.45 (4.53–15.78) 0.23 (0.03–1.70) 5.84 (0.95–35.79)
3 0.15 (0.07–0.30) 0.26 (0.04–1.93) 23.49 (10.67–51.68) n.c. 0.43 (0.24–0.80) 1.00 (0.43–2.31) 5.34 (2.48–11.47) n.c.
4 n.c. 3.14 (0.32–30.68) 4.29 (0.44–42.22) 540 (33.73–8644.34) 0.20 (0.04–0.91) 1.07 (0.13–9.00) 10.52 (2.26–48.99) n.c.
♀ 1 11.43 (7.44–17.55) 0.13 (0.09–0.21) 0.20 (0.09–0.44) 0.08 (0.02–0.29) 3.97 (2.43–6.50) 0.29 (0.17–0.49) 0.49 (0.17–1.43) 0.28 (0.09–0.91)
2 0.15 (0.10–0.24) 9.47 (5.86–15.32) 0.51 (0.12–2.18) 0.56 (0.07–4.39) 0.24 (0.14–0.42) 4.46 (2.50–7.93) 0.44 (0.06–3.37) 3.77 (1.10–12.92)
3 0.11 (0.05–0.25) 1.22 (0.49–3.06) 30.20 (12.25–74.42) 7.89 (2.02–30.80) 0.53 (0.20–1.38) 0.84 (0.24–2.95) 9.40 (2.74–32.24) n.c.
4 0.06 (0.01–0.26) 3.57 (1.11–11.46) n.c. 67.25 (17.12–264.13) 0.31 (0.06–1.57) 2.28 (0.41–12.66) n.c. 8.6 (0.93–79.88)
12
♂ 1 10.09 (5.23–19.45) 0.32 (0.14–0.72) 0.14 (0.06–0.34) 0.02 (0.00–0.18) 2.59 (1.33–5.06) 0.49 (0.21–1.15) 0.3 (0.11–0.83) 0.81 (0.16–4.00)
2 0.23 (0.10–0.55) 4.92 (1.85–13.13) 1.31 (0.29–5.96) 3.91 (0.76–19.98) 0.30 (0.13–0.70) 2.83 (1.03–7.72) 3.32 (1.00–11.04) 1.21 (0.15–10.07)
3 0.22(0.08–0.57) 0.53 (0.07–4.10) 8.84 (2.94–26.62) 5.21 (1.00–27.15) 1.16 (0.38–3.57) 0.37 (0.05–2.85) 1.59 (0.34–7.47) 1.46 (0.18–12.27)
4 0.04(0.01–0.17) 2.96 (0.77–11.38) 7.06 (1.98–25.16) 26.58 (6.10–115.89) 0.08 (0.01–0.79) 8.71 (1.18–64.30) 5.33 (0.53–54.21) n.c.
♀ 1 10.6 (5.90–19.04) 0.09 (0.05–0.17) 1.47 (0.17–12.40) 0.08 (0.01–0.76) 4.62 (2.26–9.41) 0.32 (0.14–0.70) 0.29 (0.05–1.80) 0.09 (0.02–0.52)
2 0.08 (0.04–0.14) 15.38 (8.05–29.40) n.c. 5.47 (0.76–39.65) 0.32 (0.14–0.70) 3.44 (1.48–7.99) 1.74 (0.19–15.99) 1.38 (0.16–12.19)
3 1.02 (0.21–4.93) n.c. 6.61 (0.72–60.75) 13.33 (1.26–140.91) 0.19 (0.05–0.78) 0.98 (0.12–8.21) 9.39 (0.93–95.23) 21.92 (3.34–143.69)
4 0.25 (0.04–1.81) 4.86 (0.67–35.13) n.c. n.c. 0.10 (0.01–1.09) 3.5 (0.31–39.65) n.c. 26.70 (2.07–344.80)
Open in a new tab

N.C. = not calculable.

Discussion

LCA identifies a profile that is consistent with the OCS. This class structure is very highly consistent over the ages from 7–12 and across two different countries (American and Dutch samples). The prevalence of individuals placed into a particular class may change by sample, sex, or age, but the general class structure is the same. In families with twins, the odds of a twin falling into the same class as his or her co-twin is higher than the odds of that the twins will be in different classes. Moreover, this is more likely in monozygotic compared to dizygotic twins, which supports the heritability of these classes. Thus, these data indicate that the classes are statistically and, for the most part, genetically discrete, although with some overlap particularly among the three more symptomatic groups. The statistical ability to simultaneously measure class membership and specific heritability estimates is being explored by our group and others [38, 39]. As demonstrated in ADHD [17] and mood dysregulation [25] in children, there are clear associations between sharing the same DNA and being in the same latent class for OC behavior. These findings speak to the ongoing issue of how best to characterize both problem and typically occurring behavior in studies that search for their genetic and environmental roots. Todd and colleagues have argued persuasively that these latent constructs are useful in gene finding as a complement to “top-down” DSM constructs [40, 41].

Of additional import here is the class with few symptoms. This class is always identified in general population studies of problem behavior. The odds ratios for the low or no symptoms class were higher for MZ than for DZ twins, giving some indication of a genetic influence of being in this low or no symptoms class. This speaks to the genetics of wellness: an important topic which has been much less discussed or researched [42]. The apparent heritability of the No Symptoms class may be driven by children who are especially non-anxious and non-obsessional. Modeling of scales where both strengths and weaknesses can be assessed is a focus of some of our present and future work [42, 43].

Finally, this work speaks to the usefulness of the OCS scale as a whole. While some revisions of the OCS as a measure of OCD may be warranted [10, 11], it continues to garner empirical support as a naturally occurring cluster of behaviors. Storch et al., 2006 showed that a 6-item version of the OCS dropping the “Strange Ideas” and “Has to be Perfect” items was the most robust. In the analyses here, the American samples have the “Strange Ideas” item endorsed at lower rates than the other items, even in the most severe class. Conversely, in the American samples, the “Has to be Perfect” item is frequently endorsed even in the No Symptoms class. However, excluding this item from the OCS would remove a potentially clinically meaningful class (Worries and Has to be Perfect Class), which may represent children with anxiety unrelated to OCD. It is possible that this class is capturing a temperamental trait like neuroticism that has links to more classic OC symptoms. We are exploring temperamental profiles in a sample of children who fall into this class to determine these relations.

The OCS does not contain all items for OCD from the DSM-IV; thus use of these scales is not a direct test of DSM-IV OCD or of latent classes of obsessive-compulsive behavior. Furthermore, data on maternal reports may not generalize to children using self-reports. However, the OCS was constructed to use maternal reporting to predict clinically-significant OCD as defined by the CY-BOCS [5]. Finally, we can not present data on the number of children who fell in to the latent classes who also met DSM-IV diagnostic criteria for OCD. Our group is interviewing a subset of this sample and analyzing these data to determine those relations.

Figure 2.

Figure 2

Open in a new tab

Class 2: Worries and Has to be Perfect Class.

Figure 3.

Figure 3

Open in a new tab

Class 3: Thought Problems Class.

Acknowledgments

This research was supported by grants NIMH MH58799, NWO 904-57-94, NWO 575-25-006 and the Centre for Neurogenomics and Cognition Research (CNCR). A portion of this work was previously presented at the 52nd Annual meeting of the American Academy of Child and Adolescent Psychiatry.

The authors would like to acknowledge Thomas Achenbach for his generosity in allowing the use of these data and for comments on the interpretation of these results. Finally, we would like to thank all of the participants, twins, and families who participated in this research.

Footnotes

CONFLICTS OF INTEREST

None of the authors have any conflicts of interest to disclose with regard to the content of this manuscript.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Costello EJ, Angold A, Burns BJ, Stangl DK, Tweed DL, Erkanli A, et al. The Great Smoky Mountains Study of Youth. Goals, design, methods, and the prevalence of DSM-III-R disorders. Arch Gen Psychiatry. 1996 Dec;53(12):1129–36. doi: 10.1001/archpsyc.1996.01830120067012. [DOI] [PubMed] [Google Scholar]
  • 2.Steinhausen HC, Metzke CW, Meier M, Kannenberg R. Prevalence of child and adolescent psychiatric disorders: the Zurich Epidemiological Study. Acta Psychiatr Scand. 1998 Oct;98(4):262–71. doi: 10.1111/j.1600-0447.1998.tb10082.x. [DOI] [PubMed] [Google Scholar]
  • 3.Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 Profile. Burlington, VT: University of Vermont Department of Psychiatry; 1991. [Google Scholar]
  • 4.Achenbach TM, Rescorla LA. Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families; 2001. [Google Scholar]
  • 5.Nelson EC, Hanna GL, Hudziak JJ, Botteron KN, Heath AC, Todd RD. Obsessive-compulsive scale of the child behavior checklist: specificity, sensitivity, and predictive power. Pediatrics. 2001 Jul;108(1):E14. doi: 10.1542/peds.108.1.e14. [DOI] [PubMed] [Google Scholar]
  • 6.Hudziak JJ, Althoff RR, Stanger C, van Beijsterveldt CE, Nelson EC, Hanna GL, et al. The Obsessive Compulsive Scale of the Child Behavior Checklist predicts obsessive-compulsive disorder: a receiver operating characteristic curve analysis. J Child Psychol Psychiatry. 2006 Feb;47(2):160–6. doi: 10.1111/j.1469-7610.2005.01465.x. [DOI] [PubMed] [Google Scholar]
  • 7.Hudziak JJ, Van Beijsterveldt CE, Althoff RR, Stanger C, Rettew DC, Nelson EC, et al. Genetic and environmental contributions to the Child Behavior Checklist Obsessive-Compulsive Scale: a cross-cultural twin study. Arch Gen Psychiatry. 2004 Jun;61(6):608–16. doi: 10.1001/archpsyc.61.6.608. [DOI] [PubMed] [Google Scholar]
  • 8.van Grootheest DS, Bartels M, Cath DC, Beekman AT, Hudziak JJ, Boomsma DI. Genetic and environmental contributions underlying stability in childhood obsessive-compulsive behavior. Biol Psychiatry. 2007 Feb 1;61(3):308–15. doi: 10.1016/j.biopsych.2006.05.035. [DOI] [PubMed] [Google Scholar]
  • 9.Van Grootheest DS, Cath DC, Beekman AT, Boomsma DI. Genetic and environmental influences on obsessive-compulsive symptoms in adults: a population-based twin-family study. Psychol Med. 2007 Nov;37(11):1635–44. doi: 10.1017/S0033291707000980. [DOI] [PubMed] [Google Scholar]
  • 10.Storch EA, Murphy TK, Bagner DM, Johns NB, Baumeister AL, Goodman WK, et al. Reliability and validity of the Child Behavior Checklist Obsessive-Compulsive Scale. J Anxiety Disord. 2006;20(4):473–85. doi: 10.1016/j.janxdis.2005.06.002. [DOI] [PubMed] [Google Scholar]
  • 11.Geller DA, Doyle R, Shaw D, Mullin B, Coffey B, Petty C, et al. A quick and reliable screening measure for OCD in youth: reliability and validity of the obsessive compulsive scale of the Child Behavior Checklist. Compr Psychiatry. 2006 May-Jun;47(3):234–40. doi: 10.1016/j.comppsych.2005.08.005. [DOI] [PubMed] [Google Scholar]
  • 12.Ivarsson T, Larsson B. The Obsessive-Compulsive Symptom (OCS) scale of the Child Behavior Checklist: A comparison between Swedish children with Obsessive-Compulsive Disorder from a specialized unit, regular outpatients and a school sample. J Anxiety Disord. 2008 Oct;22(7):1172–9. doi: 10.1016/j.janxdis.2007.12.004. [DOI] [PubMed] [Google Scholar]
  • 13.Neuman RJ, Todd RD, Heath AC, Reich W, Hudziak JJ, Bucholz KK, et al. Evaluation of ADHD typology in three contrasting samples: A latent class approach. J Am Acad Child Adolesc Psychiatry. 1999;38(1):25–33. doi: 10.1097/00004583-199901000-00016. [DOI] [PubMed] [Google Scholar]
  • 14.Todd RD, Rasmussen ER, Neuman RJ, Reich W, Hudziak JJ, Bucholz KK, et al. Familiality and heritability of subtypes of attention deficit hyperactivity disorder in a population sample of adolescent female twins. Am J Psychiatry. 2001 Nov;158(11):1891–8. doi: 10.1176/appi.ajp.158.11.1891. [DOI] [PubMed] [Google Scholar]
  • 15.Hudziak JJ, Heath AC, Madden PF, Reich W, Bucholz KK, Slutske W, et al. Latent class and factor analysis of DSM-IV ADHD: a twin study of female adolescents. J Am Acad Child Adolesc Psychiatry. 1998 Aug;37(8):848–57. doi: 10.1097/00004583-199808000-00015. [DOI] [PubMed] [Google Scholar]
  • 16.Rasmussen ER, Neuman RJ, Heath AC, Levy F, Hay DA, Todd RD. Familial clustering of latent class and DSM-IV defined attention-deficit/hyperactivity disorder (ADHD) subtypes. J Child Psychol Psychiatry. 2004 Mar;45(3):589–98. doi: 10.1111/j.1469-7610.2004.00248.x. [DOI] [PubMed] [Google Scholar]
  • 17.Althoff RR, Copeland WE, Stanger C, Derks EM, Todd RD, Neuman RJ, et al. The latent class structure of ADHD is stable across informants. Twin Res Hum Genet. 2006 Aug;9(4):507–22. doi: 10.1375/183242706778025008. [DOI] [PubMed] [Google Scholar]
  • 18.Keel PK, Fichter M, Quadflieg N, Bulik CM, Baxter MG, Thornton L, et al. Application of a latent class analysis to empirically define eating disorder phenotypes. Arch Gen Psychiatry. 2004 Feb;61(2):192–200. doi: 10.1001/archpsyc.61.2.192. [DOI] [PubMed] [Google Scholar]
  • 19.Bulik CM, Sullivan PF, Kendler KS. An empirical study of the classification of eating disorders. Am J Psychiatry. 2000 Jun;157(6):886–95. doi: 10.1176/appi.ajp.157.6.886. [DOI] [PubMed] [Google Scholar]
  • 20.Muthen B. Should substance use disorders be considered as categorical or dimensional? Addiction. 2006 Sep;101(Suppl 1):6–16. doi: 10.1111/j.1360-0443.2006.01583.x. [DOI] [PubMed] [Google Scholar]
  • 21.Whitesell NR, Beals J, Mitchell CM, Novins DK, Spicer P, Manson SM. Latent class analysis of substance use: comparison of two American Indian reservation populations and a national sample. Journal of studies on alcohol. 2006 Jan;67(1):32–43. doi: 10.15288/jsa.2006.67.32. [DOI] [PubMed] [Google Scholar]
  • 22.Constantino JN, Przybeck T, Friesen D, Todd RD. Reciprocal social behavior in children with and without pervasive developmental disorders. J Dev Behav Pediatr. 2000 Feb;21(1):2–11. doi: 10.1097/00004703-200002000-00002. [DOI] [PubMed] [Google Scholar]
  • 23.Rettew DC, Althoff RR, Dumenci L, Ayer L, Hudziak JJ. Latent profiles of temperament and their relations to psychopathology and wellness. J Am Acad Child Adolesc Psychiatry. 2008 Mar;47(3):273–81. doi: 10.1097/CHI.0b013e318160b403. [DOI] [PubMed] [Google Scholar]
  • 24.Grados MA, Mathews CA. Latent class analysis of gilles de la tourette syndrome using comorbidities: clinical and genetic implications. Biol Psychiatry. 2008 Aug 1;64(3):219–25. doi: 10.1016/j.biopsych.2008.01.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Althoff RR, Rettew DC, Faraone SV, Boomsma DI, Hudziak JJ. Latent class analysis shows strong heritability of the child behavior checklist-juvenile bipolar phenotype. Biol Psychiatry. 2006 Nov 1;60(9):903–11. doi: 10.1016/j.biopsych.2006.02.025. [DOI] [PubMed] [Google Scholar]
  • 26.Nestadt G, Di CZ, Riddle MA, Grados MA, Greenberg BD, Fyer AJ, et al. Obsessive-compulsive disorder: subclassification based on co-morbidity. Psychol Med. 2008 Dec 2;:1–11. doi: 10.1017/S0033291708004753. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Stewart SE, Rosario MC, Brown TA, Carter AS, Leckman JF, Sukhodolsky D, et al. Principal components analysis of obsessive-compulsive disorder symptoms in children and adolescents. Biol Psychiatry. 2007 Feb 1;61(3):285–91. doi: 10.1016/j.biopsych.2006.08.040. [DOI] [PubMed] [Google Scholar]
  • 28.Hasler G, Pinto A, Greenberg BD, Samuels J, Fyer AJ, Pauls D, et al. Familiality of factor analysis-derived YBOCS dimensions in OCD-affected sibling pairs from the OCD Collaborative Genetics Study. Biol Psychiatry. 2007 Mar 1;61(5):617–25. doi: 10.1016/j.biopsych.2006.05.040. [DOI] [PubMed] [Google Scholar]
  • 29.Boomsma DI. Twin registers in Europe: An overview. Twin Res. 1998;1(1):34–51. doi: 10.1375/136905298320566465. [DOI] [PubMed] [Google Scholar]
  • 30.Boomsma DI, de Geus EJ, Vink JM, Stubbe JH, Distel MA, Hottenga JJ, et al. Netherlands Twin Register: from twins to twin families. Twin Res Hum Genet. 2006 Dec;9(6):849–57. doi: 10.1375/183242706779462426. [DOI] [PubMed] [Google Scholar]
  • 31.Boomsma DI, Vink JM, van Beijsterveldt TC, de Geus EJ, Beem AL, Mulder EJ, et al. Netherlands Twin Register: a focus on longitudinal research. Twin Res. 2002 Oct;5(5):401–6. doi: 10.1375/136905202320906174. [DOI] [PubMed] [Google Scholar]
  • 32.Verhulst FC, Achenbach TM, Althaus M, Akkerhuis GW. A comparison of syndromes derived from the child behavior checklist for American and Dutch girls aged 6–11 and 12–16. J Child Psychol Psychiatry. 1988 Nov;29(6):879–95. doi: 10.1111/j.1469-7610.1988.tb00760.x. [DOI] [PubMed] [Google Scholar]
  • 33.Heinen T. Latent class and discrete latent trait models: Similarities and differences. Thousand Oaks, CA: Sage; 1996. [Google Scholar]
  • 34.Goodman L. Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika. 1974;61:215–31. [Google Scholar]
  • 35.Dempster A, Laird N, Rubin D. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B. 1977;39:1–38. [Google Scholar]
  • 36.Vermunt JK, Magidson J. Latent GOLD User’s Guide. Belmont, MA: Statistical Innovations Inc; 2000. [Google Scholar]
  • 37.SPSS Inc. SPSS for Windows, Rel. 15.0.1. Chicago: SPSS Inc; 2006. [Google Scholar]
  • 38.Lubke G, Neale M. Using twin data in mixture analyses. 37th Annual Meeting of the Behavior Genetics Association; Amsterdam, The Netherlands. 2007. [Google Scholar]
  • 39.Zheng M, Baker LA. Investigating ADHD subtypes: An Application of Factor Mixture Modeling to Twin Data. 37th Annual Meeting of the Behavior Genetics Association; Amsterdam, The Netherlands. 2007. [Google Scholar]
  • 40.Todd RD, Lobos EA, Sun LW, Neuman RJ. Mutational analysis of the nicotinic acetylcholine receptor alpha 4 subunit gene in attention deficit/hyperactivity disorder: evidence for association of an intronic polymorphism with attention problems. Mol Psychiatry. 2003 Jan;8(1):103–8. doi: 10.1038/sj.mp.4001257. [DOI] [PubMed] [Google Scholar]
  • 41.Todd RD, Lobos EA. Mutation screening of the dopamine D2 receptor gene in attention-deficit hyperactivity disorder subtypes: preliminary report of a research strategy. Am J Med Genet. 2002 Jan 8;114(1):34–41. doi: 10.1002/ajmg.1619. [DOI] [PubMed] [Google Scholar]
  • 42.Bartels M, Hudziak JJ. Genetically informative designs in the study of resilience in developmental psychopathology. Child Adolesc Psychiatr Clin N Am. 2007 Apr;16(2):323–39. viii. doi: 10.1016/j.chc.2006.12.008. [DOI] [PubMed] [Google Scholar]
  • 43.Polderman TJC, Derks EM, Hudziak JJ, Verhulst FC, Posthuma D, Boomsma DI. Across the continuum of attention skills: a twin study of the SWAN ADHD rating scale. J Child Psychol Psychiatry. 2007;48:1080–7. doi: 10.1111/j.1469-7610.2007.01783.x. [DOI] [PubMed] [Google Scholar]

RESOURCES