The Philadelphia Neurodevelopmental Cohort: constructing a deep phenotyping collaborative

Abstract

Background

An integrative multidisciplinary approach is required to elucidate the multiple factors that shape neurodevelopmental trajectories of mental disorders. The Philadelphia Neurodevelopmental Cohort (PNC), funded by the National Institute of Mental Health Grand Opportunity (GO) mechanism of the American Recovery and Reinvestment Act, was designed to characterize clinical and neurobehavioral phenotypes of genotyped youths. Data generated, which are recently available through the NIMH Database of Genotypes and Phenotypes (dbGaP), have garnered considerable interest. We provide an overview of PNC recruitment and clinical assessment methods to allow informed use and interpretation of the PNC resource by the scientific community. We also evaluate the structure of the assessment tools and their criterion validity.

Methods

Participants were recruited from a large pool of youths (n=13,958) previously identified and genotyped at The Children's Hospital of Philadelphia. A comprehensive computerized tool for structured evaluation of psychopathology domains (GOASSESS) was constructed. We administered GOASSESS to all participants and used factor analysis to evaluate its structure.

Results

A total of 9,498 youths (ages 8-21; mean age=14.2; European-American=55.8%; African-American=32.9%; Other=11.4%) were enrolled. Factor analysis revealed a strong general psychopathology factor, and specific ‘anxious-misery’, ‘fear’ and ‘behavior’ factors. The ‘behavior’ factor had a small negative correlation (−0.21) with overall accuracy of neurocognitive performance, particularly in tests of executive and complex reasoning. Being female had a high association with the ‘anxious-misery’ and low association with the ‘behavior’ factors. The psychosis spectrum was also best characterized by a general factor and three specific factors: ideas about ‘special abilities/persecution,’ ‘unusual thoughts/perceptions,’ and ‘negative/disorganized’ symptoms.

Conclusions

The PNC assessment mechanism yielded psychopathology data with strong factorial validity in a large diverse community cohort of genotyped youths. Factor scores should be useful for dimensional integration with other modalities (neuroimaging, genomics). Thus, PNC public domain resources can advance understanding of complex inter-relationships among genes, cognition, brain and behavior involved in neurodevelopment of common mental disorders.

Introduction

Mental illnesses are common disorders that emerge during childhood and adolescence and often persist into adulthood, with debilitating consequences. Both biologic and environmental risk factors underlie the complex phenotypes manifested in mental illnesses (Rapoport, Giedd, & Gogtay, 2012). Quantitative phenotypic brain-behavior measures enable rigorous research that can bridge molecular biology with disease phenomenology (Insel, 2014). Linking disease phenotypes to intermediate variables that modulate disease manifestations will help elucidate how these factors contribute to shape the neurodevelopment of brain systems that underlie complex behavior. To achieve this linking, large phenotypically characterized samples of youth are required for integration with genomic methods that can potentiate targeted and effective interventions. The Philadelphia Neurodevelopmental Cohort (PNC) is the first such sample in the United States (U.S.).

The PNC is a collaboration between the Children's Hospital of Philadelphia (CHOP) and the University of Pennsylvania that capitalized on an unprecedented opportunity: an already recruited and genotyped large sample of youths who had assented or consented to be contacted for further research. The primary aim of the project was to assess behavioral dimensions reflecting vulnerability to major mental illnesses by characterizing clinical and neurobehavioral phenotypes in a cohort of approximately 10,000 genotyped youths. The phenotypic dimensions included clinical assessment and neurobehavioral measures of cognitive and emotion processing related to neural systems vulnerable to neurodevelopmental aberrations (Gur, Calkins, Satterthwaite et al., 2014; Gur, Richard, Calkins et al., 2012). The clinical assessment comprehensively measured psychopathology domains including anxiety, mood, eating, behavior disorders and a broad “spectrum” of psychosis-relevant experiences. The psychosis spectrum ranges from subtle and subclinical (positive, negative or disorganized) symptoms, which would not qualify for diagnosable disorders, to clinically salient positive symptoms (threshold hallucinations and/or delusions) that fulfill criteria for serious psychotic disorders (Binbay, Drukker, Elbi et al., 2012). Behavior disorders were the most frequent mental disorder in this cohort (18.2%), followed by attention deficit hyperactivity disorder (17.6%), mood disorders (12.4%), and anxiety disorders (11.7%) (Merikangas, Calkins, He et al., 2015). Threshold positive psychosis symptoms were reported by 3.7%, an additional 12.3% reported significant subpsychotic positive symptoms, and a minority of youths (2.3%) endorsed subclinical negative/disorganized symptoms in the absence of positive symptoms (Calkins, Moore, Merikangas et al., 2014).

A subsample underwent neuroimaging to establish neural substrates of behavioral phenotypic trajectories (Satterthwaite, Elliott, Ruparel et al., 2014). The PNC provides a public domain resource for investigations and characterization of gene networks underlying neuronal vulnerability leading to mental disorders. Data generated are available through the National Institute of Mental Health's Database of Genotypes and Phenotypes (dbGaP) (dbGaP, 2014), and there were 70 approved data requests as of Feb 2015, suggesting great scientific interest in this resource.

Here we present an overview of PNC recruitment and clinical assessment methods. We detail the methods used to assess psychopathology to enable informed use of these data by the scientific community, and present the results of factor analysis that was employed to organize the common processes underlying psychopathology (Beesdo-Baum, Hofler, Gloster et al., 2009). Prior factor analytic studies in adults suggest that three factors underlie comorbidity among mental disorders, with an externalizing factor and two correlated factors, ‘anxious-misery’ and ‘fear’, forming a second-order ‘internalizing’ factor (Krueger, 1999). The three factor structure, but without the higher-order internalizing factor, has been replicated in a large prospective study of children age 14-24 (Beesdo-Baum, Hofler, Gloster, et al., 2009). However, this structure may not replicate when the number of disorders increases beyond the core disorder domains included in most investigations (Wittchen, Beesdo-Baum, Gloster et al., 2009). In addition, recent evidence suggests that psychiatric disorders are best explained by ‘internalizing’, ‘externalizing,’ ‘thought disorder’ liabilities, and a single higher order ‘general psychopathology’ dimension (Caspi et al. 2014). Finally, many factor analytic studies omitted psychotic symptoms (Krueger, 1999), evaluated psychosis in relation to few other disorder domains, or restricted analyses to psychiatric samples (Kotov, Chang, Fochtmann et al., 2011; Kotov, Ruggero, Krueger et al., 2011). We recently reported that psychosis spectrum features in the PNC are common and associated with comorbid psychopathology, distress, and neurocognitive and functional impairment (Calkins, Moore, Merikangas et al. 2014; Gur, Calkins, Satterthwaite et al. 2014). Despite growing evidence for the clinical salience of early psychotic-like experiences, to our knowledge, no prior factor-analytic investigation in youth has included measures of the broader psychosis spectrum, nor has the structure of the psychosis spectrum been examined in a large systematic sample of U.S. youth. We therefore investigated whether the previously reported three-factor structure of psychopathology fit the PNC data including the psychosis spectrum measures, and examined the structure of these psychosis spectrum tools.

Methods

Recruitment

Between 2006-2012, 50,293 youths were recruited by the Center for Applied Genomics at CHOP through a pediatric health-care network of over 30 clinical community sites in the tri-state area of Pennsylvania, New Jersey and Delaware. When undergoing blood work, patients were approached for participation in the recruitment pool. The percentage of patients undergoing blood work across recruitment sites varied from 11-53%, with a mean of 36%. Participants provided a blood sample for genomic studies and access to Electronic Medical Records (EMRs).

The EMR of each participant was screened for preliminary eligibility for PNC participation. Potential participants were included if they were between the ages of 8-21, had provided written informed consent/assent to be re-contacted for future studies, were proficient in English, and did not appear to have significant developmental delays or physical conditions that would interfere with their ability to complete study procedures. This screening yielded a pool of 19,161 eligible individuals, released to the recruitment team in weekly waves primarily over the two-year period from November 2009-December 2011 (see Figure 1). Recruitment for genomic samples was supervised by CHOP faculty and staff. Recruitment for phenotyping was conducted by a centralized team under the direction of the first author, with the assistance of CHOP staff. Potential probands (ages 18-21) or proband's caregivers/legal guardians (ages 8-17) were sent a letter introducing the study, and then contacted by phone using standardized scripts to explain the study, gauge interest, verify study eligibility, and schedule appointments. Recruitment processes and strategies are detailed in Appendix S1, available online.

Figure 1.

Recruitment flow of the Philadelphia Neurodevelopmental Cohort. Among ineligible youths (n=31,132), approximately 31% had serious medical conditions precluding participation in the study procedures (severe developmental delay, significant hearing loss, limited mobility), 44% were outside the study age range or deceased, and 36% declined to be re-contacted for future studies.

Procedures

After complete description of the study, written informed consent was obtained for participants ages>=18, and written assent and parental permission were obtained from children ages<18 and their parents/legal guardian. The University of Pennsylvania and CHOP Institutional Review Boards approved all procedures.

Psychopathology assessment

Psychopathology was assessed using a computerized, structured interview (GOASSESS) that was administered to collaterals, who were caregivers or legal guardians (ages 8-10), probands and collaterals (ages 11-17) and probands (ages 18-21). Components of the interview by informant type are presented in Table S2. The user interface of GOASSESS mimicked the paper interview on which assessors were trained rather than a data entry screen, incorporating reminders in the margins, and providing guided, as well as flexible, navigation between pages. Ample space was provided to record participant responses verbatim, along with observations and interpretative comments. Skip logic and error-checking ensured real-time and accurate data collection. Assessors uploaded data daily from their local database to the central GOASSESS database.

GOASSESS was designed to be highly structured, with screen-level symptom and episode information in order to: (1) allow rapid training and standardization across a large number of assessors conducting most assessments in the field; (2) provide for the brief administration necessary to allow assessment of 100-165 participants per week. The instrument is abbreviated and modified from the epidemiologic version of the NIMH Genetic Epidemiology Research Branch Kiddie-SADS (K. Merikangas, Avenevoli, Costello, Koretz, & Kessler, 2009). Modifications included: (1) addition of the diagnostic screening questions from the adolescent version of the World Health Organization Composite International Diagnostic Interview (CIDI), inserted at the beginning of each psychopathology domain section (K. Merikangas, Avenevoli, Costello, et al., 2009); (2) less restrictive criteria to complete the diagnostic modules (e.g., symptom counts or duration were not required to meet DSM-IV episodes in order to complete each section), and; (3) derivation of dimensional ratings of distress and impairment associated with symptoms in each diagnostic section. Components of the assessment referenced in the current article (with additional components described in the appendix S1) were:

• (1)Demographics and medical history. This section, administered to collaterals and adult probands, inquired about school and work history, developmental and medical history, and current medications. To assign a medical rating to complement up-to-date findings in the participant's EMR, we included GOASSESS medical history data to ensure that we had the most recent update for children who visited CHOP for tertiary care (further detailed in the appendix S1).
• (2)Psychopathology screen. The screen assessed psychiatric and psychological treatment history, and lifetime occurrence of major domains of psychopathology including mood (Major Depressive Episode, Manic Episode), anxiety (Generalized Anxiety Disorder, Separation Anxiety Disorder, Specific Phobia, Social Phobia, Panic Disorder, Agoraphobia, Obsessive-Compulsive Disorder, Post-traumatic Stress Disorder), Attention Deficit/Hyperactivity Disorder (ADHD), behavioral (Oppositional Defiant Disorder, Conduct Disorders), eating disorders (Anorexia, Bulimia), and suicidal thinking and behavior. Each section included a screen for relevant symptoms and additional DSM-IV criteria, including symptom frequency, duration of symptoms and episodes, and onset and offset of symptoms and episodes. For all symptoms endorsed that met threshold to continue within the section, the participant was asked to rate associated distress and impairment on separate 11-point scales ranging from 0 (no bother/problems) to 10 (extremely serious bother/problems). The Specific Phobia section is provided as an illustrative example in Figure S1. All sections are available by request to the authors, and all individual GOASSESS items are available through dbGaP with approved requests.

Three screening tools to assess the psychosis spectrum were embedded within the psychopathology screen as described recently (Calkins et al. 2014): (a) Positive sub-psychotic symptoms in the past year were assessed using the 12-item, assessor administered, PRIME Screen-Revised (PS-R (Kobayashi, Nemoto, Koshikawa et al., 2008; Miller, Cicchetti, Markovich, McGlashan, & Woods, 2004)); (b) Positive threshold psychotic symptoms (lifetime hallucinations and/or delusions) were assessed with the K-SADS psychosis screen, modified as for the other psychopathology domains described above; (c) Negative/disorganized symptoms were assessed using six assessor rated sub-scales of the Scale of Prodromal Symptoms (SOPS) from the Structured Interview for Prodromal Syndromes (SIPS) (McGlashan, Miller, Woods et al., 2003): Avolition; Expression of Emotion; Experience of Emotions and Self; Occupational Functioning; Trouble with Focus and Attention; Disorganized Communication. The SOPS Focus/Attention scale was rated based on responses and interviewer observations following administration of the ADHD items; Expression of Emotion based on interviewer observations; and the remaining four scales based on selected SIPS questions.

Psychopathology summary variables

Computerized algorithms assigned rankings for DSM-IV indices of each psychopathology domain based on endorsement of contributing items by either the proband or collateral (behavioral domains), or by the proband (all other domains). For the current analyses, psychopathology domains were considered significant if sufficient symptoms were endorsed with frequency and duration approximating DSM-IV disorder or episode criteria, accompanied by significant distress or impairment rated>=5. Comparison of the diagnostic algorithms from this interview with the full criteria using data from the National Comorbidity Survey-Adolescent (K. Merikangas, Avenevoli, Costello, et al., 2009) yielded fair (i.e., eating disorders) to excellent (i.e., ADHD) area under the receiver operator characteristic curve values for the major classes of disorders (data available upon request).

Psychosis Spectrum criteria were:

• (1)Positive-Sub-psychosis: Because age effects were observed on PRIME Screen-Revised total scores (Calkins, Moore, Merikangas, et al. 2014), an Age Deviant index was derived using proband total score, normalized within age year, identifying children with extreme total scores (z>=2) compared to age mates. Additionally, because psychosis risk may not be linearly related to total scores such that endorsement of even one symptom at a severe level (Definitely Agree=6) may be indicative of psychosis risk, and to allow comparison with prior studies with the PRIME-Screen, an Extreme Agreement index was also calculated based on traditional criteria [>=one item rated 6 or >=three items rated 5 (Somewhat Agree) (Miller, Cicchetti, Markovich, et al., 2004)].
• (2)Positive-Psychosis: Possible or definite hallucinations or delusions based on K-SADS screen, with duration>=1 day, occurring outside the context of substance, illness and medicines, and accompanied by significant impairment or distress (rating >=5).
• (3)Negative/Disorganized: An Age Deviant index of negative/disorganized symptoms was generated using Scale of Prodromal Symptoms z-scores. Specifically, SOPS total scores were normalized within age year; z>=2 cutoff reflected extreme ratings of negative or disorganized symptoms for age cohort.

Individuals evidencing any one of these three criteria were classified as having significant “psychosis spectrum” symptoms.

Assessor training and quality assurance

Bachelor's and Master's level assessors underwent a common 25-hour training protocol developed and implemented by the first author that included didactic sessions, assigned readings, and supervised pair-wise practice. They were certified for independent assessments through a standardized procedure requiring observation by a certified clinical observer who rated the proficiency of the assessor on a 60-item checklist of interview procedures (n certified assessors=55). Additionally, responses coded in GOASSESS by the assessor were required to correspond to responses coded by a certified clinical observer. Assessors who did not achieve these standards were required to undergo repeat observation until passing criteria were met. Assessor drift was monitored and corrected through periodic review of audio-recordings of real interviews, and re-training and re-certification conducted at data collection mid-point. Assessors were assigned a maximum of 10 interviews a week, with the goal of completing 5-7 interviews per week. To assure the quality of interview data, each assessment underwent a computerized error-checking algorithm that identified areas requiring assessor's attention, and a standardized post-administration review process by certified clinical reviewers. Results were reported to assessors and supervisors. A computerized chart review module provided management tools for the comprehensive review process for supervisors, reviewers, and assessors, as well as an automated check to ensure that all steps were completed successfully. Data were checked and corrected prior to final inclusion in the dataset.

Other PNC procedures

All enrolled probands were also administered a Computerized Neurocognitive Battery (CNB), described in detail elsewhere (Gur, Calkins, Satterthwaite, et al., 2014; Gur, Richard, Calkins, et al., 2012). Several additional computerized measures were administered at the time of the CNB. Substance use history was assessed by an abbreviated version of a widely used self-report measure (Han, McGue, & Iacono, 1999) computerized locally. Current pubertal status was measured by an abbreviated, computerized and privately self-administered (Satterthwaite, Vandekar, Wolf et al., 2014) version of a self-report measure of pubertal status (Morris & Udry, 1980). The Wide Range Achievement Test (WRAT-4) Reading subscale (Wilkinson & Robertson, 2006) provided an estimate of IQ. At the conclusion of each appointment, all participants were given a flyer with information about the neuroimaging study. The neuroimaging team then proceeded with recruitment of eligible probands as described elsewhere (Satterthwaite, Elliott, Ruparel, et al., 2014); a subset of 1,445 participants completed the imaging procedures.

Statistical analyses

First, we summarized the recruitment outcome and demographic characteristics of the final cohort. Second, to investigate the factor structure of the GOASSESS, we used dichotomized GOASSESS summary scores (significant v. not significant) from all assessed psychopathology domains to run four exploratory factor analyses (EFA's) ranging from 1 (unidimensional) to 4 factors. We then ran a confirmatory factor analysis based on the literature suggesting 3-factors (‘anxious-misery’, ‘fear’, ‘externalizing’). Third, to investigate dimensions of the psychosis spectrum in young people, we conducted factor analysis of psychosis spectrum screening tools. Bifactor modeling (Holzinger & Swineford, 1937) is a method to model data that have both a strong underlying factor (indicated by all variables) and several individual factors (each indicated by only some variables). The bifactor model is unique insofar as each variable loads on two factors simultaneously, such that the two factors “compete” for the explainable variance in the variable (Reise, Moore, & Haviland, 2010). We aimed to extract three factors based on previous literature (Beesdo-Baum, Hofler, Gloster, et al., 2009; Krueger, 1999). All analyses were estimated using mean- and variance-adjusted weighted least squares (wlsmv) in Mplus (Version 7) (Muthén & Muthén, 2013).

Results

Recruitment data are shown in Figure 1. From the recruitment pool, 13,598 participants were invited (71.0% of EMR eligible), 9,498 were enrolled (64.3% of invited), and 9,421 completed the assessment. Demographic data of the recruitment pool are provided in Table S1. The proportion of males to females was comparable across enrolled, not invited, declined and excluded youths. Enrolled participants were on average slightly younger than those not enrolled, however the mean difference in age among the groups was less than one year, with small corresponding effect sizes ranging from 0.12 to 0.21. A disproportionate number of enrolled and declined individuals were European-American, whereas those not invited and excluded were disproportionately other racial groups. Among those not invited, a comparatively higher number of non-European American's were unreachable. Figure 2a presents the number of assessments conducted each year, showing that the majority of assessments were completed over a two-year period. The distribution of assessments by date and participant age are shown in Figure 2b; following intentional recruitment of older probands during the start-up period, recruitment was distributed across the age range for the remainder of the recruitment period. The majority of evaluations (n=6,228; 65.6%) occurred in participant's homes, with the remainder conducted in CHOP or Penn offices (n=3,110; 32.7%) or other locations (e.g., libraries; n=160; 1.7%).

Figure 2.

a. Number of clinical assessments by date of assessment.

b. Date of clinical assessment by proband age.

Demographics of the Philadelphia Neurodevelopmental Cohort

Demographic characteristics of the final PNC sample are presented in Table 1. The mean age of the cohort is 14.2 years (s.d.=3.7), and females represent slightly more than half of the total sample (51.7%), with the proportion of females increasing significantly with age. Notably, the sample includes a large number of African-American (32.9%) and other non-European American (11.4%) youths, and the age groups are balanced by race. Youths in the 11-17 age group constitute more than half of the sample (54.9%; n=5,214). Although the age groups differed in both maternal and paternal education, inspection of the means shows that all the parental groups have a mean of 14 years of education, and that the differences between the groups are within a few decimal points (likely significant due to the large sample size).

Table 1.

Philadelphia Neurodevelopmental Cohort demographic characteristics

		Proband Age Group
	Total Sample	8-10	11-17	18-21
n	9498	2392	5214	1892
Age
Mean (years)	14.2	9.4	14.5	19.3
s.d.	3.7	0.8	2.0	1.0
Male:Female	4592:4906	1321:1071	2500:2714	771:1121
% Female	51.7	44.8	52.1	59.2
Race
European-American %	55.8	54.9	56.8	54.0
African-American %	32.9	32.3	32.0	36.1
Other %	11.4	12.7	11.2	10.0
Maternal Education
Mean (years)	14.5	14.7	14.5	14.2
s.d.	2.4	2.5	2.4	2.4
Paternal Education
Mean (years)	14.2	14.3	14.2	14.1
s.d.	2.7	2.8	2.7	2.8

Note: Race Other = Asian, Native American, Hawaiian Pacific Islander, and Multi-Racial.

s.d.=standard deviation. The proportion of females in each proband age group increases with age (Overall Pearson chi-square=89.4, df=2, p<0.001; pairwise chi-squares, all p's <0.001). The proband age groups are balanced by race (Overall Pearson Chi-Square=5.6, df=2, n.s.). Overall ANOVA's testing the significance of parental education were significant (maternal F=4.6, p<0.01; paternal F=22.6, p<0.001). However, inspection of the means shows that all the parental groups have a mean of 14 years of education, and that the differences between the groups are within a few decimal places.

Factor analysis of the GOASSESS

Exploratory factor analysis

Table 2a shows the unidimensional, 2-, 3-, and 4-factor exploratory factor analysis solutions for the GOASSESS. To evaluate the fit of the models, we applied recommended cut-off values (Hu & Bentler, 1999) as follows: comparative fit index (CFI)=0.95; root mean-square error of approximation (RMSEA)=0.06; and standardized root mean residual (SRMR)=0.08. Missing data were dealt with by pairwise deletion (final sample=9,361). The fit of the unidimensional model was moderate to poor. Though its RMSEA was good (0.040 ± 0.002), the unidimensional model's CFI and SRMR were not quite acceptable (0.89 and 0.098, respectively) (Hu & Bentler, 1999). This borderline fit suggested that a multi-factorial solution was necessary to adequately describe the latent structure of the GOASSESS. However, large loadings of the unidimensional model (mean=0.57) suggest a single strong latent factor underlying the structure, and the 2-factor model in Table 2a was consistent with this suggestion. When an additional factor was extracted, the behavioral disorders disaggregated to form their own factor, while the mood and anxiety disorders remained together. Oppositional Defiant Disorder exerted an extreme influence on determining the ‘behavior’ factor. The fit of the 2-factor model was within the acceptable range (CFI=0.98; RMSEA=0.021 ± 0.002; SRMR=0.051), suggesting additional factors might be unnecessary to model GOASSESS. However, if ‘anxious-misery’ and ‘fear’ are distinct phenomena, extraction of an additional factor would be required.

Table 2.

a. Unidimensional, 2-, 3-, and 4-factor exploratory solutions of GOASSESS (with oblique rotation)
		2-Factor		3-Factor			4-Factor
Scale	Uni	F1	F2	F1	F2	F3	F1	F2	F3	F4
Post-traumatic Stress Disorder	.55	.53		.51			.34	.20		.23
Mania	.57	.49		.40			.30	.25
Major Depressive Episode	.67	.73		.78			.68
Eating Disorders	.48	.59		.67			.54		−.21
Generalized Anxiety Disorder	.59	.66		.61			.82
Separation Anxiety Disorder	.40	.49		.37	.20		.43
Specific Phobia	.40	.39			.57			.54	.21
Social Phobia	.51	.52		.29	.38			.51
Panic Disorder	.59	.75		.69			.55	.23
Agoraphobia	.63	.60		.33	.45			.78
Obsessive-Compulsive Disorder	.64	.67		.63			.58
Attention Deficit Hyperactivity Disorder	.47		.57		.23	.57			.75
Oppositional Defiant Disorder	.70		.91			.87			.23	.67
Conduct Disorder	.70		.78			.82				.91
Psychosis Spectrum	.63	.38	.36	.31		.37	.24	.21		.32

	GOASSESS Factor correlations (phi matrices)
		F1	F2	F1	F2	F3	F1	F2	F3	F4
	F1	-		-			-
	F2	.46	-	.42	-		.57	-
	F3			.42	.30	-	.12	.11	-
	F4						.33	.41	.31	-

b. Unidimensional Solution and Exploratory 2-, 3-, and 4-Factor Oblique Solutions of the Psychosis Spectrum Screening Tools.
Summary Measure	Tool	Item		2-Factor		3-Factor			4-Factor
			Uni	F1	F2	F1	F2	F3	F1	F2	F3	F4
Auditory Perceptions	PS-R	I have had the experience of hearing faint or clear sounds of people mumbling/talking when no one is nearby.	.77	.68		.78			.66
Reality Confusion	PS-R	I think that I may get confused at times whether something I experience or perceive may be real or may just be part of my imagination or dreams.	.79	.75		.74			.37	.35	.24
Mind Tricks	PS-R	I think that I might feel that mind is “playing tricks” on me.	.79	.73		.73			.37	.35	.22
Hallucination Screen	K-SADS	--	.55	.39	.28	.72	−.26		.82
Odd/Unusual Thoughts	PS-R	I think that I have felt that there are odd or unusual things going on that I can't explain.	.73	.66		.68			.33	.35
Audible Thoughts	PS-R	I think that I may hear own thoughts being said out loud.	.75	.76		.59	.25		.28	.27	.37
Delusion Screen	K-SADS	--	.45	.31	.24	.54			.45
Superstitions	PS-R	I have had the experience of doing something differently because of my superstitions.	.72	.74		.50	.31			.32	.41
Thought Control	PS-R	I may have felt there is something interrupting or controlling my thoughts, feelings, or actions.	.77	.75		.48	.34			.37	.42
Suspicious	PS-R	I wonder if people may be planning to hurt me or even may be about to hurt me.	.72	.65		.47	.25			.31	.32	.20
Predict Future	PS-R	I think that I might be able to predict the future.	.67	.76			.75				.83
Grandiosity	PS-R	I have thought that it might be possible that other people can read my mind or that I can read others' minds.	.72	.82			.70				.86
Mind Reading	PS-R		.71	.78		.29	.52				.62
Occupational Functioning	SOPS	--	.42		.69			.73				.69
Expression of Emotion	SOPS	--	.35		.60			.65				.67
Experience of Emotions/Self	SOPS	--	.50		.63			.59		.24		.69
Trouble with Focus/Attention	SOPS	--	.41		.55			.53	.26	−.24		.48
Disorganized Communication	SOPS	--	.49		.48			.46	.22			.43

	Psychosis Spectrum Factor intercorrelations (Phi matrices)
				F1	F2	F1	F2	F3	F1	F2	F3	F4
			F1	1		1			1
			F2	.48	1	.64	1		.41	1
			F3			.51	.26	1	.60	.50	1
			F4						.47	.25	.37	1

Note. N=9,361. Loadings < .20 removed; rotation = oblimin; F=Factor; Uni = unidimensional; maximin loading (per item, per solution) bolded; solutions estimated using the mean- and variance-adjusted weighted least squares (wlsmv) estimator in Mplus (Version 7)(Muthén & Muthén, 2013). K-SADS= Kiddie Schedule for Affective Disorders and Schizophrenia. SOPS=Scale of Prodromal Syndromes. PS-R=PRIME Screen Revised. All PS-R items are self-rated from 0=Definitely Disagree to 6=Definitely Agree. For part a, unidimensional (1-factor) the comparative fit index (CFI) = 0.89, standardized root mean residual (SRMR) = 0.098, root mean square error (RMSEA) = 0.040; 2-factor CFI = 0.98, SRMR = 0.051, RSMEA = 0.021; 3-factor CFI = 0.99, SRMR = 0.042, RSMEA = 0.016; 4-factor fit not assessed due to likely over-extraction; for part b, unidimensional (1-factor) CFI = 0.94, SRMR = 0.074, RMSEA = 0.064; 2-factor CFI = 0.98, SRMR = 0.039, RSMEA = 0.039; 3-factor CFI = 0.99, SRMR = 0.027, RSMEA = 0.029; 4-factor CFI = 1.00, SRMR = 0.019, RMSEA = 0.02. All Chi-square difference tests comparing the 1-, 2-, 3- and 4- factor models were significant at the p<0.005 level (see Supplemental Table 5).

The 3-factor model (Table 2a) largely confirms that three factors are unnecessary to adequately explain the correlations among disorders. The fit of the model was excellent (CFI=0.99; RMSEA=0.016 ± 0.003; SRMR=0.042), but one of the goals of using fit indices in exploratory factor analysis is to determine the minimum number of factors necessary to explain covariance. By that standard, three factors were too many. However, examination of the factor loadings weakly supports the existence of a third, fear-related factor: the three phobias (Specific, Social, and Agoraphobia) disaggregated to form their own factor. Social Phobia and Agoraphobia both had salient cross-loadings on the ‘anxious-misery’ factor. Finally, Factor 3 clearly remained a well-defined ‘behavior’ factor. Due to the possible over-extraction of the 3-factor model (indicated by the acceptable fit of the 2-factor model), conclusions cannot be drawn from the 4-factor model.

Notably, the psychosis spectrum had numerous cross-loadings in multifactorial solutions. We therefore re-ran the analyses after decomposing the psychosis summary variable into its constituent positive-psychosis, positive-sub-psychosis and negative/disorganized variables. Negative/disorganized symptoms loaded with Behavior Disorders, whereas the positive-psychosis and positive-sub-psychosis variables both loaded exclusively with a factor composed of Mania, the Phobias, and Post-traumatic Stress Disorder (see Table S3). However, there were more cross-loadings than in the model shown in Table 2a, rendering interpretation slightly more difficult.

Finally, to further address criterion validity, we calculated factor scores using the 3-factor model shown in Table 2a, and correlated them with neurocognitive summary measures (Moore et al. 2014). The ‘behavior’ factor had a small to moderate negative correlation (−0.21) with overall accuracy of neurocognitive performance, particularly in tests of executive and complex reasoning (−0.21). Smaller negative correlations across neurocognitive domains were also observed for the ‘anxious-misery’ and ‘fear’ factors (range= −0.03 to −0.19) (see Table S4).

Confirmatory bifactor analysis

Figure 3a shows the 3-factor bifactor solution of GOASSESS, including sex as a covariate; the fit of the model was excellent (CFI=0.98; RMSEA=0.019±0.002). The general factor was very strong (mean loading=0.54), with Major Depressive Episode and Obsessive-Compulsive Disorder having the highest loadings (0.72 and 0.71, respectively), and ADHD and Specific Phobia having the lowest loading (0.27 and 0.39, respectively). After controlling for the general factor, some disorders were no longer associated with their specific factors: Mania no longer loaded on the ‘anxious-misery’ factor, Panic Disorder no longer loaded on the ‘fear’ factor, and Obsessive-Compulsive Disorder actually had a significant negative loading on the ‘fear’ factor. The ‘behavior factor’ remained well-determined (mean loading=0.55) after controlling for general ‘mental health.’ Psychosis spectrum weakly loaded on this ‘behavior’ factor. There was a significant decrease in loadings on the ‘anxious/misery’ and ‘fear’ factors in Figure 3a (compared to Table 2a), suggesting they are central to the general construct of mental health. There were strong associations with participant sex: being female had a high association with ‘anxious-misery’ and low association with ‘behavior’.

Figure 3.

a. Confirmatory bifactor analysis of the GOASSESS with sex as a covariate (n=9,361). PTSD=Post-traumatic Stress; MDE=Major Depressive Episode; Eating= Anorexia/Bulimia; GAD=Generalized Anxiety; Separation=Separation Anxiety; Panic=Panic; OCD=Obsessive Compulsive; ADHD=Attention Deficit/Hyperactivity; ODD=Oppositional Defiant.

b. Exploratory Bifactor (Schmid-Leiman) Rotation of the Psychosis Spectrum Screening Tools in Adolescents and Young Adults (n=6,963). See Table 2 for items/tools associated with each variable.

Factor analysis of psychosis spectrum screening tools

Table 2b presents unidimensional, 2-, 3-, and 4-factor solutions of the psychosis spectrum screening tools. The SOPS Avolition scale was dropped from the factor analysis because it resulted in a Heywood case (communality > 1.0; Heywood, 1931). There were two clear factors for symptom type (positive versus negative/disorganized symptoms) in all solutions except the unidimensional, and 3-factor solution revealed a clear second positive symptom factor determined by “ideas about unusual abilities”. When the 3-factor solution was expanded to a fourth factor, interpretation of the item groupings became less clear. Factor 2 was made up entirely of weak loadings, and only one of those nine loadings (odd/unusual thoughts) was the dominant loading for that item, indicating that the 4-factor solution was likely over-extracted (i.e., Factor 2 is a factor of cross-loadings). Finally, the correlations among the factors (phi matrices) were moderate to high, reflecting a general factor explaining correlations among all items.

The results suggest that: 1) a 3-factor model is likely the most appropriate for the psychosis spectrum measures, and; 2) a general factor should be included to account for overall psychosis spectrum. We therefore estimated a bifactor model, shown in Figure 3b as an exploratory bifactor (Schmid-Leiman) orthogonalization (Schmid & Leiman, 1957) where each item loads on a specific factor and a general factor (with all factors orthogonal). This shows the direct effects of the highest-order construct (the general factor – ‘psychosis spectrum’) on each item, allowing more accurate judgments about the quality of each item. One example is “auditory perceptions”, which in the unidimensional factor model (Table 2b) had a high loading of 0.77, but “reality confusion” and “mind tricks” had higher loadings. In the bifactor model (Figure 3b), however, “auditory perceptions” had the highest loading on the general factor. This indicated that “reality confusion” and “mind tricks” had inflated loadings in the unidimensional model (Table 2), and revealed auditory perceptions as the item with the highest discriminability on the scale. After accounting for the general psychosis spectrum factor, there were two positive symptom factors – ideas about ‘special abilities/persecution’; ‘unusual thoughts/perceptions’ - and a ‘negative/disorganized factor’ (Figure 3b).

Discussion

This overview of the recruitment and clinical assessment of the Philadelphia Neurodevelopmental Cohort (PNC) elucidates a strategy for accruing and psychiatrically assessing a large U.S. community sample of young people. The PNC was ascertained over a two-year period in response to an unprecedented opportunity to characterize clinical and neurobehavioral phenotypes in previously genotyped youths from the community. There are several noteworthy features of the PNC. First, the distribution of participants across a wide age range (age 8-21, mean age=14.2) allows powerful cross-sectional comparisons of age-group related differences in neurobehavioral domains (Gur, Calkins, Satterthwaite, et al., 2014; Gur, Richard, Calkins, et al., 2012). Second, being drawn from the greater Philadelphia area, the sample is racially diverse (56% European-American; 44% African-American and Other) and includes youths from surrounding urban, suburban and rural areas. Third, youths were initially recruited from a wide range of pediatric clinics, and thus include those who are healthy and those with a wide range of medical conditions, which allows comprehensive evaluation of associations among various medical conditions and mental disorders (Merikangas, Calkins, Burstein et al. 2015). Notably, participants were not recruited from psychiatric services.

In the PNC, we implemented a structured assessment tool with the aim of screening for a comprehensive array of psychopathology domains, including psychotic and sub-psychotic symptoms. Several characteristics of the structure of GOASSESS emerged. First, there is strong general ‘mental health’ factor, on which Major Depressive Episode and OCD have the highest loadings, and ADHD and Specific Phobia having the lowest. Such a strong general factor of psychopathology is consistent with prior studies reporting high inter-factor correlations or strong loadings on a higher-order factor, or both (Beesdo-Baum, Hofler, Gloster, et al., 2009). Moreover it is consistent with Caspi et al.'s (2014) recent findings of a general psychopathology factor reflecting comorbidity among all disorders above and beyond their specific disorder strata. This suggests that each individual can be assigned a general ‘mental health’ score based on GOASSESS. Second, after controlling for the general factor, there are specific ‘anxious-misery’, ‘fear’ and ‘behavior’ factors as previously reported (Kessler, Avenevoli, McLaughlin et al., 2012). However, some disorders lose their associations with their specific factors, including Mania, Panic Disorder and Obsessive-Compulsive Disorder. Conversely, the ‘behavior’ factor remains fairly well-determined. Overall, the results are consistent with distinct dimensions of mental disorders that are sometimes overwhelmed by a general ‘mental health’ factor. A partial exception, however, might be the psychosis spectrum, indicated by its numerous cross-loadings in multifactorial solutions (Table 2a). Prior studies of adults using confirmatory factor analysis and a greater number of psychosis variables have suggested that psychosis may be a distinguishable behavioral dimension from the internalizing and externalizing dimensions (Kotov, Chang, Fochtmann, et al., 2011; Kotov, Ruggero, Krueger, et al., 2011). Our model accounting for the general ‘mental health’ factor shows a summary psychosis spectrum measure in youths loading only weakly on the behavior factor, and strongly on the general mental health factor. The associations of the specific disorders with participant sex are consistent with existing literature showing higher rates of externalizing (‘behavior’) disorders in males, and internalizing (‘anxious-misery’ and ‘fear’) disorders in females (K. R. Merikangas, He, Burstein et al., 2010). Finally, psychopathology factor scores are negatively correlated with dimensions of neurocognitive functioning.

The large community surveys in the U.S. are built on a tradition of ascertaining DSM diagnostic criteria for mental disorders. Therefore, the diagnostic modules closely adhere to current diagnostic nomenclature, and do not include assessment of sub-threshold psychosis spectrum symptoms (K. Merikangas, Avenevoli, Costello, et al., 2009). The assessment of these symptoms in the PNC therefore provides a unique dataset. In the current analyses, a bifactor model revealed a well-defined general psychosis spectrum factor and three specific factors: ideas about ‘special abilities/persecution’; ‘unusual thoughts/perceptions’; and ‘negative/disorganized’ symptoms. Notably, the loading of “auditory perceptions” with hallucinations and delusions suggests that it taps a similar dimension of the psychosis spectrum as threshold hallucinations. Moreover, its highest loading on the general factor is consistent with reports that unusual auditory perceptions are generally predictive of psychotic symptoms (Kelleher, Connor, Clarke et al., 2012; Laurens, Hobbs, Sunderland, Green, & Mould, 2012). Negative symptoms, understudied in young people (Dominguez, Saka, Lieb, Wittchen, & van Os, 2010), formed a discrete factor with disorganized symptoms. While the differentiation of positive from negative/disorganization symptoms is commensurate with prior studies of psychotic features in schizophrenia (Stefanis, Hanssen, Smirnis et al., 2002) and psychosis proneness (Therman, Heinimaa, Miettunen et al., 2011), the lack of distinct negative and disorganized factors here may reflect the small number of contributing items. The results nevertheless suggest that there are separable yet related dimensions of the psychosis spectrum in this young cohort. As suggested by the literature on schizotypy in young people, these dimensions may be differentially associated with varying developmental trajectories and outcomes (Kwapil, Gross, Silvia, & Barrantes-Vidal, 2013).

Overall, it appears that the GOASSESS and psychosis spectrum screening tools have good structural validity, as established by exploratory and confirmatory factor analyses. Moreover, the correlations of the bifactor group factors with sex establish some nomological validity by suggesting GOASSESS is associated in the theoretically predicted way with measures of different but related constructs.

Limitations

Though we made efforts through our recruitment approach to enhance the representativeness of the cohort, initial ascertainment through pediatric clinics may affect the generalizability of the findings to the youths in the U.S. or elsewhere. Moreover, our results suggest that the enrolled group is representative of the original recruitment pool in age and sex, but not race. There are several possible reasons for this disparity including factors associated with living in an urban environment (e.g., decreased stability of housing/telephone numbers), which are not independent of race in the PNC. A further analysis of environmental contributors is beyond the scope of this paper, but is currently underway. We are also currently examining geocoded data to check the extent to which participants are representative of the greater Philadelphia area, as well as the extent to which sample demographics differ from those of the general U.S. population.

There are some limitations of the psychopathology assessment that should be considered when interpreting findings, or using data, from the PNC. First, the highly structured nature of the GOASSESS was necessary given the time frame and scope of the study. Although factor analytic results appear consistent with prior findings, the structured and abbreviated format may have reduced sensitivity to clinically significant symptoms or conversely, yielded a high level of false-positives for particular symptoms or disorders. Similarly, episode level information for some domains was gathered in the absence of information about their temporal relationships with other domains, but knowledge of these relationships is required for differential diagnosis among many disorders (e.g., mood disorders v. psychotic disorders). Thus, the results may not be comparable to studies adhering to strict DSM-IV disorder criteria.

Second, while both collateral and proband interviews were conducted for probands aged 11-17, only collateral and only proband interviews are available for the 8-10 and 18-21 year old probands, respectively. Our analyses used just one of many possible strategies to analyze data from different informants, but clearly the proband and collateral assessment data in the 11-17 year old age range should be a rich source for alternative and informative parent-child comparisons (Hudziak, Achenbach, Althoff, & Pine, 2007). Any analyses of differences among various age groups (e.g., 8-10, 11-17, 18-21) must be conducted with recognition of the corresponding differences in informants across these groups.

Finally, the cross-sectional nature of the assessment imposes inherent limitations on developmental inferences about psychopathology and its intra-individual trajectories. Cross-sectional designs make it difficult to differentiate true age-related changes from apparent age differences that are actually due to other factors, such as cohort factors. In the PNC, the recruitment over a very brief time span may limit the influence of intra-age group effects, but across age group cohort effects may exist. Additional clinical information obtained through longitudinal follow-up will allow further evaluation of clinical relevance and stability of the current findings.

There are also some limitations of our current analyses. First, because we aimed for consistency with studies that used DSM criteria for major domains of psychopathology, we required the “significant” cut-off level to include significant distress or impairment reported by the proband (or collateral). However, there are multiple ways the data could be analyzed and many possible cut-offs; classification choices will depend on the questions being asked. Similarly, we selected factor analysis, but a number of data reduction methods exist and can be applied to the GOASSESS data. The computerization of GOASSESS afforded the opportunity to capture and archive individual item responses that will permit alternative approaches to diagnostic classification and analyses. Combined with PNC neurocognitive and neuroimaging data, they also offer a rich dataset for dimensional approaches such as those embodied in the NIMH Research Domain Criteria (RDoC) (Insel, 2014). Second, in factor analysis, it is difficult to specify fit-index cutoff values that apply across a wide range of models. We opted to follow the recommendation in the most commonly cited simulation study of fit indices (Hu & Bentler, 1999), but alternative cut-offs are possible (Marsh, Hau, & Wen, 2004; Sivo, Fan, Witta, & Willse, 2006) and can be applied in the future. In addition, given evidence that the factor structure of psychopathology may vary across age groups (Wittchen, Beesdo-Baum, Gloster, et al., 2009), future analyses can focus on the developmental stability of observed comorbidities.

Conclusion

Advances in genomics are revolutionizing medicine with discoveries that help elucidate mechanisms and design novel treatments. For mental illnesses to benefit from genomics, data are needed linking behavior to brain function in large prospective samples. In a diverse cohort of nearly 9,500 genotyped youths, we have taken the first steps to characterize neurodevelopmental phenotypes – clinical, neurocognitive and, in a subsample, neuroimaging - creating a landmark dataset that we hope will propel understanding and treatment of developmental neuropsychiatric disorders and allow their genomic underpinnings to be dissected in future studies. Factor analytic results of the clinical data are consistent with the suggestion that studies of the relationships among psychopathology, genomics and biobehavioral measures (CNB, imaging), may increase efficiency by focusing on groups of comorbid disorders with shared general factors, rather than on individual disorders (Kendler, Prescott, Myers, & Neale, 2003). Collaborative efforts to analyze the rich, accessible, PNC resource will accelerate research on the complex inter-relationships among genes, cognition, brain and behavior involved in the neurodevelopment of common mental disorders (National Institute of Health, 2014).

Key points.

• The Philadelphia Neurodevelopmental Cohort (PNC) of nearly 9,500 young people (ages 8-21) was established as a public domain resource of clinical and neurobehavioral phenotypes from a community cohort of previously genotyped youths.

• This paper provided an overview of PNC recruitment and clinical assessment methods to allow informed use and interpretation of PNC data for the scientific community.

• Results of factor analysis employed to organize the common processes underlying psychopathology yielded psychopathology data with strong factorial validity.

• The PNC resource should enable advancement of understanding of the inter-relationships among genes, cognition, brain and behavior involved in the neurodevelopment of common mental disorders.