Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Feb 2.
Published in final edited form as: Psychophysiology. 2013 Aug 2;50(10):974–982. doi: 10.1111/psyp.12081

P300 amplitude reduction is associated with early-onset and late-onset pathological substance use in a prospectively studied cohort of 14 year-old adolescents

Greg Perlman 1, Abe Markin 2, William G Iacono 3
PMCID: PMC3872222  NIHMSID: NIHMS493445  PMID: 23905780

Abstract

P3 amplitude reduction (P3AR) is associated with risk for adolescent-onset pathological substance use (PSU). In this longitudinal study, data from over 1100 adolescent twins were used to examine P3AR in relation to: Early Adolescent Onset PSU (e.g., by age 14), Late Adolescent Onset PSU (e.g., between age 14 and age 18), misuse of different classes of substances (PSU-Nicotine, PSU-Alcohol, PSU-Illicit), degree of PSU comorbidity, and gender differences. P3 amplitude was recorded at age 14 from two midline electrodes during a visual oddball paradigm. PSU was defined as meeting criteria for any symptom of a substance use disorder assessed using semi-structured clinical interviews. P3AR was associated with degree of drug class comorbidity, Early Adolescent Onset PSU for all three substance classes, and Late Adolescent Onset PSU for Alcohol and Illicit PSU. Gender differences in P3AR were not statistically significant. These findings provide further evidence that P3AR indexes a non-specific diathesis for adolescent-onset PSU.


Four decades of research have provided evidence that P300 amplitude reduction (P3AR), or a reduced component of the third positive peak in the visual event-related potential (ERP) waveform generated in an oddball task, is associated with an increased liability to engage in alcohol and drug misuse (Iacono, Malone et al. 2003). P3AR has been found to correlate with use of a variety of drugs, including alcohol (Porjesz, Begleiter et al. 1980; Begleiter, Porjesz et al. 1984; Iacono, Carlson et al. 2002), cocaine (Bauer 1997), marijuana (Kempel, Lampe et al. 2003), opiods (Singh, Basu et al. 2009), and tobacco (Anokhin, Vedeniapin et al. 2000). A recent meta-analysis estimated that individuals with a positive history of a substance use disorder (SUD) differ from individuals without a positive history of a SUD by about half a standard deviation in P300 amplitude (d = 0.51; Euser, Arends et al. 2012).

Additional lines of evidence support the notion that P3AR indexes a familial liability that emerges prior to the onset of substance abuse. For example, P3AR has been found to manifest in unaffected biological offspring of individuals with substance use problems (for a recent meta-analysis, see Euser, Arends et al. 2012; d = 0.28). Moreover, in longitudinal studies of youth P3AR has been found to correlate with substance use outcomes years later, including a factor-analysis-derived composite of substance use habits, (Berman, Whipple et al. 1993), age of onset of substance use (Hill, Shen et al. 2000), and the development of a SUD (Hill, Shen et al. 2000; Iacono, Carlson et al. 2002; Carlson, Iacono et al. 2004; Habeych, Charles et al. 2005; Carlson, McLarnon et al. 2007; Hill, Steinhauer et al. 2009).

Contributions of the Current Study

The present investigation used data from over 1100 adolescents from a longitudinal community-based study. We examined several features of the association between P3AR recorded at age 14 years and pathological substance use (PSU), where PSU is operationalized as evidencing any symptom of substance abuse or dependence for a DSM-IIIR or DSM-IV SUD. Specifically, we examined the association between P3AR and the timing of PSU onset for three drug classes (e.g. PSU-Nicotine, PSU-Alcohol, and PSU Illicit), the effect of PSU-comorbidity (i.e. meeting criteria for PSU in more than one drug class), and gender differences.

Different PSU drug classes and comorbidity

A primary objective of the present investigation was to clarify whether P3AR recorded at age 14 is a marker of risk for PSU generally (Iacono, Malone et al. 2008), or is specific to risk for misuse of certain kinds of drugs. To accomplish this objective, we assigned adolescents to three mutually-inclusive PSU groups based on the substances for which they manifested criteria: PSU-Nicotine, PSU-Alcohol, and PSU-Illicit; the latter includes various illicit substances, such as cannabis and cocaine. A fourth group was comprised of individuals who did not meet criteria for PSU within the developmental time frame of our study (“Controls”). Prior studies from our laboratory examined the specificity of P3AR to single versus multiple drug classes using an older cohort of twins. Across several studies, P3AR recorded from 18 year-olds adolescents was found to be associated with general risk for substance use (non-specific) instead of with only one drug class (Iacono, Carlson et al. 2002; Yoon, Iacono et al. 2006). These findings, as well as similar findings from other laboratories examining several outcomes (Hill, Steinhauer et al. 2009), contributed to the hypothesis that P3AR indexes an underlying neurobiological risk factor for the development of PSU that is not specific to any class of drug. The current study is of the same theme but is unique in that it examines a different, younger twin cohort than the late adolescent sample used in our prior studies. Use of these younger participants made it possible to focus on the early appearance of symptoms from mid (age 14) to late (age 18) adolescence rather than on the development of manifest psychiatric disorder.

We also examined the association between P3AR and degree of PSU drug class comorbidity (i.e., meeting criteria for PSU in more than one drug class). Finding that P3AR is greater in those with PSU comorbidity would contradict the notion that P3AR is marker of risk for one particular drug. In contrast, finding that P3AR was associated with only one PSU drug class and not associated with comorbidity would provide evidence for drug specificity.

Timing of PSU onset

A second aim of the present investigation was to examine P3AR as a function of the timing of PSU onset. Within each of the three PSU drug class groups, adolescents were segregated based on whether PSU manifested before age 14 (“Early Adolescent Onset”) or between age 14 and age 18 (“Late Adolescent Onset”). This provided an opportunity to test two independent predictions reported in the literature regarding the association between P3AR and PSU. The first prediction was that P3AR distinguishes adolescents who have already developed PSU from unaffected adolescents (Euser, Arends et al. 2012). This prediction was tested by comparing Controls to each Early Adolescent Onset PSU group. The second prediction was that P3AR distinguishes adolescents who will develop PSU at a later time point from adolescents who remain unaffected. This prediction was tested by comparing Controls to each Late Adolescent Onset PSU group.

Gender and sample size in prospective P3AR studies

A third aim of this study was to examine gender differences in P3AR. Cross-sectional studies have provided evidence that P3AR indexes risk for PSU for males more effectively than among females (Euser, Arends et al. 2012). However, the relationship of P3AR with PSU has not been examined in a longitudinal analysis with adequate sample size to detect subtle gender differences. Previous prospective analyses examined either an exclusively male cohort (Berman, Whipple et al. 1993; Iacono, Carlson et al. 2002; Carlson, McLarnon et al. 2007), a mixed gender sample including a relatively small cohort of females (Hill, Steinhauer et al. 1995; Hill, Shen et al. 2000) or a mixed gender sample including approximately 200 females (Carlson, Iacono et al. 2004; Hill, Steinhauer et al. 2009). To our knowledge, the current study examines the largest cohort of female adolescents in a prospective design to date (n = 619). Furthermore, while two prospective studies have examined a large cohort of about 500 18 year-old males (Iacono, Carlson et al. 2002; Carlson, McLarnon et al. 2007), the current study examines 593 14 year-old males. This is, to our knowledge, the first prospective study of P3AR and substance use in adolescents younger than age 18 to examine large samples of both boys and girls.

Hypotheses

Based on the literature, we hypothesized that P3AR recorded at age 14 reflects a premorbid risk factor for all three drug classes of PSU rather than any single class of drug. Thus, we predicted that P3AR would be evident in those with (a.) Early Adolescent Onset PSU in each drug class and comorbidity, and (b.) Late Adolescent Onset PSU in each drug class and comorbidity. Furthermore, we expected the hypothesized effects to be stronger in males than females.

Method

Participants

Data for this project were taken from the second and third assessments of the younger cohort of the Minnesota Twin Family Study (MTFS), a community-based, longitudinal study of same-sex twins born in the state of Minnesota and their parents (Iacono and McGue 2002). Twins were originally identified by public birth record and then contacted by phone or mail. Twins and their parents were invited to participate if they lived within a one-day drive of University of Minnesota and if neither twin was physically or mentally handicapped. The assessment generally lasted an entire day and covered many domains, including psychiatric diagnosis, psychophysiology, and substance use. Characteristics of the younger cohort have been described elsewhere (McGue, Iacono et al. 2001; Blazei, Iacono et al. 2008; Perlman, Johnson et al. 2009). Briefly, participants were born between 1977 and 1985 and were between 10 and 12 years-old at the time of their initial assessment. Twins and their parents were invited to return for a follow-up assessment every three to four years, targeting the years during which they turned 14 and when they turned 17. At the second assessment, ages of the participants ranged from 13 years 6 months to 16 years 9 months (x=14years, 10 months; SD = 6 months). At the third assessment, ages of the participants ranged from 16 years 6 months to 20 years 4 months (x=18years 2 months; SD = 8 months). The current report is based on the 1212 participants (N= 619 female) who provided age-14 EEG data.

Psychophysiological Assessment

EEG data were collected at the second assessment when participants were age 14. Our data collection procedure has been described elsewhere in more detail (Iacono, Carlson et al. 2002; Carlson, McLarnon et al. 2007; Perlman, Johnson et al. 2009). Participants sat upright in a padded recliner and were instructed to watch a computer screen placed approximately from the recliner. 65 cm EEG data were collected using the Grass Systems Model 12 Neurodata Acquisition System (Grass Instruments, Quincy, MA) with 1/2 amplitude low- and high frequency filter settings at 0.01 Hz and 30 Hz, respectively. Linked earlobes were used as a reference and impedance was less than 10 kΩ. Impedance for the ground (placed on the participant’s right shin) and EEG electrodes was kept under 5 kΩ. EEG data were digitized online at a rate of 256 samples per second in 2 s epochs, with 500 ms proceeding and 1500 ms following each stimulus presentation. The PZ, P3, P4, and FZ electrodes were used for EEG data collection. This project analyzes data from PZ because this electrode has received the most attention in previous research (Iacono, Carlson et al. 2002; Carlson, McLarnon et al. 2007), and is where P3 amplitude is maximal. We have also included data from FZ because this electrode has received attention in the substance use literature (Costa, Bauer et al. 2000). We did not examine P300 latency because it has not proven to be a useful marker of risk for PSU in our past studies (Malone, Iacono et al. 2001).

P300 amplitude was elicited using a modified version of the rotated heads task (Begleiter, Porjesz et al. 1984). For 80 target trials, participants were instructed to identify the side of an oval (“head”) on which a triangle (“ear”) was located on by pressing a corresponding button. For half of the target trials, the head was facing downward, which requires the participant to identify the ear as being located on the contralateral side from which it is presented. This is sometimes referred to as the “difficult” target condition. The ovals were presented with a horizontal radius of 4.3 cm and a vertical radius of 5.3 cm. No response was required during the 160 nontarget trials (presentation of only the oval). Stimuli were presented sequentially in a pseudorandom order for 98 ms with an inter-trial interval that varied randomly between 1 s and 2 s. Participants were given practice trials at the beginning of the task.

After the session, EEG data underwent several processing steps to assure data quality. Trained research assistants flagged EEG trials with apparent artifacts or electrode malfunctions and subjects with atypical ERP waveforms. Individual trials were excluded from averaging if any point in the epoch was greater than 100 μv or less than −100 μv. The Gratton, Coles, and Donchin (1983) method was used to correct for eye blink artifacts. Consistent with the P3AR literature and studies from our lab, easy and difficult target conditions were averaged together. Average waveforms were low-pass filtered at 7.5-Hz. A Matlab computer algorithm was used to calculate P3 amplitude as the maximum within a predefined latency window. P3 amplitude was defined at the peak amplitude between 350ms and 550ms at the FZ electrode (FZ-P300) and between 350ms and 750ms at the PZ electrode (PZ-P300). These latency windows were chosen because of morphological differences in P3 amplitude and inter-subject variability in the manifest ERP waveforms at anterior versus posterior scalp locations. Outliers and data recording artifacts were identified by examining histograms of P3 amplitudes and the number of “good” (e.g., non-artifact) trials per subject. In order to assure high data quality and reduce the influence of outliers on group mean differences, a participant’s P3 amplitude was excluded from analysis if statistically deviant (e.g., more than 3.0 standard deviations from its electrode mean) or from a noisy recording (e.g., fewer than 40 target trials were non-artifact). These decisions were made blind to the participant’s PSU history. After these exclusions, PZ-P3 amplitude data was available for 1202 participants, FZ-P3 amplitude data was available for 1209 participants, and 1199 participants had data from both electrode sites. The vast majority of these participants had SUD data by their age 18 assessment: 1148 participants had PZ-P3 data and 1154 participants with FZ-P3 data. This comprises the full sample in this study. An attrition analysis compared P3 amplitude between (a.) participants who were excluded due to missing substance use data by age 18 and (b.) participants for whom we had complete SUD data by age 18 and were assigned to our study groups. The group difference was not statistically significant at either electrode by independent samples t-test (both ps > 0.50).

Preliminary analyses showed that age at the time of EEG assessment was associated with P3 amplitude. Age was inversely correlated with PZ-P3 amplitude, r = −0.10, p < 0.01; beta = −1.77, and positively correlated with FZ-P3 amplitude r = 0.07, p < 0.05; beta = 0.94. Because SUD risk may also increase with age, it was plausible that age could confound our results (e.g., older adolescents at the age 14 assessment had more opportunity to use substances and have lower PZ-P3 amplitudes). To negate the possibility of an age confound, age at the time of EEG assessment was regressed out of FZ- and PZ-P3 amplitude, and residual variables were used in all statistical analyses. The correlation between FZ-P3 amplitude and PZ-P3 amplitude was 0.34, p < 0.01.

Assessment of Pathological Substance Use

Pathological substance use (PSU) status was determined by considering SUD diagnostic information at three time points: an age 11 assessment, an age 14 assessment, and an age 18 assessment using procedures that have been described in greater detail elsewhere (Iacono, Carlson et al. 1999; McGue, Iacono et al. 2001). For the purposes of this investigation PSU was operationalized as meeting criteria for at least one symptom of DSM-IIIR or DSM-IV abuse or dependence of alcohol, nicotine, or any illicit drug disorder. Controls had no symptoms of PSU by the age of 18. Early Adolescent Onset cases had at least one symptom by age 14, and Late Adolescent Onset cases had no symptoms at age 14 but developed one by age 18. This PSU definition was chosen due to several considerations. First, past research has shown that up to 30% of adolescents identified as regular alcohol users met one or two of the three symptoms of alcohol dependence needed for a full diagnosis, resulting in pathological but undiagnosed “diagnostic orphans” (Martin and Winters 1998; Pollock and Martin 1999). A lower threshold enabled us to capture these cases for all classes of drugs at an age when fully manifest disorder remains uncommon. Second, the Early Adolescent Onset PSU Alcohol and Illicit group sample sizes were too small to analyze at a higher PSU threshold. Third, interpretation of our prospective analyses (e.g., predicting PSU at age 18 using P3AR at age 14) was much more straight forward if all adolescents evidencing any signs of substance use at age 14 were classified in the Early Adolescent Onset groups. This left unambiguous the meaning of membership in the Late Adolescent Onset groups since those in these groups were symptom free at the age-14 assessment.

Diagnostic interviews were conducted by trained research assistants with at least a bachelor’s degree in psychology or a related field. The adolescent and at least one parent were independently interviewed about the adolescent’s substance use habits. SUD was assessed using a modified version of the Diagnostic Interview for Children (Reich and Welner 1998) at the age 11 and age 14 assessments, and using the expanded Substance Abuse Module of the Composite International Diagnostic Interview (Robins, Babor et al. 1987) at the age 18 assessment. DSM-3R symptoms were assessed at all assessments. Based on the timing of its release, DSM-IV criteria were introduced during the age 14 assessment for females and the age 18 assessment for males. The diagnostic categories of abuse and dependence were assessed for alcohol, cannabis, amphetamine, cocaine, hallucinogen, inhalant, opiate, PCP, and “miscellaneous” drugs, which applies the diagnostic criteria to drugs, such as steroids or nitrous oxide, which do not have formal DSM diagnostic categories. We also assessed nicotine dependence, but not abuse, which is consistent with the DSM.

A minimum of two advanced graduate students reviewed the diagnostic data and assigned symptoms in diagnostic case conferences. Symptoms were considered present if either the parent or the child endorsed the symptom because this method has been shown to improve diagnostic reliability over single informant report (Reich and Earls 1987; Burt, Krueger et al. 2001). Using these methods, kappa coefficients for the reliability of PSU-related DSM diagnoses exceed .90.

749 participants (64.8%) did not meet our definition of PSU by their age 18 assessment (e.g., Controls), 134 participants (11.6%) met our definition of PSU by their age 14 assessment (Early Adolescent Onset), and 273 participants (23.6%) first met our definition of Late Adolescent Onset PSU. 43.9% (236/537) of males and 27.6% (171/619) of females met criteria for our PSU group. We measured drug class comorbidity as a sum score that ranged from 0 (e.g., Controls) to 3 (e.g., the adolescent met our definition of PSU for PSU-Alcohol, PSU-Nicotine, and PSU-Illicit) separately for Early Adolescent Onset and Late Adolescent Onset. PSU-Nicotine dependence was the most common form of PSU. There was substantial drug class comorbidity (see Table 1).

Table 1. Frequency of Pathological Substance Use by Drug Class, Onset Group, and Comorbidity.

Early
Adolescent Late Adolescent
Type of PSU Onset Onset
Any PSU 134 (66) 273 (170)
 PSU-Nicotine 123 (61) 181 (116)
 PSU-Alcohol 52 (28) 164 (110)
 PSU-Illicit 41 (23) 114 (82)
Classes of PSU met
 1 out of 3 classes of PSU 70 (30) 139 (73)
 2 out of 3 classes of PSU 46 (26) 82 (56)
 3 out of 3 classes of PSU 18 (10) 52 (41)

Note: “PSU-Illicit” includes adolescents that met our study’s criteria for PSU based on the use of Amphetamine, Cannabis, Cocaine, Hallucinogens, Inhalant, Opiate, PCP, or miscellaneous. The number of males in each group is presented in parenthesis.

Data Analysis

To examine potential associations between P3AR and each PSU class, we used the PROC MIXED procedure in SAS version 9.2 to conduct separate MANOVA analyses for PSU-Nicotine, PSU-Alcohol, and PSU-Illicit. PSU status (0 = Controls, 1 = Early Adolescent Onset, 2 = Late Adolescent Onset), Gender, and Electrode site were fixed factors in a full factorial linear mixed model MANOVA design. Because twins violate the assumption of independent sampling, we used the PROC MIXED procedure to adjust for twin similarity in P3 at the two electrodes and the covariance between the electrodes. This provides an appropriate adjustment for statistical significance in each analysis. Restricted maximum likelihood (REML) estimation was utilized to account for missing data. REML yields unbiased estimates if data are missing at random (Little and Rubin 1987), and produces estimates that agree with ANOVA estimates when data are balanced. Fisher’s least significant difference (LSD) post hoc comparisons were used to test for group mean differences (e.g., Early Adolescent Onset vs. Controls, Late Adolescent Onset vs. Controls, Early Adolescent Onset vs. Late Adolescent Onset) within each MANOVA. Effect sizes were calculated using the pooled-standard deviation formula for Cohen’s d.

The drug class comorbidity analyses were structured exactly like the three main analyses except that drug class comorbidity was entered as a continuous predictor from 0 to 3 instead of as a fixed factor.

Results

Figure 1 presents the grand average waveforms for the PZ (top) and FZ (bottom) electrodes. For convenience, we present the waveforms for Early and Late Adolescent Onset groups (respectively) by PSU class. Figure 2 presents the grand average waveforms for Early and Late Adolescent Onset groups (respectively) by comorbidity. The waveform for Controls is presented in each graph in each Figure for comparison purposes.

Figure 1.

Figure 1

Event-related potential grand means recorded at age 14 by PSU group for PZ (top) and FZ (bottom) by Diagnostic Group.

Figure 2.

Figure 2

Event-related potential grand means recorded at age 14 by PSU group for PZ (top) and FZ (bottom) by Comorbidity

Table 2 summarizes the statistical results for the three main analyses examining the association between each PSU drug class and P3AR. The main effect of PSU group for PSU-Nicotine, PSU-Alcohol, and PSU-Illicit was statistically significant. The right side of Table 2 summarizes the results of the post hoc comparisons for the Early Adolescent Onset group (vs. Controls) and Late Adolescent Onset groups (vs. Controls) for each of the three drug classes. For each drug class, the difference between Early Adolescent Onset PSU group and Controls was statistically significant, which indicates that P3AR was associated with Early Adolescent Onset PSU. Individuals in the Late Adolescent Onset PSU-Alcohol and PSU-Illicit, but not PSU-Nicotine group demonstrated statistically significant P3AR relative to Controls. In other words, P3AR preceded the eventual manifestation of Late Adolescent Onset PSU-Alcohol and Late Adolescent Onset PSU-Illicit. Although P3 amplitude was smaller than that of controls for both PSU nicotine groups, this effect was not statistically significant for Late Onset PSU-Nicotine.

Table 2. Adolescent P3 Amplitude as a Function of PSU Onset.

Substanc
e Group
PSU
Onset
Electrode n Mean (SD)
P3 Amplitude,
uV
Main effect
for PSU
group vs
Controls
LSD post hoc
test for PSU
Group Onset
vs. Controls, p
value
Effect
Size for PSU
Onset vs.
Controls
PSU-
Nicotine
Early PZ 122 30.21 (9.63) F(2,1013) =
3.58, p =
0.03
0.01 0.19
FZ 123 5.96 (6.30)
Late PZ 181 30.49 (9.35) 0.10 0.16
FZ 181 6.26 (7.81)
PSU-
Alcohol
Early PZ 51 29.06(9.56) F(2,924) =
4.93, p =
0.01
0.02 0.30
FZ 52 5.24 (5.44)
Late PZ 161 30.37 (8.63) 0.02 0.21
FZ 164 5.69 (7.00)
PSU-
Illicit
Early PZ 40 29.16 (9.54) F(2,869) =
5.37, p <
0.01
0.02 0.32
FZ 41 4.91 (5.19)
Late PZ 111 30.19 (9.12) 0.01 0.23
FZ 114 5.48 (7.53)

Note: The characteristics of the control group for PZ were: n=743, mean = 32.01; SD = 9.09. The characteristics of the control group for FZ were: n = 746; mean = 7.31; SD = 7.36.

Consistent with our hypotheses, the number of comorbid drug class diagnoses was inversely associated with P3AR: F (1, 1138) = 11.36, p < 0.01, beta = −0.92. This effect remained significant when examining the Early Adolescent Onset and Late Adolescent Onset groups separately (F(1, 764) = 7.93, p < 0.01, beta = −1.28 and F(1,1013) = 7.19, p < 0.01, beta = −0.85, respectively).

Certain results were consistent across analyses and are discussed here for brevity. The main effect of gender and the interactions involving the gender term were not statistically significant in any main analysis or posthoc test (all p-values > 0.20). This includes the Gender by PSU status interaction term, which for each PSU class tested our apriori hypothesis that P3AR would be more strongly associated with risk for PSU in males than females. The main effect of Electrode was statistically significant in each analysis (e.g., PZ-P3, compared to FZ-P3, had a larger mean amplitude). The interactions involving Electrode were not statistically significant in any analysis (all ps > 0.20), suggesting that FZ-P3 and PZ-P3 do not significantly differ in the strength of their association with PSU. For simplicity, in Table 2 we presented Cohen’s d effect size as the average of the two electrodes. The Late Adolescent Onset group, although having slightly larger P3 amplitudes than the Early Adolescent Onset group, did not differ significantly from the Early Adolescent Onset group in any analysis. Lastly, response accuracy to targets was not associated with PSU group status for any of the three drug classes relative to controls, nor of PSU comorbidity (all groups performed at about 97% accuracy, all ps > 0.50).

Discussion

Based on converging lines of evidence, P3AR has been proposed to index a liability to misuse licit and illicit substances. Previous studies have noted that 1) P3AR distinguishes individuals who meet criteria for a SUD from controls, and 2) P3AR identifies individuals who are at an elevated familial/genetic risk for SUD in the absence of manifest substance use problems (Euser, Arends et al. 2012). To better characterize the association between P3AR and PSU, we utilized data from over 1100 adolescents from a community sample of 14 year-old male and female adolescents. We examined the timing of PSU onset, three PSU drug classes, drug class comorbidity, and gender differences. P3 amplitude was assessed at age 14 from PZ and FZ scalp locations during a visual oddball paradigm. PSU was assessed at age 11, age 14, and age 18 by multi-informant report using trained diagnostic interviewers.

We found that P3AR recorded at age 14 was associated with Early Adolescent Onset PSU for nicotine, alcohol, illicit drugs, and comorbidity and with Late Adolescent Onset PSU for alcohol, illicit drugs, and comorbidity. Except for PSU-Nicotine, where the observed diminished P3 amplitude was not significant, our findings also support the notion that P3AR predates the expression of PSU (Berman, Whipple et al. 1993; Hill, Steinhauer et al. 1995; Iacono, Carlson et al. 2002). Moreover, that we found P3AR across Early Adolescent Onset PSU groups supports the link between P3AR and particularly precocious substance misuse (Hill, Steinhauer et al. 2009). Based on these findings, P3AR may be more parsimoniously described as a marker of a general liability for substance misuse (Iacono, Malone et al. 2003), rather than as a marker of risk for any specific class of drug. While similar findings have been reported for older cohorts of adolescents (e.g., ages 18-21 years), our study is the first to show that P3AR is associated with a variety of substance use outcomes in a large cohort of younger adolescents (e.g., age 14 years). That we showed a robust association between P3AR recorded from 14 year-old adolescents and PSU is especially important because the onset of substance use before age 14 is associated with increased risk for substance use disorders in adulthood (Grant and Dawson 1997). Thus, our results suggest that P3AR may be a valuable tool for developmental researchers to study this risk across adolescence.

In our study, P3 amplitude in the Late Adolescent Onset PSU-Nicotine group was not significantly different from that of the Control group, although the effect was in the hypothesized direction. One interpretation consistent with our findings is that Early-Onset PSU-Nicotine may be indicative of a general liability to misuse drugs (that is also indexed by P3AR), whereas Late-Onset PSU-Nicotine may be subject to etiological factors less associated with a general liability to misuse drugs (as suggested in the molecular genetic study of this sample by Vrieze, McGue et al. 2012) and is thus less associated with P3AR. Although speculative, P3AR and most forms of PSU may relate to degree of social deviance in disinhibited behavior (Carlson, McLarnon et al. 2007). Accordingly, onset of nicotine misuse after the age of 14 might have been more socially normative for this cohort than other forms of PSU that we examined. Consistent with these ideas, there is growing evidence that the genetic and environmental factors influencing the development of SUDs are dynamic across the lifespan rather than fixed (Vrieze, Hicks et al. 2012).

We failed to find support for our hypothesis that P3AR would index risk for PSU in males more than females. This is in contrast to previous cross-sectional research that found P3AR to predict risk for PSU in males more so than in females (Euser, Arends et al. 2012). However, others have reported the absence of gender differences in P3AR (Nelson, Patrick et al. 2011). The cause of this inconsistency is unclear. It may be due to characteristics of our sample (e.g., their young age) or how we defined PSU (e.g., including subclinical cases). It should be noted that we examined a relatively large sample of female adolescents for this literature.

Etiology of P3AR and Association with PSU

Our findings may help contribute to refining theory as to why P3AR indexes risk for PSU. One theory that has been proposed in the literature is that P3AR indexes risk for externalizing psychopathology (EXT). EXT is a highly heritable (h2 = 0.80; Hicks, Krueger et al. 2004) disinhibitory trait marked by impulsivity, recklessness, and aggressiveness, that is postulated to account for comorbidity between classes of drug misuse, antisocial personality disorder, and childhood disruptive disorders (Kendler, Prescott et al. 2003; Krueger, Markon et al. 2005). In this framework, genetic factors interact with experiential factors, such as substance availability and peer role models (Kessler, Cox et al. 2011) to help to determine which disorders are manifested by an individual with an elevated EXT trait. In support of the notion that P3AR indexes risk for PSU via EXT, EXT has been found to mediate associations between P3AR and specific disorders within the EXT spectrum, such as alcohol dependence (Patrick, Bernat et al. 2006).

Furthermore, P3AR may represent an endophenotype for EXT (Hesselbrock, Begleiter et al. 2001). As described by Iacono and Malone (2003), an endophenotype is an objectively-assessed inherited attribute that indexes genetic risk for disease. Candidate endophenotypes must be heritable, reliably assessed, associated with the disease phenotype longitudinally, and manifest in carriers of genes associated with risk for disease regardless of disease state. P3AR has demonstrated many of these qualities (see Iacono and Malone 2011). Importantly, twin studies employing biometric modeling have found that the correlation between P3AR and EXT can be largely accounted for by shared genetic processes in the best fit model (King, Burt et al. 2005; Hicks, Bernat et al. 2007; Gilmore, Malone et al. 2010). The P3AR-EXT association has now been replicated across laboratories, countries, decades, and several operational definitions for EXT and is thus consistent with the notion that these phenotypes share genetic variants in common. Along these lines, there is emerging evidence from molecular genetics research as to which candidate alleles influence P3AR and risk for substance abuse Along these lines, there is emerging evidence from molecular genetics research as to which candidate alleles influence P3AR and risk for substance abuse (Zlojutro, Manz et al. 2011), although replication in this area is needed.

In contrast to the endophenotype hypothesis of P3AR, protracted misuse of drugs could affect brain maturation in a manner that leads to P3AR. This possibility is difficult to examine empirically because experimentation in humans is not feasible and substance-exposed populations may manifest both putative consequences of PSU and preexisting risk factors. Previous research with this cohort has found little support for this neurotoxicity hypothesis when examining the association between PZ-P3AR and cumulative alcohol use behaviors by age 18 (Perlman, Johnson et al. 2009) and when controlling for prenatal alcohol exposure (Iacono, Carlson et al. 2002). In addition, P3AR has been found to index familial risk for substance misuse even among substance-naïve 8 year-olds (Viana-Wackermann, Furtado et al. 2007). All of these findings detract from the possibility that P3AR is associated with PSU as a result of neurotoxicity effects, although we cannot rule out the neurotoxicity hypothesis in this study.

Limitations of the Present Investigation

The current investigation is limited in several ways. First, some members of the control group may have gone on to develop PSU after the period studied. This may led us to underestimate the predictive strength of P3AR. Second, although our sample accurately reflects the population of individuals born in Minnesota during the birth-years from which participants were drawn, it is disproportionately Caucasian compared to current national demographics. Replication with diverse samples is needed to generalize these findings to other populations. Third, it is not clear whether P3AR would show similar prognostic value among clinical populations of adolescents or adolescents drawn from families with multiple members in treatment for addiction. The P3AR effect size has been found to be greater when contrasting drug abusers to controls from a clinical sample relative to an epidemiology sample (Euser, Arends et al. 2012). However, that we examined an epidemiological sample allowed us to generalize our findings to the greater proportion of the population. Fourth, it is possible that our results might have been different had we defined PSU more stringently. Our definition of PSU was designed to be inclusive and developmentally sensitive, allowing us to catch adolescents with emerging substance use problems, and to make our longitudinal analyses easier to interpret. Our results may have been different had we included only severe cases of PSU, or examined quantitative substance use variables instead of diagnostic constructs. Research into the optimal measurement of substance use phenotypes is needed for future P3AR studies, especially when examining adolescents. Fifth, our manuscript was focused on P3 amplitude because this component has the strongest empirical validation as a marker of risk for PSU. As a result, we may have missed other relevant ERP variants, such as the late slow wave, the P2, and component latencies. While other components and latencies have not stood out as markers of risk for PSU, more comprehensive and exploratory ERP research might identify additional components relevant to understanding PSU.

Our study was also not explicitly designed to provide a neurophysiological or neurocognitive explanation for P3AR. Instead, our study was designed to look at the association between P3AR and PSU based on drug class and time of onset. One current theory suggests that P3 amplitude reflects a temporary period of cortical inhibition influenced by a norepinephrine-mediated system that modulates attention and ultimately contributes to efficient stimulus processing and decision making (Nieuwenhuis, Aston-Jones et al. 2005; Nieuwenhuis, De Geus et al. 2010). Thus, P3AR might represent a dysfunction in this system, perhaps in cortical inhibition, dysregulated attention, and/or maintaining vigilance. However, whether these constructs explain the association between P3AR and PSU is unclear because concepts such as distractibility and cortical inhibition are both relatively far removed from the day to day behavior of drug use. Furthermore, many cortical and subcortical regions may influence oddball stimulus processing (Polich 2007), and there are many physiological and psychological phenomenon that might reduce P300 amplitude. Unfortunately, multi-sensor arrays were not added until a later assessment and we are unable to examine topographic differences related to PSU that might shed light on the underlying neural substrates. Many psychiatric disorders are associated with P3AR, including schizophrenia, autism, and bipolar disorder. Perhaps, P3AR indexes a deficit or weakness in attention present across many psychiatric disorders. Alternatively, the mechanism for P3AR in SUDs may be unique and separate relative to mechanisms for reduced P300 amplitude for other types of disorders. Evidence suggests that P3AR presents in many EXT-related phenotypes due to a common mechanism (Patrick, Bernat et al. 2006). There is no evidence at this time to suggest that other psychiatric disorders evidence P3AR due to a common mechanism. Further work is needed to more precisely delineate the neural processes that underlie P3AR in those at risk for PSU, and to clarify the significance of P3AR from a multi-level perspective (genetic, cognitive, and physiologic) in PSU, as well as in other kinds of disorders.

Conclusions

Our findings support the notion that P3AR signals the presence of elevated risk for PSU during adolescence and strengthens evidence that P3AR may represent a candidate endophenotype for PSU. As described by Iacono and Malone (2011), endophenotypes, such as P3AR, are useful in psychopathology research because they may provide an index of latent risk for disease even though disease expression may change across development. Importantly, P3AR seems to index risk for substance use broadly, rather than risk for a specific class of drug. Broader phenotypes are advantageous in molecular genetic research where very large sample sizes are needed for statistical power. In contrast to broader phenotypes, it may be difficult to collect very large samples of a narrow phenotype (e.g., Alcohol Dependence). Further understanding of the association between P3AR and risk for PSU may help refine developmental models of risk for PSU.

Acknowledgments

This research was supported by the NIH grants DA 005147, DA 013240, DA 024417, and AA 09367. The first author’s effort on this project was supported by T32 grant MH017069 from the National Institute of Mental Health. Special thanks to Steve Malone for helping with data processing and analysis.

Contributor Information

Greg Perlman, University of Minnesota.

Abe Markin, University of Minnesota.

William G. Iacono, University of Minnesota

References

  1. Anokhin AP, Vedeniapin AB, et al. The P300 brain potential is reduced in smokers. Psychopharmacology (Berl) 2000;149(4):409–413. doi: 10.1007/s002130000387. [DOI] [PubMed] [Google Scholar]
  2. Bauer LO. Frontal P300 decrements, childhood conduct disorder, family history, and the prediction of relapse among abstinent cocaine abusers. Drug Alcohol Depend. 1997;44(1):1–10. doi: 10.1016/s0376-8716(96)01311-7. [DOI] [PubMed] [Google Scholar]
  3. Begleiter H, Porjesz B, et al. Event-related brain potentials in boys at risk for alcoholism. Science. 1984;225(4669):1493–1496. doi: 10.1126/science.6474187. [DOI] [PubMed] [Google Scholar]
  4. Berman SM, Whipple SC, et al. P3 in young boys as a predictor of adolescent substance use. Alcohol. 1993;10(1):69–76. doi: 10.1016/0741-8329(93)90055-s. [DOI] [PubMed] [Google Scholar]
  5. Blazei RW, Iacono WG, et al. Father-child transmission of antisocial behavior: the moderating role of father’s presence in the home. J Am Acad Child Adolesc Psychiatry. 2008;47(4):406–415. doi: 10.1097/CHI.0b013e3181642979. [DOI] [PubMed] [Google Scholar]
  6. Burt SA, Krueger RF, et al. Sources of covariation among attention-deficit/hyperactivity disorder, oppositional defiant disorder, and conduct disorder: the importance of shared environment. J Abnorm Psychol. 2001;110(4):516–525. doi: 10.1037/0021-843X.110.4.516. [DOI] [PubMed] [Google Scholar]
  7. Carlson SR, Iacono WG, et al. P300 amplitude in nonalcoholic adolescent twin pairs who become discordant for alcoholism as adults. Psychophysiology. 2004;41(6):841–844. doi: 10.1111/j.1469-8986.2004.00238.x. [DOI] [PubMed] [Google Scholar]
  8. Carlson SR, McLarnon ME, et al. P300 amplitude, externalizing psychopathology, and earlier- versus later-onset substance-use disorder. J Abnorm Psychol. 2007;116(3):565–577. doi: 10.1037/0021-843X.116.3.565. [DOI] [PubMed] [Google Scholar]
  9. Costa L, Bauer L, et al. Frontal P300 decrements, alcohol dependence, and antisocial personality disorder. Biol Psychiatry. 2000;47(12):1064–1071. doi: 10.1016/s0006-3223(99)00317-0. [DOI] [PubMed] [Google Scholar]
  10. Euser AS, Arends LR, et al. The P300 event-related brain potential as a neurobiological endophenotype for substance use disorders: a meta-analytic investigation. Neuroscience and biobehavioral reviews. 2012;36(1):572–603. doi: 10.1016/j.neubiorev.2011.09.002. [DOI] [PubMed] [Google Scholar]
  11. Gilmore CS, Malone SM, et al. Relationship between the P3 event-related potential, its associated time-frequency components, and externalizing psychopathology. Psychophysiology. 2010;47(1):123–132. doi: 10.1111/j.1469-8986.2009.00876.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gottesman, Gould TD. The endophenotype concept in psychiatry: etymology and strategic intentions. Am J Psychiatry. 2003;160(4):636–645. doi: 10.1176/appi.ajp.160.4.636. [DOI] [PubMed] [Google Scholar]
  13. Grant BF, Dawson DA. Age at onset of alcohol use and its association with DSM-IV alcohol abuse and dependence: results from the National Longitudinal Alcohol Epidemiologic Survey. J Subst Abuse. 1997;9:103–110. doi: 10.1016/s0899-3289(97)90009-2. [DOI] [PubMed] [Google Scholar]
  14. Gratton G, Coles MG, et al. A new method for off-line removal of ocular artifact. Electroencephalogr Clin Neurophysiol. 1983;55(4):468–484. doi: 10.1016/0013-4694(83)90135-9. [DOI] [PubMed] [Google Scholar]
  15. Habeych ME, Charles PJ, et al. Direct and mediated associations between P300 amplitude in childhood and substance use disorders outcome in young adulthood. Biol Psychiatry. 2005;57(1):76–82. doi: 10.1016/j.biopsych.2004.09.028. [DOI] [PubMed] [Google Scholar]
  16. Hesselbrock V, Begleiter H, et al. P300 event-related potential amplitude as an endophenotype of alcoholism--evidence from the collaborative study on the genetics of alcoholism. Journal of biomedical science. 2001;8(1):77–82. doi: 10.1007/BF02255974. [DOI] [PubMed] [Google Scholar]
  17. Hicks BM, Bernat E, et al. Genes mediate the association between P3 amplitude and externalizing disorders. Psychophysiology. 2007;44(1):98–105. doi: 10.1111/j.1469-8986.2006.00471.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hicks BM, Krueger RF, et al. Family transmission and heritability of externalizing disorders: a twin-family study. Arch Gen Psychiatry. 2004;61(9):922–928. doi: 10.1001/archpsyc.61.9.922. [DOI] [PubMed] [Google Scholar]
  19. Hill SY, Shen S, et al. Factors predicting the onset of adolescent drinking in families at high risk for developing alcoholism. Biol Psychiatry. 2000;48(4):265–275. doi: 10.1016/s0006-3223(00)00841-6. [DOI] [PubMed] [Google Scholar]
  20. Hill SY, Steinhauer S, et al. Eight-year longitudinal follow-up of P300 and clinical outcome in children from high-risk for alcoholism families. Biol Psychiatry. 1995;37(11):823–827. doi: 10.1016/0006-3223(95)00041-E. [DOI] [PubMed] [Google Scholar]
  21. Hill SY, Steinhauer SR, et al. Childhood risk factors for young adult substance dependence outcome in offspring from multiplex alcohol dependence families: a prospective study. Biol Psychiatry. 2009;66(8):750–757. doi: 10.1016/j.biopsych.2009.05.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Iacono WG, Carlson SR, et al. P3 event-related potential amplitude and the risk for disinhibitory disorders in adolescent boys. Arch Gen Psychiatry. 2002;59(8):750–757. doi: 10.1001/archpsyc.59.8.750. [DOI] [PubMed] [Google Scholar]
  23. Iacono WG, Carlson SR, et al. Behavioral disinhibition and the development of substance-use disorders: findings from the Minnesota Twin Family Study. Dev Psychopathol. 1999;11(4):869–900. doi: 10.1017/s0954579499002369. [DOI] [PubMed] [Google Scholar]
  24. Iacono WG, Malone SM. Developmental Endophenotypes: Indexing Genetic Risk for Substance Abuse With the P300 Brain Event-Related Potential. Child Development Perspectives. 2011;5(4):239–247. doi: 10.1111/j.1750-8606.2011.00205.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Iacono WG, Malone SM, et al. Substance use disorders, externalizing psychopathology, and P300 event-related potential amplitude. Int J Psychophysiol. 2003;48(2):147–178. doi: 10.1016/s0167-8760(03)00052-7. [DOI] [PubMed] [Google Scholar]
  26. Iacono WG, Malone SM, et al. Behavioral disinhibition and the development of early-onset addiction: common and specific influences. Annu Rev Clin Psychol. 2008;4:325–348. doi: 10.1146/annurev.clinpsy.4.022007.141157. [DOI] [PubMed] [Google Scholar]
  27. Iacono WG, McGue M. Minnesota Twin Family Study. Twin Res. 2002;5(5):482–487. doi: 10.1375/136905202320906327. [DOI] [PubMed] [Google Scholar]
  28. Kempel P, Lampe K, et al. Auditory-evoked potentials and selective attention: different ways of information processing in cannabis users and controls. Neuropsychobiology. 2003;48(2):95–101. doi: 10.1159/000072884. [DOI] [PubMed] [Google Scholar]
  29. Kendler KS, Prescott CA, et al. The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women. Arch Gen Psychiatry. 2003;60(9):929–937. doi: 10.1001/archpsyc.60.9.929. [DOI] [PubMed] [Google Scholar]
  30. Kessler RC, Cox BJ, et al. The effects of latent variables in the development of comorbidity among common mental disorders. Depress Anxiety. 2011;28(1):29–39. doi: 10.1002/da.20760. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. King SM, Burt SA, et al. Etiological contributions to heavy drinking from late adolescence to young adulthood. J Abnorm Psychol. 2005;114(4):587–598. doi: 10.1037/0021-843X.114.4.587. [DOI] [PubMed] [Google Scholar]
  32. Krueger RF, Markon KE, et al. Externalizing psychopathology in adulthood: a dimensional-spectrum conceptualization and its implications for DSM-V. J Abnorm Psychol. 2005;114(4):537–550. doi: 10.1037/0021-843X.114.4.537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Little RJA, Rubin DB. Statistical analysis with missing data. Wiley; New York: 1987. [Google Scholar]
  34. Malone SM, Iacono WG, et al. Event-related potentials and comorbidity in alcohol-dependent adult males. Psychophysiology. 2001;38(3):367–376. [PubMed] [Google Scholar]
  35. Martin CS, Winters KC. Diagnosis and assessment of alcohol use disorders among adolescents. Alcohol health and research world. 1998;22(2):95–105. [PMC free article] [PubMed] [Google Scholar]
  36. McGue M, Iacono WG, et al. Origins and consequences of age at first drink. I. Associations with substance-use disorders, disinhibitory behavior and psychopathology, and P3 amplitude. Alcohol Clin Exp Res. 2001;25(8):1156–1165. [PubMed] [Google Scholar]
  37. Nelson LD, Patrick CJ, et al. Operationalizing proneness to externalizing psychopathology as a multivariate psychophysiological phenotype. Psychophysiology. 2011;48(1):64–72. doi: 10.1111/j.1469-8986.2010.01047.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Nieuwenhuis S, Aston-Jones G, et al. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychol Bull. 2005;131(4):510–532. doi: 10.1037/0033-2909.131.4.510. [DOI] [PubMed] [Google Scholar]
  39. Nieuwenhuis S, De Geus EJ, et al. The anatomical and functional relationship between the P3 and autonomic components of the orienting response. Psychophysiology. 2010 doi: 10.1111/j.1469-8986.2010.01057.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Patrick CJ, Bernat EM, et al. P300 amplitude as an indicator of externalizing in adolescent males. Psychophysiology. 2006;43(1):84–92. doi: 10.1111/j.1469-8986.2006.00376.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Perlman G, Johnson W, et al. The heritability of P300 amplitude in 18-year-olds is robust to adolescent alcohol use. Psychophysiology. 2009;46(5):962–969. doi: 10.1111/j.1469-8986.2009.00850.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Polich J. Updating P300: an integrative theory of P3a and P3b. Clin Neurophysiol. 2007;118(10):2128–2148. doi: 10.1016/j.clinph.2007.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Pollock NK, Martin CS. Diagnostic orphans: adolescents with alcohol symptom who do not qualify for DSM-IV abuse or dependence diagnoses. Am J Psychiatry. 1999;156(6):897–901. doi: 10.1176/ajp.156.6.897. [DOI] [PubMed] [Google Scholar]
  44. Porjesz B, Begleiter H, et al. Visual evoked potential correlates of information processing deficits in chronic alcoholics. Adv Exp Med Biol. 1980;126:603–623. doi: 10.1007/978-1-4684-3632-7_46. [DOI] [PubMed] [Google Scholar]
  45. Reich W, Earls F. Rules for making psychiatric diagnoses in children on the basis of multiple sources of information: preliminary strategies. J Abnorm Child Psychol. 1987;15(4):601–616. doi: 10.1007/BF00917244. [DOI] [PubMed] [Google Scholar]
  46. Reich W, Welner A. Department of Psychiatry; St. Louis, Washington University School of Medicine: 1998. Revised version of the Diagnostic Interview Schedule for Children and Adolescents (DICA-R) [Google Scholar]
  47. Robins LN, Babor TF, et al. Composite international diagnostic interview: Expanded substance abuse module. Authors; St. Louis: 1987. [Google Scholar]
  48. Singh SM, Basu D, et al. Auditory P300 event-related potentials and neurocognitive functions in opioid dependent men and their brothers. Am J Addict. 2009;18(3):198–205. doi: 10.1080/10550490902786975. [DOI] [PubMed] [Google Scholar]
  49. Viana-Wackermann PC, Furtado EF, et al. Lower P300 amplitude in eight-year-old offspring of alcoholic fathers with a delinquent history. Eur Arch Psychiatry Clin Neurosci. 2007;257(4):211–216. doi: 10.1007/s00406-006-0709-8. [DOI] [PubMed] [Google Scholar]
  50. Vrieze SI, Hicks BM, et al. Decline in Genetic Influence on the Co-Occurrence of Alcohol, Marijuana, and Nicotine Dependence Symptoms From Age 14 to 29. The American journal of psychiatry. 2012 doi: 10.1176/appi.ajp.2012.11081268. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Vrieze SI, McGue M, et al. The interplay of genes and adolescent development in substance use disorders: leveraging findings from GWAS meta-analyses to test developmental hypotheses about nicotine consumption. Human genetics. 2012;131(6):791–801. doi: 10.1007/s00439-012-1167-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Yoon HH, Iacono WG, et al. Using the brain P300 response to identify novel phenotypes reflecting genetic vulnerability for adolescent substance misuse. Addictive behaviors. 2006;31(6):1067–1087. doi: 10.1016/j.addbeh.2006.03.036. [DOI] [PubMed] [Google Scholar]
  53. Zlojutro M, Manz N, et al. Genome-wide association study of theta band event-related oscillations identifies serotonin receptor gene HTR7 influencing risk of alcohol dependence. Am J Med Genet B Neuropsychiatr Genet. 2011;156(1):44–58. doi: 10.1002/ajmg.b.31136. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES