Abstract
Cognitive deficits are a core feature of psychiatric disorders like schizophrenia and bipolar disorder. Evidence supports a genome-wide polygenic score (GPS) for educational attainment (GPSEDU) can be used to explain variability in cognitive performance. We aimed to identify different cognitive domains associated with GPSEDU in a transdiagnostic clinical cohort of chronic psychiatric patients with known cognitive deficits. Bipolar and schizophrenia patients from the PsyCourse cohort (N = 730; 43% female) were used. Likewise, we tested whether GPSs for schizophrenia (GPSSZ) and bipolar disorder (GPSBD) were associated with cognitive outcomes. GPSEDU explained 1.5% of variance in the backward verbal digit span, 1.9% in the number of correctly recalled words of the Verbal Learning and Memory Test, and 1.1% in crystallized intelligence. These effects were robust to the influences of treatment and diagnosis. No significant associations between GPSSZ or GPSBD with cognitive outcomes were found. Furthermore, these risk scores did not confound the effect of GPSEDU on cognitive outcomes. GPSEDU explains a small fraction of cognitive performance in adults with psychiatric disorders, specifically for domains related to linguistic learning and working memory. Investigating such a proxy-phenotype longitudinally, could give intriguing insight into the disease course, highlighting at what time genes play a more influential role on cognitive performance. Better understanding the origin of these deficits might help identify those patients at risk for lower levels of functioning and poor social outcomes. Polygenic estimates may in the future be part of predictive models for more personalized interventions.
Subject terms: Genomics, Bipolar disorder, Schizophrenia, Clinical genetics, Learning and memory
Introduction
Cognitive deficits are a core and robust feature of psychiatric disorders like bipolar disorder and schizophrenia, present even during periods of remission1–3. These deficits are key predictors of long-term functional and social outcomes and are difficult to treat with current pharmaceutical options or behavioral interventions4–6. Considering the associated psychosocial burden and high prevalence of these deficits among patients, psychiatric researchers have put considerable effort towards understanding their underlying mechanisms. Thus far, genome-wide association studies (GWAS) have provided evidence supporting the polygenic architecture and remarkable heritability of cognitive performance in population-based cohorts7–10. Furthermore, evidence supports the phenotypic and genetic stability of individual cognitive differences across the lifetime in domains including executive functioning, attention, and verbal memory11–14. As studies have shown evidence of impairments even in unaffected first-degree relatives of individuals with psychiatric disorders, cognitive deficits have been hypothesized as a valuable endophenotype of interest for better understanding the genetic risk factors of psychiatric disorders15–17.
Intelligence, encompassing cognition, is highly heritable and an imperative predictor of occupational and health outcomes18. Despite high heritability estimates of intelligence, indicated to be up to 80% in adulthood19, unraveling the underlying genetic contribution of intelligence differences using GWAS has been challenging and thus far little of the observed heritability has been explained11,20. To date, studies on intelligence have been limited by insufficient sample sizes and further complicated by the challenge of precise and reliable measurements for this complex phenotype7–10,20. The latest GWAS on intelligence identified 205 genomic loci implicating up to 1016 genes, which explained approximately five percent of the variance in intelligence21. Another large study reported a genome-wide polygenic score (GPS) that could explain 4.3% of variance in general cognitive function11.
Educational attainment is moderately heritable and has been obtained as a demographic item in countless medical datasets and for cohorts of which genetic data is available22. In the last decade, educational attainment has been proposed as a proxy-phenotype for cognition, as it is highly associated with intelligence both on a phenotypic (0.50) and genetic level (0.65)18,20,23–26. Notably, GPS based on GWAS summary statistics for years of education predict more variance in intelligence than the phenotype years of education per se18,25, reflecting the substantial genetic correlation between both phenotypes. The largest GWAS of educational attainment published to date, based on 1.1 million individuals, identified 1271 lead single nucleotide polymorphisms (SNPs)22. Through a multi-phenotype analysis of educational attainment and three cognitive phenotypes, the authors were able to generate a GPS which explained 7–10% of variance in cognitive performance in the general population. The SNPs identified implicated genes involved in neurodevelopmental processes and neuron-to-neuron communication22. The authors showed that the use of educational attainment as a proxy-phenotype could uncover genetic variants to be used as a set of “empirically-based candidate genes” for future studies, for example testing associations with important endophenotypes like cognition27.
Studies have already demonstrated an important association of educational attainment GPS (GPSEDU) with cognitive performance, showing that, in a general population, a higher GPS is associated with higher performance on neurocognitive tests28. However, limited evidence exists supporting this association in patients with known cognitive deficits29,30, and there remains a need to investigate this association across different cognitive domains. Here we analyzed whether GPSEDU could be used to explain variability in different cognitive domains in chronic patients with schizophrenia and bipolar disorder from the PsyCourse cohort31. This transdiagnostic approach aligns with the growing evidence for dimensional models that cut across diagnostic categories in psychiatry and is supported by the large cognitive, clinical and genetic overlaps between both disorders32,33. Particularly, the genetic overlap between both disorders has been firmly established by heritability estimates derived from population-based multi-generation registers34 and by recent molecular studies that have reported an outstanding genetic correlation (rg = 0.70 ± 0.02)35.
Considering the positive genetic correlations reported between education and both schizophrenia (rg = 0.10) and bipolar risk (rg = 0.28)36, we further assessed how GPSs for both schizophrenia (GPSSZ) and bipolar disorder (GPSBD) were associated with cognitive performance in our sample.
Materials & methods
Participants
Data were used from the multicenter, PsyCourse study in Germany and Austria, consisting of participants of European ancestry (www.PsyCourse.de)31. Participants were phenotyped using a comprehensive battery including data on socio-demographics, psychopathology, cognition, and functioning assessed at each of four visits (6-month intervals). Recruitment strategies and characterization of all participants has been previously described in detail31. The sample selected for this project comprised a total of 730 participants with a DSM-IV37 diagnosis of schizophrenia, schizoaffective disorder, or bipolar disorder (type I or II). Additionally, cognitive data available from 320 nonclinical (control) participants was used to give an orientation to the range of phenotypic data available in the PsyCourse cohort and to confirm general, well-replicated findings of lower cognitive performance in patients with psychiatric disorders compared to healthy controls. The study was approved by the local ethics committee for each study center and was carried out following the rules of the Declaration of Helsinki. All individuals provided written informed consent as previously described31.
Psychopathology psychometric instrument
The Positive and Negative Syndrome Scale (PANSS) is a clinical instrument used to measure symptom severity in schizophrenia and routinely used to assess a variety of disorders including bipolar disorder38. A continuous, total score of the three subscales, i.e., positive (e.g., hallucinations and delusions), negative (e.g., emotional withdrawal and blunted affect), and general symptoms (e.g., somatic concern and poor attention) was used as an indication of disease severity at the time of testing.
Cognitive performance psychometric instruments
Cognitive tests were administered at each study visit. The Verbal Learning and Memory Test (VLMT) was introduced at visit 2. For all other cognitive measures, scores from visit 1 were used for analyses.
Crystallized intelligence
The MWT–B (Mehrfachwahl–Wortschatz–Intelligenz test) was used to measure crystallized intelligence39,40. In this test, subjects were presented with 37 sets of five words arranged according to the level of difficulty. Four words of each set were fictitious constructions of known vernacular (i.e., they do not exist in the German language), while one word really exists. Subjects were asked to cross out the word they know to exist. The total number of correctly marked lines was used as a score40.
Trail-Making-Test (TMT)
The TMT is a measure of visual attention and task switching and is one of several executive functioning measures. The test consists of two parts, part A assesses psychomotor speed of the participant, and part B assesses switching between two automated tasks (counting and reciting the alphabet). The time taken to complete each part of the test was measured and the difference in time needed (part B-part A) was used, as it is considered a more accurate measure of the divided attention and alternating sequencing tasks tested in part B41–43. In this case, a higher score meant worse cognitive performance.
Verbal digit span
The verbal digit span, from the Wechsler Adult Intelligence Scale, assesses short-term (forward digit-span) and working memory capacity (backward digit-span). Briefly, participants were asked to recall verbally a sequence of digits, with increasingly longer sequences in each trial. For each correctly recalled string of digits, one point was given. The test was ended when the participant was unable to correctly repeat two presented strings of the same length. The difference between the forward and backward task is that the latter involves mental manipulation as the participant is required to repeat the digits in backward order44. A score for each task was considered.
Digit-Symbol-Test (DST)
The DST is a subset of the Wechsler Adult Intelligence Scale45 and measures processing speed, working memory, visuospatial processing and attention. In this test, the participant was asked to use a key of numbers 1–9 with coinciding symbols to draw the appropriate symbol that matched the number given. The participant was given 120 s to fill in as many corresponding symbols as possible. In the end, the correct number of symbols drawn was totaled to get an overall score.
Verbal Learning and Memory Test (VLMT)
The VLMT is the German version of the Auditory Verbal Learning Test46. This word-list learning paradigm assesses several memory parameters through serial list learning with subsequent distraction, retrieval after distraction and half-hour time delay, and through a recognition task. The test consists of two different word lists which are each 15 independent words and a recognition list which includes 30 words from the two lists and 20 similar distractor words. Four VLMT scores were rated, the first for the number of correctly recalled words from the first list, a second score for the number of words lost after distraction, a third score of words lost after a time interval, and a fourth score of correctly recalled words from the recognition list47.
Biological samples
Peripheral blood samples were used for DNA extraction using standard techniques. DNA samples were then used to genotype patients for calculation of GPSs. Genotype data for controls was not available at the time of this investigation and they have not been used for GPS analyses.
GPS estimation
DNA samples were genotyped using the Infinium PsychArray Beadchip (Psychip, Illumina, San Diego, CA, USA). Following standard quality control procedures, imputation was performed using the 1000 Genomes Phase 3 reference panel as previously described in detail31,48. GPSs were calculated for all individuals using PLINK 1.90b5.349. Summary statistics for educational attainment were obtained from the Social Science Genetic Association Consortium (https://www.thessgac.org/data)22. These summary stats are derived from analyses excluding 23andMe samples. Summary statistics from the most recent Psychiatric Genomics Consortium GWASs for schizophrenia50 and bipolar disorder51 were used. All GPSs were calculated based on summary statistics from the discovery datasets, excluding low quality imputed variants (info score < 80%) in the test dataset, rare SNPs (minor allele frequency < 0.05), and ambiguous markers (A/T and C/G). Following the methodology of previous studies52, SNPs in the extended major histocompatibility complex region (chromosome 6: 25–34 Mbp) were completely excluded for the calculation of GPSEDU while only the top-associated SNP in this region was included for the calculations of GPSSZ and GPSBD. Data was clumped in windows of 500 kbp, discarding variants in LD (R2 > .1) with another more significantly associated marker.
GPSs were then calculated by multiplying the imputation dosage for each risk allele by the log(OR) of each genetic variant. The resulting values were summed to obtain an individual estimate of the genetic burden in each individual across different SNP p-value thresholds (pT). Scores for GPSSZ and GPSBD were calculated based on best discrimination thresholds according to previous findings, i.e., pT < 0.0550 and pT < 0.0151, respectively. GPSEDU was calculated at four different p-value thresholds, from including only genome-wide significant SNPs to inclusion of all SNPs: pT < 5 × 10−8, 0.05, 0.1, and 1. All GPSs were approximately normally distributed and standardized via z-score transformation.
Statistical analyses
Sample characteristics
As proof of concept, the effect of case status on cognitive performance was investigated using participants from the PsyCourse cohort. Visual inspection of boxplots comparing case versus control scores was performed and the effect of case status on cognitive domains was further determined through linear regression models, adjusting for age and sex. Socio-demographic and clinical characteristics were tested for between-group differences using the independent sample t-test for continuous data and Pearson’s chi-squared tests for categorical variables. As an additional validation analysis, we investigated the relationship between GPSEDU and educational attainment in our sample using ordinal logistic regression, adjusting for age, sex, the interaction between age and sex, and the first 10 PCs, according to previous work22. All analyses were performed using R statistical software version 3.4.053. An initial examination of the distributions of raw cognitive scores was performed to identify and exclude outliers based on Tukey’s definition (removal of values beyond 3× the interquartile range)54.
GPS analyses of cognitive performance
The effect of GPSEDU on cognitive performance of cases was explored. Blockwise linear regression models were used to estimate the amount of variation in cognitive performance explained by the z-standardized GPSEDU at the four thresholds previously described. For each cognitive outcome, all base models were adjusted for confounding variables measured at the time of testing, i.e., age, age2, sex, in/outpatient status, study center, and PANSS sum scores. Although our participants are chronic patients, duration of illness was considered an important covariate which could confound our results. However, as duration of illness proved to be well correlated with age (r = 0.53), ultimately only age was kept in the models. To guard against population stratification, the first 20 ancestry principal components (PCs) were included in our models and selected for each cognitive outcome tested using backward model selection (p < 0.05)55. The significant PCs were as follows: PCs 12 and 17 for the Verbal digit span (forward task); PC 7 for the DST; PCs 1 and 18 for the MWT-B; PCs 1 and 5 for the VLMT- loss of words after time; and PC 16 for the VLMT- correctly recognized words. No significant PCs were found for the other cognitive outcomes. For each cognitive outcome of interest, we measured the incremental adjusted-R2, that is the gain in the coefficient of determination when the GPSEDU was added as covariate to the regression model for each phenotype (cognition score) on a set of baseline covariates. Multiple testing was corrected for using the False Discovery Rate (FDR) method correcting for the polygenic profiles at all four thresholds and for all phenotypes investigated. Visual inspection of the residuals for each model was performed to be sure the requirement of normally distributed model residuals had been fulfilled.
GPS analyses of schizophrenia and bipolar disorder
Using blockwise linear regression models as described above, we tested whether polygenic scores for schizophrenia and bipolar disorder influenced cognitive outcomes. This was tested for both the GPSSZ and GPSBD separately. We then determined how the genetic risk for schizophrenia and bipolar disorder influenced the effect of GPSEDU on cognitive outcomes. Both scores were included (separately) in those models in which GPSEDU was significantly associated with the cognitive outcome tested.
Additional analyses
Post hoc analyses were performed to determine the robustness of our findings when correcting for diagnosis (bipolar-I disorder, bipolar-II disorder, schizophrenia, or schizoaffective disorder) and medication (number of antipsychotics, antidepressants, mood stabilizers, and tranquilizers at time of assessment). Furthermore, taking into consideration the significant correlation between memory and crystallized intelligence56, we performed a mediation analysis introducing the DST and VLMT (number of correctly recalled words) as covariates in our model testing the association between GPSEDU (pT < 1) and crystallized intelligence. Multicollinearity diagnostics were performed.
Results
A description of socio-demographic variables for participants is presented in Table 1. Seven-hundred and thirty patients with schizophrenia and bipolar disorder were used for analyses. The mean age of these participants was 43.19 years, the majority of which were male. The majority of cases (46.2%) were diagnosed with schizophrenia, 10.0% were schizoaffective, 35.1% were bipolar-I patients and 8.7% were bipolar-II patients. During baseline visits, 47.7% of patients were being treated as day/inpatients.
Table 1.
Cases (n = 730)b | Controls (n = 320)b | Test statistic | Degrees of freedom (df) | p-value | |
---|---|---|---|---|---|
Age at baseline | 43.19 (13.01) | 37.53 (15.83) | 5.62 | 516.11 | <0.001 |
Sex | 24.22 | 1 | <0.001 | ||
Male | 414 (56.7) | 128 (40.0) | |||
Diagnosis | N/A | N/A | N/A | N/A | |
Schizophrenia | 337 (46.2) | ||||
Schizoaffective | 73 (10.0) | ||||
Bipolar-I disorder | 256 (35.1) | ||||
Bipolar-II disorder | 64 (8.7) | ||||
Education a | 154.11 | 6 | <0.001 | ||
0 | 10 (1.4) | 0 (0.0) | |||
1 | 46 (6.3) | 2 (0.6) | |||
2 | 146 (20.0) | 8 (2.5) | |||
3 | 179 (24.5) | 98 (30.6) | |||
4 | 130 (17.8) | 31 (9.7) | |||
5 | 87 (11.9) | 35 (10.9) | |||
6 | 114 (15.6) | 142 (44.4) | |||
Missing | 18 (2.5) | 4 (1.3) | |||
Duration of illness | 12.93 (10.81) | N/A | N/A | N/A | N/A |
Baseline treatment | N/A | N/A | N/A | N/A | |
None | 23 (3.2) | ||||
Outpatient | 355 (48.6) | ||||
Day patient | 38 (5.2) | ||||
Inpatient | 310 (42.5) | ||||
Missing | 4 (0.5) |
aThe PsyCourse study measures status in the German educational system in detail. In order to make the German educational system comparable to English-speaking systems information on specialized schools, high school and professional education in Germany have been combined to form an ordinal educational scale with “6” being the highest level of education obtained
bAge and duration of illness have been reported as mean (standard deviation), while all other categorical variables have been reported as n (%). A t-test was used for comparison of mean age and X2-tests were used for all categorical comparisons
Socio-demographic information of participants
The correlations between cognitive domains were assessed (Supplementary Fig. 1). Boxplots depicting case versus control performance across all cognitive domains are shown in Supplementary Fig. 2. Investigation of linear models to test the effect of case status on cognitive performance, after adjusting for age and sex, showed a significant effect in the direction expected, i.e., a decreased performance for cases (Supplementary Table 1). Educational attainment was significantly associated with GPSEDU in the direction expected (Supplementary Table 2).
Our investigation of the effect of GPSEDU (pT < 1) on cognitive performance in patients resulted in a significant increase in Nagelkerke’s R2 of 1.5% for the verbal digit span (backward; Fig. 1a), 1.9% for the VLMT number of correctly recalled words (Fig. 1b) and 1.1% for crystallized intelligence (Fig. 1c). With more stringent p-value thresholds used, i.e., the inclusion of less SNPs, the change in adjusted-R2 decreased. For the verbal digit span (backward) and the VLMT, the GPSEDU based on the p-value thresholds pT < 0.05, 0.1, and <1 were significant (FDR-adjusted p < 0.05; Supplementary Table 3). The score was significant at all p-value thresholds for crystallized intelligence (FDR-adjusted p < 0.05). The examination of model residuals via quantile–quantile (QQ) plots did not show any extreme deviation from normality (Supplementary Figs. 3–5). Further inspection of model residuals against GPSEDU quartiles showed evidence of increased performance on all three domains with increased GPSEDU scores (Fig. 2). Our results remained robust after correcting for medication (Supplementary Table 4) and diagnosis (Supplementary Table 5). Furthermore, our mediation analysis supports a robust association between GPSEDU and crystallized intelligence that is not mediated by memory parameters (GPSEDU p < 0.05; change in adjusted-R2 = 0.0091).
No significant associations between cognitive outcomes and polygenic scores for schizophrenia or bipolar disorder were observed (Supplementary Table 6). Furthermore, neither risk score influenced the significant effects of GPSEDU on the three cognitive domains reported above (Supplementary Table 7). Multicollinearity diagnostics showed no issues of collinearity in our regression analysis (variance inflation factor <5 for all independent variables).
Discussion
Our study aimed to identify the influence of GPSEDU on several cognitive domains in a transdiagnostic cohort of psychiatric patients. Confirming results of previous research, patients with bipolar disorder and schizophrenia in the PsyCourse cohort performed worse on tests of neurocognitive functioning in comparison to nonclinical controls. In patients, we observed a significant improvement in prediction of cognitive performance with inclusion of GPSEDU for the backward verbal digit span, VLMT (correctly recalled words), and for crystallized intelligence. These findings confirm the ability of GPSEDU to explain variability in linguistic cognitive performance related to working memory and learning in patients with known cognitive deficits. Furthermore, our findings show that cognitive performance measured for these domains were associated with the genetic underpinnings of GPSEDU and not confounded by or associated with GPSSZ or GPSBD.
Previous studies have investigated the association between cognitive performance and GPSEDU using summary statistics from an earlier GWAS on educational attainment by Okbay et al.24. Our findings compliment earlier evidence supporting an association between cognitive performance and educational attainment, but not schizophrenia genetic risk, in clinical patients. For example, a study by Shafee et al. compared the effect of GPSSZ on three cognitive phenotypes i.e. general cognitive function, premorbid intellectual potential, and years of education completed30. The authors found that among healthy individuals, GPSSZ was significantly associated with lower general cognitive functioning, however, found no association between GPSSZ with any cognitive phenotype in patients with psychosis. Furthermore, the authors found significant positive correlations between GPSEDU and both educational attainment and premorbid intelligence in patients with and without psychosis. Another study by Bansal et al. showed GPSEDU could predict 2.09% of variance in premorbid IQ in a large schizophrenia sample29. Our findings support earlier suggestions that different cognitive phenotypes vary in their etiologic relationship with schizophrenia and in their genetic overlap with educational attainment30. Furthermore, our findings are in line with evidence from the first educational attainment GWAS of 126,559 individuals which identified variants which implicated genes (including BSN, GBX2, LRRN2, and PIK3C2B) linked to processes such as learning and long-term memory27. These findings are especially interesting given that learning and working memory are among some of the most impaired cognitive process for patients with psychiatric disorders57.
While a polygenic score for educational attainment in the general population explained 7–10% of variance in cognitive performance, the score explained at most ~2% in our transdiagnostic cohort22. It is difficult to determine whether the smaller effect in our cohort was the result of a different phenotype being measured, i.e., specific cognitive domains and not a composite score, or whether this might reflect the cognitive performance of this unique, transdiagnostic sample being related to other complex genetic-environmental factors. Clearly, future investigations looking at other measures of cognition in large cohorts are warranted. Confounding variables such as acute symptoms may also contribute to the lack of variability explained in this case, although we have tried to capture this by controlling for current in/outpatient status and symptom severity. Furthermore, although based on big samples, polygenic scores “may not be sufficiently powerful to capture signs of disrupted neurodevelopment” in these patients as they exclude rare copy number variations and deleterious exonic mutations which may have important consequences52.
On both a phenotypic and genetic level, intelligence has been associated with psychiatric disorders. For example, individuals with a level of intelligence one standard deviation below the mean, have ~60% higher risk of hospitalization for schizophrenia58. There is also evidence supporting an association between poorer school performance and higher risk for schizophrenia52. In addition, several longitudinal studies have linked deficits in premorbid IQ with subsequent schizophrenia development, which was also shown for mood disorders52,59. The evidence, however, linking intelligence and affective disorders has been more inconsistent. For example, bipolar disorder has been associated with higher childhood IQ and an increased genetic risk of bipolar disorder has been associated with creativity and higher education60–62. However, no such associations have been reported by studies of adolescent or adult IQ60. Nevertheless, there are known genetic variants influencing both intelligence and psychiatric disorders which, in part, explain the phenotypic link between intelligence and these disorders58.
We investigated the potential influence of GPSSZ and GPSBD on cognitive performance. These relationships seem to be complex and while the genetic overlap between schizophrenia susceptibility with cognitive performance has been widely investigated in the literature with conflicting findings, less has been done in bipolar disorder8,63–65. The lack of an association observed between either the GPSSZ or GPSBD with cognitive performance in our study emphasizes several issues inherent to these types of investigations. The first is that GPSEDU is based on a much larger discovery sample than GPSSZ and GPSBD, meaning GPSEDU had higher statistical power to capture smaller effect sizes and more accurate estimates for single SNPs of which the score is based on. Presuming the most optimistic estimate for variance explained in cognitive performance by the GPSSZ of 1.6% that has previously been reported8, a sample of ~500 participants would be required to drive the effect of schizophrenia genetic risk scores on cognitive performance. However, given a more conservative estimate of 0.3%66 variance explained, a sample size of over 2 600 participants would be required, suggesting that power may indeed be an issue in our study (Supplementary Fig. 6). This is also true with regards to GPSBD in which genetic effects are likely to be at least as subtle. This again highlights the value of analyzing a proxy-phenotype such as educational attainment.
The second issue is in relation to the cognitive domains that were analyzed. As studies often use different cognitive tests from the wide variety that are available, it could be that the genetic risk of schizophrenia and bipolar disorder are more closely linked to domains that went unmeasured in our study. Perhaps if we had used a composite score across all domains or different neurocognitive tests in general, a significant effect would have been observed. Unfortunately, due to the longitudinal nature of our study which led to missing data across the different cognitive outcomes tested, a composite score analysis with adequate power was not feasible.
Our findings should be considered in light of a few limitations. The first is that our patients represent a chronic sample of heterogeneously treated patients. As these patients have been prescribed a wide range of medications at different dosages, correcting for the possible influence of medication is not an easy task. Not knowing how different drugs might interact with or influence cognition throughout the course of the disorder is a limitation that always must be considered in psychiatric research, and this problem has yet to have a perfect solution. A second limitation of our study is generalizability considering we investigated raw scores for several cognitive domains. As mentioned above, one of the major issues in the field at this time is the complexity in measuring this phenotype and with a plethora of tests that can be used, it is difficult to say how generalizable our findings are to other cognitive tests within the same cognitive domain in different cohorts. For example, while crystallized intelligence was measured, our study failed to consider fluid intelligence which has a higher heritability component than crystallized intelligence67. It is also important to note that while executive functions are related, they are also diverse68. While the TMT used in our study is a measure of task switching, other executive functions like the updating process of working memory and inhibition should be explored. Lastly, we must acknowledge that our study has only assessed linguistic memory and not visuospatial memory. As these are two unique types of memory69, future investigations are warranted to determine how the two might differ in association with GPSEDU.
Although remarkable heterogeneity of cognitive deficits exists among individuals with psychiatric disorders, in general these deficits are, by a moderate degree, less severe in chronic bipolar patients in comparison to chronic schizophrenia patients. Furthermore, the trajectories of these impairments are quite different70. Often, cognitive deficits are apparent before the onset of disease in individuals with schizophrenia71. Approximately 70% of bipolar patients exhibit cognitive deficits, especially related to verbal memory and attention57, which often manifest in young adults60. Despite these known differences for bipolar disorder and schizophrenia, we did not observe a significant effect of diagnosis on the effect of GPSEDU related to cognitive performance. These diagnostic differences were most likely captured by the PANSS sum scores included in our models, which was highly significant. Evidence also supports an increase in the heritable component of intelligence with age72. Considering this knowledge, future studies, longitudinal in design, would be highly beneficial. It would be intriguing to see how the polygenic score for educational attainment can explain variability in cognitive performance throughout the course of the disorder. While our sample consisted of chronic mid-aged patients in which cognitive performance was rather stable across visits, it would be valuable to investigate younger cohorts of patients, even before the onset of disease, to determine how instability in cognitive performance throughout the disease course might influence the association between GPSEDU and cognition. This would help determine at which points the underlying genetic components are most influential and help identify at which periods environmental influences might be more prominent in determining cognitive abilities.
Conclusions
Identifying a genetic component related to distinct neurocognitive profiles has potential to identify a more burdened subgroup of patients that in turn might be at risk for lower levels of functioning and poor social outcomes. This sort of information targets patients for more personalized interventions73,74. Here we have explained only a small fraction of variance in cognitive performance in patients with psychiatric disorders using the genetic variants associated with educational attainment. These findings highlight the importance of other uncaptured environmental exposures that have major influences on cognitive abilities and ultimately levels of functioning in these patients. Future studies, over the course of the disorder, would be informative to determine how this association changes over time, and at which periods environment may play the most influential role60. Furthermore, future studies should factor in the complex pleiotropic relationships between these traits to generate enhanced polygenic scores to further clarify their genetic architecture75. Moreover, hypothesis-based polygenic scores could help uncover biological pathways related to cognitive performance.
Supplementary information
Acknowledgements
Thomas G. Schulze and Peter Falkai are supported by the German Research Foundation (Deutsche Forschungsgemeinschaft; DFG) within the framework of the projects www.kfo241.de and www.PsyCourse.de (SCHU 1603/4-1, 5-1, 7-1; FA241/16-1). The genotyping was in part funded by the German Federal Ministry of Education and Research (BMBF) through the Integrated Network IntegraMent (Integrated Understanding of Causes and Mechanisms in Mental Disorders), under the auspices of the e:Med Program with grants awarded to Thomas G. Schulze (01ZX1614K), Marcella Rietschel (01ZX1614G), and Markus M. Nöthen (01ZX1614A). Thomas G. Schulze received additional support from the German Federal Ministry of Education and Research (BMBF) within the framework of the BipoLife network (01EE1404H) and the Dr. Lisa Oehler Foundation (Kassel, Germany). Sergi Papiol was supported by a 2016 NARSAD Young Investigator Grant (25015) from the Brain and Behavior Research Foundation. Marcella Rietschel and Markus M. Nöthen received funding from the German Research Foundation (DFG, grant FOR2107; RI 908/11-2 to Marcella Rietschel, NO 246/10-2 to Markus M. Nöthen). Franziska Degenhardt received support from the BONFOR Programme of the University of Bonn, Germany. Jens Wiltfang is supported by an Ilídio Pinho professorship and iBiMED (UID/BIM/04501/2013), at the University of Aveiro. Udo Dannlowski was funded by the German Research Foundation (DFG, grant FOR2107 DA1151/5‐1; SFB‐TRR58, Project C09) and the Interdisciplinary Center for Clinical Research (IZKF) of the medical faculty of Münster (grant Dan3/012/17 to Udo Dannlowski). Volker Arolt was funded by the European Union FP-7 Research Grant MOODINFLAME. We would like to express our profound gratitude to all study participants without whom this work would not have been possible.
Conflict of interest
Jens Reimer received honoraria from Otsuka-Lundbeck for participation in a speakers’ bureau. The remaining authors declare that they have no conflict of interest.
Footnotes
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
These authors contributed equally: Thomas G. Schulze, Sergi Papiol
Supplementary information
Supplementary Information accompanies this paper at (10.1038/s41398-019-0547-x).
References
- 1.Solé B, et al. Cognitive impairment in bipolar disorder: treatment and prevention strategies. Int. J. Neuropsychopharmacol. 2017;20:670–680. doi: 10.1093/ijnp/pyx032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sanches M, Bauer IE, Galvez JF, Zunta-Soares GB, Soares JC. The management of cognitive impairment in bipolar disorder: current status and perspectives. Am. J. Ther. 2015;22:477–486. doi: 10.1097/MJT.0000000000000120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Tripathi A, Kar SK, Shukla R. Cognitive Deficits in Schizophrenia: Understanding the Biological Correlates and Remediation Strategies. Clin. Psychopharmacol. Neurosci. 2018;16:7–17. doi: 10.9758/cpn.2018.16.1.7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Goff DC, Hill M, Barch D. The treatment of cognitive impairment in schizophrenia. Pharmacol. Biochem. Behav. 2011;99:245–253. doi: 10.1016/j.pbb.2010.11.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Green MF. What are the functional consequences of neurocognitive deficits in schizophrenia? Am. J. Psychiatry. 1996;153:321–330. doi: 10.1176/ajp.153.3.321. [DOI] [PubMed] [Google Scholar]
- 6.Bowie CR, Harvey PD. Cognitive deficits and functional outcome in schizophrenia. Neuropsychiatr. Dis. Treat. 2006;2:531–536. doi: 10.2147/nedt.2006.2.4.531. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Davies G, et al. Genome-wide association studies establish that human intelligence is highly heritable and polygenic. Mol. Psychiatry. 2011;16:996–1005. doi: 10.1038/mp.2011.85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lencz T, et al. Molecular genetic evidence for overlap between general cognitive ability and risk for schizophrenia: a report from the Cognitive Genomics consorTium (COGENT) Mol. Psychiatry. 2014;19:168–174. doi: 10.1038/mp.2013.166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Benyamin B, et al. Childhood intelligence is heritable, highly polygenic and associated with FNBP1L. Mol. Psychiatry. 2014;19:253–258. doi: 10.1038/mp.2012.184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kirkpatrick RM, McGue M, Iacono WG, Miller MB, Basu S. Results of a “GWAS plus:” general cognitive ability is substantially heritable and massively polygenic. PLoS ONE. 2014;9:e112390. doi: 10.1371/journal.pone.0112390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Davies G, et al. Study of 300,486 individuals identifies 148 independent genetic loci influencing general cognitive function. Nat. Commun. 2018;9:2098. doi: 10.1038/s41467-018-04362-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Deary IJ. The stability of intelligence from childhood to old age. Curr. Dir. Psychol. Sci. 2014;23:239–245. doi: 10.1177/0963721414536905. [DOI] [Google Scholar]
- 13.Deary IJ, et al. Genetic contributions to stability and change in intelligence from childhood to old age. Nature. 2012;482:212–215. doi: 10.1038/nature10781. [DOI] [PubMed] [Google Scholar]
- 14.Lyons MJ, et al. A longitudinal twin study of general cognitive ability over four decades. Dev. Psychol. 2017;53:1170–1177. doi: 10.1037/dev0000303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Snitz BE, Macdonald AW, 3rd, Carter CS. Cognitive deficits in unaffected first-degree relatives of schizophrenia patients: a meta-analytic review of putative endophenotypes. Schizophr. Bull. 2006;32:179–194. doi: 10.1093/schbul/sbi048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Bora E, Yucel M, Pantelis C. Cognitive endophenotypes of bipolar disorder: a meta-analysis of neuropsychological deficits in euthymic patients and their first-degree relatives. J. Affect. Disord. 2009;113:1–20. doi: 10.1016/j.jad.2008.06.009. [DOI] [PubMed] [Google Scholar]
- 17.Gottesman II, Gould TD. The endophenotype concept in psychiatry: etymology and strategic intentions. Am. J. Psychiatry. 2003;160:636–645. doi: 10.1176/appi.ajp.160.4.636. [DOI] [PubMed] [Google Scholar]
- 18.Plomin R, von Stumm S. The new genetics of intelligence. Nat. Rev. Genet. 2018;19:148–159. doi: 10.1038/nrg.2017.104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Deary IJ, Johnson W, Houlihan LM. Genetic foundations of human intelligence. Hum. Genet. 2009;126:215–232. doi: 10.1007/s00439-009-0655-4. [DOI] [PubMed] [Google Scholar]
- 20.Trampush JW, et al. GWAS meta-analysis reveals novel loci and genetic correlates for general cognitive function: a report from the COGENT consortium. Mol. Psychiatry. 2017;22:336–345. doi: 10.1038/mp.2016.244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Savage JE, et al. Genome-wide association meta-analysis in 269,867 individuals identifies new genetic and functional links to intelligence. Nat. Genet. 2018;50:912–919. doi: 10.1038/s41588-018-0152-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lee JJ, et al. Gene discovery and polygenic prediction from a genome-wide association study of educational attainment in 1.1 million individuals. Nat. Genet. 2018;50:1112–1121. doi: 10.1038/s41588-018-0147-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Davies G, et al. Genome-wide association study of cognitive functions and educational attainment in UK Biobank (N=112 151) Mol. Psychiatry. 2016;21:758–767. doi: 10.1038/mp.2016.45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Okbay A, et al. Genome-wide association study identifies 74 loci associated with educational attainment. Nature. 2016;533:539–542. doi: 10.1038/nature17671. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Rietveld CA, et al. Common genetic variants associated with cognitive performance identified using the proxy-phenotype method. Proc. Natl Acad. Sci. USA. 2014;111:13790–13794. doi: 10.1073/pnas.1404623111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Trampush JW, et al. Independent evidence for an association between general cognitive ability and a genetic locus for educational attainment. Am. J. Med. Genet. B Neuropsychiatr. Genet. 2015;168B:363–373. doi: 10.1002/ajmg.b.32319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Rietveld CA, et al. GWAS of 126,559 individuals identifies genetic variants associated with educational attainment. Science. 2013;340:1467–1471. doi: 10.1126/science.1235488. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Elliott Maxwell L, Belsky Daniel W, Anderson Kevin, Corcoran David L, Ge Tian, Knodt Annchen, Prinz Joseph A, Sugden Karen, Williams Benjamin, Ireland David, Poulton Richie, Caspi Avshalom, Holmes Avram, Moffitt Terrie, Hariri Ahmad R. A Polygenic Score for Higher Educational Attainment is Associated with Larger Brains. Cerebral Cortex. 2018;29(8):3496–3504. doi: 10.1093/cercor/bhy219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Bansal V, et al. Genome-wide association study results for educational attainment aid in identifying genetic heterogeneity of schizophrenia. Nat. Commun. 2018;9:3078. doi: 10.1038/s41467-018-05510-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Shafee R, et al. Polygenic risk for schizophrenia and measured domains of cognition in individuals with psychosis and controls. Transl. Psychiatry. 2018;8:78. doi: 10.1038/s41398-018-0124-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Budde M, et al. A longitudinal approach to biological psychiatric research: The PsyCourse study. Am. J. Med. Genet. B Neuropsychiatr. Genet. 2019;180:89–102. doi: 10.1002/ajmg.b.32639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Forstner AJ, et al. Identification of shared risk loci and pathways for bipolar disorder and schizophrenia. PLoS ONE. 2017;12:e0171595. doi: 10.1371/journal.pone.0171595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Anttila V, et al. Analysis of shared heritability in common disorders of the brain. Science. 2018;360:6395. doi: 10.1126/science.aap8757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Lichtenstein P, et al. Common genetic determinants of schizophrenia and bipolar disorder in Swedish families: a population-based study. Lancet. 2009;373:234–239. doi: 10.1016/S0140-6736(09)60072-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Lee, P. H. et al. Genome wide meta-analysis identifies genomic relationships, novel loci, and pleiotropic mechanisms across eight psychiatric disorders. bioRxiv 2019: 528117. [DOI] [PMC free article] [PubMed]
- 36.Hill WD, et al. A combined analysis of genetically correlated traits identifies 187 loci and a role for neurogenesis and myelination in intelligence. Mol. Psychiatry. 2018;24:169–181. doi: 10.1038/s41380-017-0001-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.American Psychiatric Association. Diagnostic and statistical manual of mental disorders (4th ed): Washington, DC, 2002.
- 38.Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 1987;13:261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
- 39.Brown RE. Hebb and Cattell: The Genesis of the Theory of Fluid and Crystallized. Intell. Front. Hum. Neurosci. 2016;10:606. doi: 10.3389/fnhum.2016.00606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Lehrl, S. Mehrfachwahl-Wortschatz-Intelligenztest (MWT-B). Spitta Verlag: Balingen, Germany, 2005.
- 41.Partington JE, Leiter RG. Partington’s Pathways Test. Psychol. Serv. Cent. J. 1949;1:11–20. [Google Scholar]
- 42.Strauss E, Sherman EM, Spreen O. A compendium of neuropsychological tests: Administration, norms, and commentary. 3 edn. New York: Oxford University Press; 2006. [Google Scholar]
- 43.Lamberty GJ, et al. Derived Trail Making Test indices: A preliminary report. Neuropsychiatry Neuropsychol. Behav. Neurol. 1994;7:230–234. [Google Scholar]
- 44.Aster, M., Neubauer, A. & Horn, R. Wechsler Intelligenztest für Erwachsene. Wechsler Intelligence Test for Adults (German revision and adaptation of the WAIS-III of David Wechsler). Harcourt Test Services: Frankfurt, Germany, 2006.
- 45.Wechsler, D. Manual for the Wechsler Adult Intelligence Scale. Psychological Corp.: New York, 1955, vi, 110-vi, 110pp.
- 46.Lezak, M. D. Neuropsychological Assessment (2nd ed.). Oxford University Press: New York, 1983.
- 47.Helmstaedter, C., Lendt, M. & Lux, S. Verbaler Lern- und Merkfähigkeitstest (VLMT). Beltz: Göttingen, Germany, 2001.
- 48.Kalman JL, et al. Investigating polygenic burden in age at disease onset in bipolar disorder: Findings from an international multicentric study. Bipolar Disord. 2019;21:68–75. doi: 10.1111/bdi.12659. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Chang CC, et al. Second-generation PLINK: rising to the challenge of larger and richer datasets. Gigascience. 2015;4:7. doi: 10.1186/s13742-015-0047-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Pardiñas AF, et al. Common schizophrenia alleles are enriched in mutation-intolerant genes and in regions under strong background selection. Nat. Genet. 2018;50:381–389. doi: 10.1038/s41588-018-0059-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Stahl EA, et al. Genome-wide association study identifies 30 loci associated with bipolar disorder. Nat. Genet. 2019;51:793–803. doi: 10.1038/s41588-019-0397-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Sørensen HJ, et al. Polygenic risk scores, school achievement, and risk for schizophrenia: a danish population-based study. Biol. Psychiatry. 2018;84:684–691. doi: 10.1016/j.biopsych.2018.04.012. [DOI] [PubMed] [Google Scholar]
- 53.R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing: Vienna, Austria.
- 54.Tukey, J. W. Exploratory data analysis. Addison-Wesley Pub. Co.: Reading, Mass., 1977.
- 55.Price AL, et al. Principal components analysis corrects for stratification in genome-wide association studies. Nat. Genet. 2006;38:904–909. doi: 10.1038/ng1847. [DOI] [PubMed] [Google Scholar]
- 56.Unsworth N. On the division of working memory and long-term memory and their relation to intelligence: a latent variable approach. Acta Psychol. 2010;134:16–28. doi: 10.1016/j.actpsy.2009.11.010. [DOI] [PubMed] [Google Scholar]
- 57.Barbosa IG, et al. Predictors of cognitive performance in bipolar disorder: the role of educational degree and inflammatory markers. J. Psychiatr. Res. 2018;106:31–37. doi: 10.1016/j.jpsychires.2018.09.003. [DOI] [PubMed] [Google Scholar]
- 58.Hill WD, Harris SE, Deary IJ. What genome-wide association studies reveal about the association between intelligence and mental health. Curr. Opin. Psychol. 2018;27:25–30. doi: 10.1016/j.copsyc.2018.07.007. [DOI] [PubMed] [Google Scholar]
- 59.Peyrot WJ, et al. The association between lower educational attainment and depression owing to shared genetic effects? Results in ~25,000 subjects. Mol. Psychiatry. 2015;20:735–743. doi: 10.1038/mp.2015.50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Mistry S, Harrison JR, Smith DJ, Escott-Price V, Zammit S. The use of polygenic risk scores to identify phenotypes associated with genetic risk of bipolar disorder and depression: a systematic review. J. Affect. Disord. 2018;234:148–155. doi: 10.1016/j.jad.2018.02.005. [DOI] [PubMed] [Google Scholar]
- 61.Vreeker A, et al. High educational performance is a distinctive feature of bipolar disorder: a study on cognition in bipolar disorder, schizophrenia patients, relatives and controls. Psychol. Med. 2016;46:807–818. doi: 10.1017/S0033291715002299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.MacCabe JH, et al. Excellent school performance at age 16 and risk of adult bipolar disorder: national cohort study. Br. J. Psychiatry. 2010;196:109–115. doi: 10.1192/bjp.bp.108.060368. [DOI] [PubMed] [Google Scholar]
- 63.Schaupp S, Schulze T, Budde M. Let’s talk about the association between schizophrenia polygenic risk scores and cognition in patients and the general population: a review. J. Psychiatry Brain Sci. 2018;3:12. [Google Scholar]
- 64.Mistry S, Harrison JR, Smith DJ, Escott-Price V, Zammit S. The use of polygenic risk scores to identify phenotypes associated with genetic risk of schizophrenia: Systematic review. Schizophr. Res. 2017;S0920-9964:30665–30665. doi: 10.1016/j.schres.2017.10.037. [DOI] [PubMed] [Google Scholar]
- 65.Ranlund S, et al. A polygenic risk score analysis of psychosis endophenotypes across brain functional, structural, and cognitive domains. Am. J. Med. Genet. B Neuropsychiatr. Genet. 2018;177:21–34. doi: 10.1002/ajmg.b.32581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Riglin L, et al. Schizophrenia risk alleles and neurodevelopmental outcomes in childhood: a population-based cohort study. Lancet Psychiatry. 2017;4:57–62. doi: 10.1016/S2215-0366(16)30406-0. [DOI] [PubMed] [Google Scholar]
- 67.Cattell RB, Schuberger JM, Ahern FM, Kameoka V. The heritability of fluid and crystallized intelligences: By the mava design and oses analysis. Aust. J. Psychol. 1981;33:355–374. doi: 10.1080/00049538108254704. [DOI] [Google Scholar]
- 68.Miyake A, et al. The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: a latent variable analysis. Cogn. Psychol. 2000;41:49–100. doi: 10.1006/cogp.1999.0734. [DOI] [PubMed] [Google Scholar]
- 69.Baddeley A. Working memory: looking back and looking forward. Nat. Rev. Neurosci. 2003;4:829–839. doi: 10.1038/nrn1201. [DOI] [PubMed] [Google Scholar]
- 70.Bortolato B, Miskowiak KW, Köhler CA, Vieta E, Carvalho AF. Cognitive dysfunction in bipolar disorder and schizophrenia: a systematic review of meta-analyses. Neuropsychiatr. Dis. Treat. 2015;11:3111–3125. doi: 10.2147/NDT.S76700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Reichenberg A, et al. Static and dynamic cognitive deficits in childhood preceding adult schizophrenia: a 30-year study. Am. J. Psychiatry. 2010;167:160–169. doi: 10.1176/appi.ajp.2009.09040574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Plomin R, Deary IJ. Genetics and intelligence differences: five special findings. Mol. Psychiatry. 2014;20:98–108. doi: 10.1038/mp.2014.105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Tickell AM, et al. Neurocognitive clusters: a pilot study of young people with affective disorders in an inpatient facility. J. Affect. Disord. 2019;242:80–86. doi: 10.1016/j.jad.2018.08.045. [DOI] [PubMed] [Google Scholar]
- 74.Kapur S, Phillips AG, Insel TR. Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Mol. Psychiatry. 2012;17:1174–1179. doi: 10.1038/mp.2012.105. [DOI] [PubMed] [Google Scholar]
- 75.Lam, M. et al. Pleiotropic meta-analysis of cognition, education, and schizophrenia differentiates roles of early neurodevelopmental and adult synaptic pathways. bioRxiv 2019: 519967. [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.