Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Jan 9.
Published in final edited form as: Schizophr Res. 2005 Aug 26;80(0):10.1016/j.schres.2005.07.028. doi: 10.1016/j.schres.2005.07.028

Olfactory deficits, cognition and negative symptoms in early onset psychosis

Cheryl Corcoran 1,*, Agnes Whitaker 1, Eliza Coleman 1, Jane Fried 1, Judith Feldman 1, Nora Goudsmit 1, Dolores Malaspina 1
PMCID: PMC3886553  NIHMSID: NIHMS542752  PMID: 16125904

Abstract

Background

Smell identification deficits (SID) are common in adult schizophrenia, where they are associated with negative symptoms and lower intelligence. However, smell identification has not been examined in adolescents with early onset psychosis, wherein diagnosis is often obscure, and there are few prognostic predictors.

Method

We examined smell identification, diagnosis, neuropsychological performance and symptoms in 26 well characterized adolescents with early onset psychosis, age 11–17 years.

Results

SID existed in the sample and were more common in patients with schizophrenia and psychotic depression than in patients with psychosis NOS and bipolar disorder. As in adults, SID were significantly associated with greater negative symptoms and lower verbal IQ. However, the associations of verbal IQ (and other verbal tasks) to smell identification in this pediatric sample were explained by the relation of both of these types of variables to negative symptoms.

Conclusions

SID existed across this sample of youths with psychotic disorder, and were specifically related to typical characteristics of schizophrenia, such as negative symptoms and lower intelligence, but not to features of bipolar disorder, such as grandiosity. SID is a characteristic of early onset psychosis that may be useful for prognostic purposes.

Keywords: Olfaction, Psychosis, Negative symptoms, Cognition, Schizophrenia and children

1. Introduction

In adults, poor smell identification characterizes schizophrenia (Houlihan et al., 1994; Hurwitz et al., 1988; Moberg et al., 1999) and other psychotic disorders (Striebel et al., 1999), and is associated with typical characteristics of schizophrenia such as negative symptoms and low IQ. As smell identification ability shows developmental maturation into adolescence, it is not clear whether poor smell identification would also characterize psychotic disorders in children and adolescents, or be related to clinical characteristics. Hence, we aimed to determine if smell identification deficits (SID) existed in a pediatric sample with psychosis, and whether any SID identified would be related to negative symptoms (Brewer et al., 1996) and low IQ, as has been seen in adults.

Smell identification deficits (SID) are well documented in a number of neurodegenerative disorders, but among psychiatric disorders, they are observed in illnesses involving the orbitofrontal cortex, including mild deficits in obsessive compulsive disorder (Barnett et al., 1999) and post–traumatic stress disorder (Vasterling et al., 2000), and more profound deficits in Asperger’s syndrome (Suzuki et al., 2003) and schizophrenia (Houlihan et al., 1994; Hurwitz et al., 1988; Moberg et al., 1999). In schizophrenia, SID exist irrespective of clinical state, schizophrenia subtype, age, phase of illness, ethnicity, socioeconomic status, cigarette smoking or medication treatment (Coleman et al., 2002; Moberg et al., 1999). SID in schizophrenia are associated with negative symptoms (Brewer et al., 1996), and with the deficit syndrome, a pervasive and enduring pattern of negative symptoms (Malaspina et al., 2002; Seckinger et al., 2004), in particular with diminished social drive (Malaspina and Coleman, 2003). However, these findings have been reported for adult patients with schizophrenia and it is unknown if similar associations would be observed in early onset (age <18) schizophrenia-like psychosis (McClellan and Werry, 1992), which would suggest a common pathophysiology.

While smell identification ability has not yet been examined in children with psychosis, such studies have been conducted in adolescents with Down’s syndrome and mental retardation (Hemdal et al., 1993; McKeown et al., 1996; Nijjar and Murphy, 2002), in incarcerated male juveniles (Roussy and Toupin, 2000), and in blind children (Rosenbluth et al., 2000). None of these pediatric samples showed SID although olfactory functioning in adolescents has implications for the neurobiology of each of these conditions. For example, SID is well-described in adults with Down’s syndrome but is absent in adolescents with this disorder, and therefore may be related to the neurodegenerative processes associated with Alzheimer-type pathology found with Down’s syndrome in midlife, rather than neurodevelopmental processes (Murphy and Jinich, 1996). Likewise, the absence of SID in incarcerated male juveniles suggests that SID in adults with antisocial behavior may be due to confounding factors, such as traumatic brain injury or substance use (Roussy and Toupin, 2000). Children with congenital or early onset blindness were significantly more proficient in smell identification, consistent with a compensatory neural process or cognitive strategy (Rosenbluth et al., 2000).

Children have comparable olfactory sensitivity to adults, but differ in their odor memory and naming capacity (Cain et al., 1995; Lehrner et al., 1999) owing in part to semantic factors such as linguistic capability and familiarity with odor names. Smell identification tests that use identifying photographs rather than words for common odorants (baby powder, bubble gum, candy cane, fish and orange) showed that children could reach a plateau of correct responses of about 90% by the age of six (Richman et al., 1992). Available sex- and age-adjusted norms show improvement in performance (percent correct identifications) in verbal identification of odors with increasing age of the child (Doty et al., 1984).

In this study, we investigated the relation of smell identification to demographic, clinical and neuropsychological features in children and adolescents with psychotic disorders. We hypothesized that as in adults, pediatric patients with schizophrenia-like psychosis would have SID that are, in turn, related to negative symptoms and neuropsychological deficits, and possibly diagnosis.

2. Materials and methods

2.1. Subjects

The study sample included children and adolescents enrolled in a study of early onset psychotic disorders at the Children’s Day Unit, a clinical research facility at New York State Psychiatric Institute, which functions as a day treatment program. The study was approved by the New York State Psychiatric Institute Institutional Review Board, and all participants gave assent to participate and their parents provided informed consent. All participants were of school age (11–17 years) and had exhibited at least one positive symptom (delusions, hallucinations, formal thought disorder or bizarre or disorganized behavior) for at least 1 week in the 6 months prior to referral. Exclusion criteria were: (1) comorbid disorder causing cognitive impairment; (2) known neurological disorder (i.e. seizure disorder or traumatic brain injury); (3) diagnosis of psychoactive substance abuse or dependence; (4) medical illness severe enough to preclude participation; and (5) severe mental retardation (IQ <55).

Consensus best estimate diagnoses (Leckman et al., 1982) were made by an experienced child psychiatrist (AW) who reviewed all available information, including diagnostic interview, data from at least 4 weeks of staff observation, chart review and case conference, while blind to scores on the pediatric assessments of specific neuropsychological function, including smell identification.

Clinical features described included the age at first positive symptoms, number of previous hospitalizations, and current global assessment of functioning (GAF). Pharmacological assessments included current medication status, and the chlorpromazine dose equivalents of any antipsychotic prescribed. Present exposure to anticholinergic, antidepressant or anxiolytic agents was recorded. Handedness was determined using either a hand preference demonstration test or the Annett Handedness Questionnaire (Annett, 1970), which was administered by a psychiatric nurse. Of note, patients were not screened for facial or nasal fractures, and did not have any formal tests of threshold for odor detection.

2.2. Smell identification

Smell identification was assessed using the University of Pennsylvania Smell Identification Test (UPSIT) (Doty et al., 1984), a standardized forced choice scratch and sniff test consisting of four test booklets with 10 items each. To optimize performance, the UPSIT was directly supervised by a member of the research staff, who scratched the scent-impregnated areas and read aloud the four possible answers for each item to the subject. If the patient could not identify an odor, the examiner rescratched the area and restated the four possible choices.

UPSIT data were summarized as percentile scores and as a dichotomous variable (normosmia vs. microsmia), based on published age- and sex-matched norms (Doty et al., 1984). The threshold scores for normosmia for ages 10–14 are 32 for both boys and girls. For ages 15–19, threshold scores are 34 for boys and 35 for girls. Scores below these thresholds for each age group define microsmia, which identifies a significant impairment in smell identification that interferes with functioning. Very low scores on the UPSIT indicate anosmia, the inability to detect odors. For purposes of this study, SID was defined as UPSIT scores below the threshold for microsmia.

2.3. Symptom assessment

Symptoms were assessed with the Positive and Negative Symptom Scale (PANSS) (Kay et al., 1987). The psychometric properties of the PANSS have been well established, including in children and adolescents (Kay et al., 1987, 1988; Fields et al., 1994). Positive and negative symptom scores were calculated. In addition, a composite symptom score was determined as the difference between positive and negative symptom scores. Negative composite scores are considered to constitute a “negative syndrome” and positive composite scores are considered to constitute a “positive syndrome”.

2.4. Neuropsychological function

IQ was assessed using the Wechsler Intelligence Scale for Children-Revised (WISC-R) (Wechsler, 1981). Other neuropsychiatric measures included the Wide Range Achievement Test-Revised (WRAT-R)(Jastak and Jastak, 1978), Ravens Progressive Matrices (Court and Raven, 1995), FAS and Animal Naming (Miller, 1984), Stroop (Jensen and Rohwer, 1966), and Trails A and B (Reitan, 1986).

2.5. Data analysis

First, percentile UPSIT scores were determined for each participant using published age- and sex-matched norms, and the mean percentile score and the proportion with microsmia were calculated both for the sample as a whole and for individual diagnostic groups. We explored any potential association of smell identification with demographic and clinical variables. Associations between continuous measures were examined with Pearson correlations, those between categorical and continuous measures with the t- or F-test, and those between two or more categorical measures with the Pearson chi-square. To assess the relation of smell identification to antipsychotic exposure, a dichotomous variable (“ever used” vs. “never used”) was created from charted history of medication use; additionally, chlorpromazine equivalents of current antipsychotic medication dosages were calculated.

The main hypotheses were that smell identification deficits would be associated with negative symptoms and IQ; an exploratory hypothesis was that SID may be related to diagnosis, specifically schizophrenia. Diagnostic groups were described in terms of demographics, symptoms, cognition and smell identification ability, and Pearson chi-square was used to determine any significant difference in rates of microsmia. Sample sizes for various assessments differ slightly due to missing data and scheduling conflicts. For hypothesized associations, an alpha level of 0.05 was used to establish results as significant, whereas Bonferroni correction was employed to assess associations that were not hypothesized. Post hoc analyses were done to examine relations of specific PANSS items and discrete cognitive tests to UPSIT percentile scores. Fisher exact test was used to evaluate any relationship between type of syndrome (positive vs. negative) and existence of microsmia vs. normosmia.

Partial correlations were used to examine the relation of negative symptoms to UPSIT scores, controlling for IQ, and the relation of IQ to UPSIT scores, controlling for negative symptoms. They were also done to test the relationship between smell identification and other cognitive tests, once negative symptoms were accounted for.

P values between 0.05 and 0.10 are reported for partial correlation analyses.

3. Results

3.1. Prevalence of SID in this sample

Participants included 26 children and adolescents (16 males and 10 females), mean age (S.D.) 15.0 (1.8) years. Nineteen participants were Hispanic, 4 were African-American and 3 Caucasian. Thirteen participants had Spanish as their native language, although all participants spoke English fluently. Half of the participants came from families that relied on public assistance. The mean age of onset (S.D.) of first positive symptoms was 13.2 (3.2) years (range 5–17) and mean (S.D.) global assessment of functioning was 51.2 (8.5). Sixteen participants (61.5%) had a previous hospitalization. Half of the participants were right-handed and half showed mixed handedness. The mean (S.D.) UPSIT score was 32.1 (4.7). Compared to age- and sex-matched norms (Doty et al., 1984) there was a group deficit in UPSIT performance, as the average score was in the 23rd percentile (S.D. 20.5 percentiles). Although the majority of the participants (14/26 or 53%) had normal smell identification, nearly half (12/26 or 46%) had microsmia, while none were anosmic.

3.2. Association of SID with demographic variables and potential confounders

As in normative samples, better smell identification was related to age. This was true not only for raw UPSIT scores (r =0.45, p =0.02), but also for age-adjusted percentile scores (r =0.39, p =0.05), consistent with a greater relative deficit in younger patients. UPSIT percentile scores were unrelated to any other demographic (sex, native language, ethnicity, or family support by public assistance) or clinical (handedness, previous hospitalizations, age at first psychotic symptom, and current global function) characteristics (see Table 1).

Table 1.

Correlations of demographic and clinical variables with UPSIT percentile scores

N =26 Mean (S.D.) UPSIT percentile score t p
Sex 0.73 0.47
 Male (16) 25.7 (4.8)
 Female (10) 19.7 (7.0)
Native language −1.25 0.23
 English 18.5 (4.7)
 Spanish 28.2 (6.2)
Public assistance 0.25 0.81
 Yes 22.4 (5.3)
 No 24.4 (6.0)
Handedness −0.42 0.68
 Right 21.7 (5.7)
 Mixed 25.1 (5.6)
Previous hospitalizations 0.40 0.70
 Yes 22.1 (22.6)
 No 25.4 (16.1)
ANOVA ( F)

Ethnicity 1.311 0.317
 Hispanic 23.0 (20.4)
 Caucasian 13.0 (7.9)
 African-American 32.8 (24.4)
Mean (S.D.) of characteristic Pearson r

Age first psychotic symptom 13.2±3.2 0.28 0.16
Global assessment of function 51.2±8.5 −0.06 0.76

The UPSIT percentile score was also unrelated to anti-psychotic exposure (antipsychotics “ever vs. never” t24 =0.005, p =0.98; chlorpromazine equivalents r =0.04, p =0.85), and was unlikely to be related to either fatigue or learning, as there was no correlation between item number and number of times an item was identified correctly by the 26 participants (r =−0.01, p =0.95).

3.3. Characterization of diagnostic groups

Thirteen participants were determined to have DSM-III-R schizophrenia or schizoaffective disorder. The other diagnoses were major depression (5), bipolar disorder, manic (3), and psychosis NOS (5). Mean age was comparable across diagnostic groups; all bipolar patients were male but other diagnoses included both sexes. Bipolar patients had a greater severity of positive symptoms, in particular grandiosity, whereas the schizophrenia group had the highest negative symptom score. Raw UPSIT scores were mean (S.D.)=31.4 (1.5) for schizophrenia or schizoaffective disorder, mean (S.D.)=29.8 (1.8) for major depression with psychotic features, mean (S.D.) =34.4 (1.3) for psychosis NOS and mean (S.D.)=35.3 (1.3) for bipolar disorder. The UPSIT percentile scores were mean (S.D.)=18.7 (19.5) for schizophrenia or schizoaffective disorder, mean (S.D.)=11.8 (10.1) for major depression with psychotic features, mean (S.D.)=31.6 (20.5) for psychosis NOS, and mean (S.D.)=49.0 (8.7) for bipolar disorder. Pearson chi-square showed a significant association of diagnosis with microsmia vs. normosmia (χ2(3)=11.54, p <0.01). Microsmia was observed in 8 of 13 patients with schizophrenia or schizoaffective disorder, 4 of 5 depressed patients, and none of the patients with bipolar disorder or psychosis NOS. Of those with microsmia, 2/3 had schizophrenia or schizoaffective disorder and 1/3 had major depression with psychotic features (Table 2).

Table 2.

Diagnostic groups: demographics, symptoms and smell identification

Schizophrenia or schizoaffective Depression with psychotic features Psychosis NOS Bipolar, manic
Age 15.5 14.6 14.2 15.0
Sex (% male) 61.5% 60% 40% 100%
Positive score (PANSS) 19.1 (5.6) 13.8 (4.7) 15.3 (5.9) 25.0 (1.0)
Negative score (PANSS) 26.7 (9.4) 20.6 (4.7) 21.3 (7.0) 19.7 (5.5)
Grandiosity (PANSS) 1.6 (1.0) 0 1.3 (0.5) 3.3 (1.2)
Verbal IQ (WISC-R) 81.6 (15.2) 94.8 (11.1) 88.3 (3.8) 95.0 (37.6)
UPSIT percentile score 18.7 (19.5) 11.8 (10.1) 31.6 (20.5) 49.0 (8.7)
Microsmia (%) 61.5% 80% 0% 0%

3.4. Relation of SID to positive and negative symptoms and to neuropsychological test performance

The UPSIT percentile scores were inversely associated with negative symptom severity but unrelated to positive symptom severity or general psychopathology (see Table 3). Post hoc analyses of individual PANSS items showed that only grandiosity survived Bonferroni correction (positive association with UPSIT percentile scores). UPSIT percentile scores were also positively associated with the PANSS composite score (the difference between positive and negative scores). Correspondingly, there was a significant association between type of syndrome (positive vs. negative) and existence of microsmia (Fisher’s exact test: χ2=5.63, df =1, p =0.018). Of the 13 individuals with normal smell identification who had symptom ratings, 7 had a negative syndrome and 6 had a positive syndrome. Of the 8 individuals with microsmia who had symptom ratings, all had a negative syndrome, and none had a positive syndrome.

Table 3.

Correlations of PANSS scores with UPSIT percentile scores

PANSS scores (N =21) Mean (S.D.) Pearson r p
Positive score 18.0 (6.0) 0.38 0.08
Positive scale items
 Delusions 3.5 (1.9) 0.22 0.34
 Conceptual disorganization 2.3 (1.1) −0.23 0.31
 Hallucinatory behavior 3.7 (1.8) 0.38 0.09
 Excitement 1.8 (1.1) 0.54 0.01
 Grandiosity 1.6 (1.1) 0.62* 0.003
 Suspiciousness/persecution 3.4 (1.8) 0.00 0.99
 Hostility 1.6 (1.3) 0.17 0.64
Negative score 23.2 (7.8) −0.47* 0.03
Negative scale items
 Blunted affect 3.7 (1.4) −0.51 0.02
 Emotional withdrawal 3.5 (1.4) −0.18 0.44
 Poor rapport 3.3 (1.3) −0.43 0.05
 Passive/apathetic social withdrawal 3.4 (1.5) −0.35 0.12
 Difficulty in abstract thinking 3.5 (2.0) −0.40 0.07
 Lack of spontaneity and flow of conversation 3.2 (1.3) −0.31 0.17
 Stereotyped thinking 2.6 (1.2) −0.36 0.11
General score 41.1 (11.1) −0.03 0.88
General scale items
 Somatic concern 2.2 (1.7) −0.15 0.51
 Anxiety 3.3 (1.6) −0.09 0.71
 Guilt feelings 1.4 (0.6) −0.07 0.76
 Tension 2.3 (1.1) −0.15 0.52
 Mannerisms and posturing 1.8 (1.0) −0.19 0.41
 Depression 3.3 (1.7) −0.04 0.88
 Motor retardation 2.7 (1.4) −0.28 0.22
 Uncooperativeness 1.8 (0.8) 0.11 0.64
 Unusual thought content 3.2 (1.8) 0.43 0.05
 Disorientation 1.6 (0.9) −0.38 0.09
 Poor attention 2.5 (1.1) 0.00 1.00
 Lack of judgment and insight 3.9 (1.2) 0.39 0.08
 Disturbance of volition 2.6 (1.3) −0.21 0.37
 Poor impulse control 2.1 (1.4) 0.23 0.32
 Preoccupation 3.2 (1.3) 0.09 0.69
 Active social avoidance 3.3 (1.5) −0.23 0.32
Composite score −5.2 (8.5) 0.71* < 0.001

Except for the negative score, where the test was hypothesis-driven, significance was determined with the Bonferroni correction.

*

p <0.05.

Means and standard deviations were calculated for IQ, Trails A and B, Stroop, Animal Naming, FAS, the Ravens Progressive Matrices and the Wide Range Achievement Test-reading (see Table 4). Scores for full-scale, verbal and performance IQ scores were 87.6 (19.0), 87.7 (20.5) and 89.7 (18.6), respectively. Verbal skills, as assessed with the WISC-R, animal naming, FAS and WRAT (reading and spelling), were similar to age- and sex-matched norms, but Trails and Stroop tests, measures of executive function, were impaired in this sample (Butler et al., 1991; Spreen and Strauss, 1998).

Table 4.

Correlations of neuropsychological performance with UPSIT scores

Test Mean (S.D.) Pearson r p
WISC-R age scaled (N =23)
 Full-scale IQ 87.6 (19.0) 0.51** 0.01
 Verbal IQ 87.7 (20.5) 0.53** 0.01
 Performance IQ 89.7 (18.6) 0.38 0.08
Verbal subscales
 Arithmetic 8.0 (3.0) 0.48 0.02
 Comprehension 7.2 (4.2) 0.52 0.01
 Information 8.1 (4.3) 0.23 0.30
 Similarities 9.0 (4.0) 0.64* 0.001
 Vocabulary 7.5 (3.7) 0.26 0.23
Performance subscales
 Coding 7.4 (3.6) 0.25 0.25
 Block design 9.0 (3.7) 0.08 0.70
 Object assembly 8.7 (3.6) 0.17 0.44
 Picture arrangement 8.8 (3.0) 0.40 0.06
 Picture completion 8.8 (4.5) 0.37 0.08
Other tests
 Trails A (N =20) 34.9 (12.7) −0.49 0.03
 Trails B (N =20) 92.4 (39.1) −0.51 0.02
 Trails B–A (N =20) 57.5 (32.7) −0.43 0.06
 Stroop (N =19) 29.8 (9.5) 0.55 0.02
 Animal naming (N =19) 16.3 (5.4) 0.62* 0.004
 FAS 34.6 (12.4) 0.67* 0.002
 Ravens matrices (N =18) 31.1 (4.4) 0.43 0.08
WRAT percentile scores (N =18)
 Arithmetic 33.0 (41.3) 0.39 0.11
 Reading 65.1 (36.2) 0.45 0.06
 Spelling 48.5 (36.6) 0.51 0.03

Except for full-scale and verbal IQ, where the tests were hypothesis-driven, significance was determined with the Bonferroni correction.

*

p <0.05.

**

p ≤0.01.

To determine whether smell identification was related to IQ and other tests of cognition, correlations with UPSIT percentile scores were computed. As hypothesized, UPSIT percentile scores were related to full-scale and verbal IQ (see Table 4). Verbal subscales associated with UPSIT percentile score included arithmetic, comprehension, and similarities. UPSIT percentile scores were also related to Trails A, Trails B, Stroop, FAS, animal naming, and the WRAT spelling score. Only similarities, animal naming and FAS survived Bonferroni correction.

Not only were verbal IQ and negative symptoms related to UPSIT percentile scores, but these variables were highly associated with one another (r =−0.63, p =0.003). Therefore, to examine whether the relation between negative symptoms and UPSIT would remain if the effect of IQ were controlled for, we conducted partial correlation analyses. Verbal IQ could not fully account for the relation between negative symptoms and UPSIT percentile scores (r(17) =−0.42, p =0.07). In contrast, UPSIT percentile scores and verbal IQ were no longer associated after controlling for symptoms (r(17)=0.19, p =0.44). Likewise, partial correlation analyses showed that after controlling for negative symptoms, neither similarities (r =0.34, p =0.15) nor animal naming (r =0.29, p =0.31) remained associated with UPSIT percentile scores, although negative symptoms could not fully account for the relation of UPSIT percentile scores with FAS (r =0.48, p =0.09). In sum, nearly all associations between UPSIT percentile scores and neuropsychological assessments could be explained by the relation of cognitive functioning to negative symptoms in this sample of adolescent patients with schizophrenia-like psychosis.

4. Discussion

Children and adolescents with psychotic disorders had impaired smell identification compared with age-and sex-matched population norms (Doty et al., 1984). The proportion of the sample with microsmia (46%) is analogous to what has been found in samples of schizophrenia patients (31–44%) (Kopala et al., 1993; Seidman et al., 1997). In this sample, these deficits were unrelated to demographic factors, age of onset of psychosis, past hospitalizations, current functioning or types or doses of medication taken. As in adult schizophrenia, SID were related to both negative symptoms and cognitive deficits, including IQ, though negative symptoms accounted for the association of IQ and other test scores with UPSIT scores in this pediatric sample. Among diagnostic groups, only the participants who received a bipolar diagnosis had UPSIT scores near the 50th percentile. The UPSIT therefore may be of potential utility in the differential diagnosis of early onset psychotic disorders, though replication in larger samples is needed. These data demonstrate the similarity of early onset and adult schizophrenia-like psychosis and support the expanding data linking SID with negative symptoms and cognitive deficits in schizophrenia.

The presence of deficits across psychotic disorders in this juvenile sample is similar to what has been found in adults (Striebel et al., 1999). However, within this small sample, smell identification appeared to vary by diagnosis. Bipolar patients had the highest UPSIT performance, at the middle of the range of normal performance, whereas participants with psychosis NOS had intermediate performance, and patients deemed likely to have schizophrenia or major depression with psychotic features had the lowest performance. Correspondingly, the presence of features associated with mania, such as grandiosity, was associated with better performance on the UPSIT, whereas negative symptoms predicted poor UPSIT scores. Initial diagnoses of psychotic disorders have some instability, with psychosis NOS being a particularly malleable label (Schwartz et al., 2000) that can presage schizophrenia (Jarbin and von Knorring, 2003). However, although major depression with psychotic features may include a subset of patients who will later be diagnosed with schizophrenia or schizoaffective disorder, many patients will likely remain with an affective disorder diagnosis (Jarbin and von Knorring, 2003). Future studies with larger samples can clarify the relationship of SID with specific disorders in psychotic children.

The finding of an association between negative symptoms and SID in children and adolescents with psychotic disorders is of particular interest, as this association also exists in adult schizophrenia (Brewer et al., 2001). In fact, in adults with schizophrenia, SID is specifically related to the deficit syndrome (Malaspina and Coleman, 2003), a subtype of schizophrenia characterized by pervasive and enduring negative symptoms that are unrelated to clinical state or medications. It is speculated (though not yet tested) that the social deficits in Asperger’s syndrome may be related to the profound SID observed (Suzuki et al., 2003). The relationship of smell identification to social functioning appears to exist beyond schizophrenia, as in normal individuals, olfactory identification is related to empathy (Spinella, 2002) and to personality characteristics such as assertiveness (Larsson et al., 2000).

In the present study, pediatric patients with psychotic disorders had cognitive impairment in disparate domains, consistent with previous studies of children and adolescents with schizophrenia or psychosis NOS (Kumra et al., 2000; Seidman et al., 1997). Smell identification ability was most prominently associated with verbal tests, such as FAS and animal naming. Therefore, verbal ability may be related to the correct identification of odors in children and adolescents with psychotic disorders. This is analogous to what has been found with adults who have schizophrenia, for whom UPSIT scores are related to verbal ability (Seidman et al., 1997; Stedman and Clair, 1998) but not to attention or executive function (Seidman et al., 1997). Of interest, a relationship of smell identification to verbal ability has also been found in normal older adults (Larsson et al., 2000; Westervelt et al., 2005).

However, the associations of verbal IQ and other verbal tasks to smell identification in this pediatric sample were found to be largely explained by the relation of both of these variables to negative symptoms. In contrast, in a larger sample of adults with schizophrenia, both IQ and negative symptoms, such as social drive, accounted separately for variance in UPSIT percentile scores (Malaspina and Coleman, 2003). It is possible that the presence of negative symptoms during childhood and adolescence may interfere with the acquisition of verbal skills necessary to correctly identify odors. Alternatively, verbal IQ may be a marker of negative symptoms in pediatric psychotic disorders that itself has no causal relationship to SID. Nonetheless, an interference of symptoms with maturation of olfactory identification is consistent with the finding in this study of an association of age and olfactory identification, as patients had similar durations of illness. Further studies are required with larger sample sizes to determine the relative contributions of verbal IQ and negative symptoms to disruption of smell identification in children and adolescents with psychotic disorders.

The strong correlation between social drive and olfaction demonstrated by Malaspina and Coleman (2003) suggests that there may be overlap in the neural pathways that interpret odors and those that facilitate socialization in humans. Higher olfactory processing regions that merit particular interest with respect to negative symptoms include medial temporal lobe structures, particularly the amygdala and entorhinal–hippocampal areas, as well as the orbital prefrontal cortex and the parietal lobe. Seckinger et al. (2004) have speculated that parietal lobe function may represent a critical connection between the neuroanatomy and neurocognitive processes linking olfactory dysfunction to the schizophrenia deficit syndrome. This region, the parietal lobe, has shown significant hypometabolism in schizophrenia patients performing the UPSIT in comparison with healthy subjects (Malaspina et al., 1998).

However, the orbitofrontal cortex (OFC) is also implicated in olfactory identification across different types of studies, including basic lesion studies (Tanabe et al., 1975; Zatorre and Jones-Gotman, 1991), in clinical evaluations of OFC injury, and in functional imaging of olfactory identification tasks (reviewed in Barnett et al., 1999). Also, SID exists in some form in a host of psychiatric disorders that involve the OFC, including obsessive compulsive disorder (Barnett et al., 1999), posttraumatic stress disorder (Vasterling et al., 2000), Asperger’s syndrome (Suzuki et al., 2003) and in the “inattentive” subtype of attention deficit disorder (Gansler et al., 1998).

An association of SID with negative symptoms in both children and adults with psychotic disorders is consistent with SID being a traitlike marker of schizophrenia vulnerability, as negative symptoms themselves are traitlike: they often precede the first onset of psychosis in schizophrenia, are more refractory to treatment than positive symptoms, and are found in unaffected relatives of schizophrenia probands as well as in high-risk individuals (Tamminga et al., 1998; Hafner et al., 2005; Lencz et al., 2004). SID may be an early, core, trait-like characteristic that is seen in individuals with a liability to schizophrenia-like psychosis. Among schizophrenia patients, UPSIT scores remain stable despite changes in clinical state and medication (Malaspina et al., 1994), and are present as early as the first episode of psychosis (Brewer et al., 2001; Kopala et al., 1993). SID also are present among men with schizotypal disorder (Park and Schoppe, 1997) and are associated with magical ideation in otherwise normal adults (Mohr et al., 2001). SID may be a marker of liability to schizophrenia-like psychosis, as among monozygotic twin pairs discordant for schizophrenia, both affected and unaffected twins had lower mean UPSIT scores compared to controls, and no significant differences in smell identification were found between affected and unaffected twins (Kopala et al., 1998). Twins with a positive family history of serious mental illness have significantly lower UPSIT scores than do those with a negative family history, consistent with a genetic contribution to olfactory dysfunction that is also linked to schizophrenia vulnerability. Further, unaffected family members of schizophrenia probands have smell identification intermediate to that of probands and healthy controls (Kopala et al., 2001). SID are also found in high risk patients who later develop schizophrenia spectrum disorder (Brewer et al., 2003).

The UPSIT requires sufficient verbal skills, as observed in the relation of verbal IQ and verbal tests to UPSIT performance in this sample. However, verbal skills are unlikely to account for the group deficit in smell identification in this sample, as the verbal test scores in this sample were similar to age-matched norms (Spreen and Strauss, 1998). Also, the association of verbal IQ and smell identification ability was explained by the association of both with negative symptoms. Future studies can clarify the relation of negative symptoms to SID independent of verbal ability through the use of nonverbal assessments of smell identification, such as the match-to-sample odor discrimination tasks (MODT) (Richman et al., 1995). However, the MODT has had limited use, has no known age-and sex-matched norms, and has been rarely used in assessments of psychosis and schizophrenia. By contrast, the UPSIT has become the standard smell identification test in individuals with psychotic and other disorders.

The present study demonstrates that, as in adults, smell identification deficits exist in early onset psychotic disorders, and are related to cognitive and negative symptoms. SID may be a marker of an underlying neurodevelopmental pathophysiology that leads to vulnerability for schizophrenia spectrum and other psychotic disorders. Correspondingly, the presence of intact smell identification ability in children with psychotic disorders may have important diagnostic and prognostic implications, suggesting a more affective illness with fewer negative symptoms and less neuropsychological impairment. The specificity of the relationship of smell identification to social and neuropsychological function can be evaluated in future studies that include equally comprehensive evaluation in age-matched psychiatrically healthy children.

Acknowledgments

Funding provided in part by PIRSG (New York State Psychiatric Institute), NARSAD and the G. Harold and Leila Mathers Foundation.

Contributor Information

Cheryl Corcoran, Email: cc788@columbia.edu.

Agnes Whitaker, Email: whitakea@childpsych.columbia.edu.

Jane Fried, Email: friedj@childpsych.columbia.edu.

Judith Feldman, Email: feldmanj@childpsych.columbia.edu.

Nora Goudsmit, Email: goudsmi@pi.cpmc.columbia.edu.

Dolores Malaspina, Email: dm9@columbia.edu.

References

  1. Annett M. A classification of hand preference by association analysis. Br J Psychol. 1970;61:303–321. doi: 10.1111/j.2044-8295.1970.tb01248.x. [DOI] [PubMed] [Google Scholar]
  2. Barnett R, Maruff P, Purcell R, Wainwright K, Kyrios M, Brewer W, Pantelis C. Impairment of olfactory identification in obsessive-compulsive disorder. Psychol Med. 1999;29:1227–1233. doi: 10.1017/s0033291799008818. [DOI] [PubMed] [Google Scholar]
  3. Brewer WJ, Edwards J, Anderson V, Robinson T, Pantelis C. Neuropsychological, olfactory, and hygiene deficits in men with negative symptom schizophrenia. Biol Psychiatry. 1996;40:1021–1031. doi: 10.1016/0006-3223(95)00594-3. [DOI] [PubMed] [Google Scholar]
  4. Brewer WJ, Pantelis C, Anderson V, Velakoulis D, Singh B, Copolov DL, McGorry PD. Stability of olfactory identification deficits in neuroleptic-naive patients with first-episode psychosis. Am J Psychiatry. 2001;158:107–115. doi: 10.1176/appi.ajp.158.1.107. [DOI] [PubMed] [Google Scholar]
  5. Brewer WJ, Wood SJ, McGorry PD, Francey SM, Phillips LJ, Yung AR, Anderson V, Copolov DL, Singh B, Velakoulis D, Pantelis C. Impairment of olfactory identification ability in individuals at ultra-high risk for psychosis who later develop schizophrenia. Am J Psychiatry. 2003;160:1790–1794. doi: 10.1176/appi.ajp.160.10.1790. [DOI] [PubMed] [Google Scholar]
  6. Butler P, Whitaker A, Setterberg S, Rao U, Ramirez P, Jaffer M, Upstill C. Executive function deficits in early onset schizophrenia. Scientific Proceedings of the 38th Annual Meeting of the American Academy of Child and Adolescent Psychiatry; October 15–20th; San Francisco. 1991. Abstract No. NR-38. [Google Scholar]
  7. Cain WS, Stevens JC, Nickou CM, Giles A, Johnston I, Garcia-Medina MR. Life-span development of odor identification, learning, and olfactory sensitivity. Perception. 1995;24:1457–1472. doi: 10.1068/p241457. [DOI] [PubMed] [Google Scholar]
  8. Coleman E, Goetz RR, Leitman D, Yale S, Stanford A, Gorman JM, Malaspina D. Odor identification impairments in schizophrenia: relationship with demographic measures, clinical variables, and diagnostic subtypes. CNS Spectr. 2002;7:43–48. doi: 10.1017/s1092852900022252. [DOI] [PubMed] [Google Scholar]
  9. Court JH, Raven J. Manual for the Raven’s Progressive Matrices and Vocabulary Scales. Oxford Psychologists Press; Oxford, England: 1995. [Google Scholar]
  10. Doty RL, Shaman P, Dann M. Development of the University of Pennsylvania Smell Identification Test: a standardized microencapsulated test of olfactory function. Physiol Behav. 1984;32:489–502. doi: 10.1016/0031-9384(84)90269-5. [DOI] [PubMed] [Google Scholar]
  11. Fields JH, Grochowski S, Lindenmayer JP, Kay SR, Grosz D, Hyman RB, Alexander G. Assessing positive and negative symptoms in children and adolescents. Am J Psychiatry. 1994;151:249–253. doi: 10.1176/ajp.151.2.249. [DOI] [PubMed] [Google Scholar]
  12. Gansler DA, Fucetola R, Krengel M, Stetson S, Zimering R, Makary C. Are there cognitive subtypes in adult attention deficiency/hyperactivity disorder? J Nerv Ment Dis. 1998;186:776–781. doi: 10.1097/00005053-199812000-00006. [DOI] [PubMed] [Google Scholar]
  13. Hafner H, Maurer K, Trendler G, An der Heiden W, Schmidt M. The early course of schizophrenia and depression. Eur Arch Psychiatry Clin Neurosci. 2005;255:167–173. doi: 10.1007/s00406-005-0584-8. [DOI] [PubMed] [Google Scholar]
  14. Hemdal P, Corwin J, Oster H. Olfactory identification deficits in Down’s syndrome and idiopathic mental retardation. Neuropsychologia. 1993;31:977–984. doi: 10.1016/0028-3932(93)90152-p. [DOI] [PubMed] [Google Scholar]
  15. Houlihan DJ, Flaum M, Arnold SE, Keshavan M, Alliger R. Further evidence for olfactory identification deficits in schizophrenia. Schizophr Res. 1994;12:179–182. doi: 10.1016/0920-9964(94)90075-2. [DOI] [PubMed] [Google Scholar]
  16. Hurwitz T, Kopala L, Clark C, Jones B. Olfactory deficits in schizophrenia. Biol Psychiatry. 1988;23:123–128. doi: 10.1016/0006-3223(88)90081-9. [DOI] [PubMed] [Google Scholar]
  17. Jarbin H, von Knorring AL. Diagnostic stability in adolescent onset psychotic disorders. Eur Child Adolesc Psychiatry. 2003;12:15–22. doi: 10.1007/s00787-003-0300-z. [DOI] [PubMed] [Google Scholar]
  18. Jastak JF, Jastak S. Wide Range Achievement Test Manual. Western Psychological Services; Los Angeles: 1978. [Google Scholar]
  19. Jensen AR, Rohwer WD. The Stroop color–word test: a review. Acta Psychol. 1966;25:36–93. doi: 10.1016/0001-6918(66)90004-7. [DOI] [PubMed] [Google Scholar]
  20. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13:261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  21. Kay SR, Opler LA, Lindenmayer JP. Reliability and validity of the positive and negative syndrome scale for schizophrenics. Psychiatry Res. 1988;23:99–110. doi: 10.1016/0165-1781(88)90038-8. [DOI] [PubMed] [Google Scholar]
  22. Kopala LC, Clark C, Hurwitz T. Olfactory deficits in neuroleptic naive patients with schizophrenia. Schizophr Res. 1993;8:245–250. doi: 10.1016/0920-9964(93)90022-b. [DOI] [PubMed] [Google Scholar]
  23. Kopala LC, Good KP, Torrey EF, Honer WG. Olfactory function in monozygotic twins discordant for schizophrenia. Am J Psychiatry. 1998;155:134–136. doi: 10.1176/ajp.155.1.134. [DOI] [PubMed] [Google Scholar]
  24. Kopala LC, Good KP, Morrison K, Bassett AS, Alda M, Honer WG. Impaired olfactory identification in relatives of patients with familial schizophrenia. Am J Psychiatry. 2001;158:1286–1290. doi: 10.1176/appi.ajp.158.8.1286. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kumra S, Wiggs E, Bedwell J, Smith AK, Arling E, Albus K, Hamburger SD, McKenna K, Jacobsen LK, Rapoport JL, Asarnow RF. Neuropsychological deficits in pediatric patients with childhood-onset schizophrenia and psychotic disorder not otherwise specified. Schizophr Res. 2000;42:135–144. doi: 10.1016/s0920-9964(99)00118-8. [DOI] [PubMed] [Google Scholar]
  26. Larsson M, Finkel D, Pederson NL. Odor identification: influences of age, gender, cognition, and personality. J Geront, Ser B Psychol Sci Soc Sci. 2000;55:P304–P310. doi: 10.1093/geronb/55.5.p304. [DOI] [PubMed] [Google Scholar]
  27. Leckman JF, Sholomskas D, Thompson WD, Belanger A, Weissman MM. Best estimate of lifetime psychiatric diagnosis: a methodological study. Arch Gen Psychiatry. 1982;39:879–883. doi: 10.1001/archpsyc.1982.04290080001001. [DOI] [PubMed] [Google Scholar]
  28. Lehrner JP, Walla P, Laska M, Deecke L. Different forms of human odor memory: a developmental study. Neurosci Lett. 1999;272:17–20. doi: 10.1016/s0304-3940(99)00566-2. [DOI] [PubMed] [Google Scholar]
  29. Lencz T, Smith CW, Auther A, Correll CU, Cornblatt B. Nonspecific and attenuated negative symptoms in patients at clinical high-risk for schizophrenia. Schizophr Res. 2004;68:37–58. doi: 10.1016/S0920-9964(03)00214-7. [DOI] [PubMed] [Google Scholar]
  30. Malaspina D, Coleman E. Olfaction and social drive in schizophrenia. Arch Gen Psychiatry. 2003;60:578–584. doi: 10.1001/archpsyc.60.6.578. [DOI] [PubMed] [Google Scholar]
  31. Malaspina D, Wray AD, Friedman JH, Amador X, Yale S, Hasan A, Gorman JM, Kaufmann CA. Odor discrimination deficits in schizophrenia: association with eye movement dysfunction. J Neuropsychiatry Clin Neurosci. 1994;6:273–278. doi: 10.1176/jnp.6.3.273. [DOI] [PubMed] [Google Scholar]
  32. Malaspina D, Perera GM, Lignelli A, Marshall RS, Esser PD, Storer S, Furman V, Wray AD, Coleman E, Gorman JM, Van Heertum RL. SPECT imaging of odor identification in schizophrenia. Psychiatry Res. 1998;82:53–61. doi: 10.1016/s0925-4927(98)00008-0. [DOI] [PubMed] [Google Scholar]
  33. Malaspina D, Coleman E, Goetz RR, Harkavy-Friedman J, Corcoran C, Amador X, Yale S, Gorman JM. Odor identification, eye tracking and deficit syndrome schizophrenia. Biol Psychiatry. 2002;51:809–815. doi: 10.1016/s0006-3223(01)01319-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McClellan JM, Werry JS. Schizophrenia. Pediatric psychopharmacology Psychiatr Clin North Am. 1992;15:131–148. [PubMed] [Google Scholar]
  35. McKeown DA, Doty RL, Perl DP, Frye RE, Simms I, Mester A. Olfactory function in young adolescents with Down’s syndrome. J Neurol Neurosurg Psychiatry. 1996;61:412–414. doi: 10.1136/jnnp.61.4.412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Miller E. Verbal fluency as a function of a measure of verbal intelligence and in relation to different types of cerebral pathology. Br J Clin Psychol. 1984;23:53–57. doi: 10.1111/j.2044-8260.1984.tb00626.x. [DOI] [PubMed] [Google Scholar]
  37. Moberg PJ, Agrin R, Gur RE, Gur RC, Turetsky BI, Doty RL. Olfactory dysfunction in schizophrenia: a qualitative and quantitative review. Neuropsychopharmacology. 1999;21:325–340. doi: 10.1016/S0893-133X(99)00019-6. [DOI] [PubMed] [Google Scholar]
  38. Mohr C, Rohrenbach CM, Laska M, Brugger P. Unilateral olfactory perception and magical ideation. Schizophr Res. 2001;47:255–264. doi: 10.1016/s0920-9964(00)00113-4. [DOI] [PubMed] [Google Scholar]
  39. Murphy C, Jinich S. Olfactory dysfunction in Down’s Syndrome. Neurobiol Aging. 1996;17:631–637. doi: 10.1016/0197-4580(96)00008-5. [DOI] [PubMed] [Google Scholar]
  40. Nijjar RK, Murphy C. Olfactory impairment increases as a function of age in persons with Down syndrome. Neurobiol Aging. 2002;23:65–73. doi: 10.1016/s0197-4580(01)00263-9. [DOI] [PubMed] [Google Scholar]
  41. Park S, Schoppe S. Olfactory identification deficit in relation to schizotypy. Schizophr Res. 1997;26:191–197. doi: 10.1016/s0920-9964(97)00045-5. [DOI] [PubMed] [Google Scholar]
  42. Reitan RM. Theoretical and methodological bases of the Halstead–Reitan neuropsychological test battery. In: Grant I, Adams KM, editors. Neuropsychological Assessment of Neuropsychiatric Disorders. Oxford University Press; New York: 1986. [Google Scholar]
  43. Richman RA, Post EM, Sheehe PR, Wright HN. Olfactory performance during childhood: I. Development of an odorant identification test for children. J Pediatr. 1992;121:908–911. doi: 10.1016/s0022-3476(05)80337-3. [DOI] [PubMed] [Google Scholar]
  44. Richman RA, Sheehe PR, Wallace K, Hyde JM, Coplan J. Olfactory performance during childhood: II. Developing a discrimination task for children. J Pediatr. 1995;127:421–426. doi: 10.1016/s0022-3476(95)70074-9. [DOI] [PubMed] [Google Scholar]
  45. Rosenbluth R, Grossman ES, Kaitz M. Performance of early-blind and sighted children on olfactory tasks. Perception. 2000;29:101–110. doi: 10.1068/p3001. [DOI] [PubMed] [Google Scholar]
  46. Roussy S, Toupin J. Behavioral inhibition deficits in juvenile psychopaths. Aggress Behav. 2000;26:413–424. [Google Scholar]
  47. Schwartz JE, Fennig S, Tanenberg-Karant M, Carlson G, Craig T, Galambos N, Lavelle J, Bromet EJ. Congruence of diagnoses 2 years after a first-admission diagnosis of psychosis. Arch Gen Psychiatry. 2000;57:593–600. doi: 10.1001/archpsyc.57.6.593. [DOI] [PubMed] [Google Scholar]
  48. Seckinger RA, Goudsmit N, Coleman E, Harkavy-Friedman J, Yale S, Rosenfield PJ, Malaspina D. Olfactory identification and WAIS-R performance in deficit and nondeficit schizophrenia. Schizophr Res. 2004;69:55–65. doi: 10.1016/S0920-9964(03)00124-5. [DOI] [PubMed] [Google Scholar]
  49. Seidman LJ, Goldstein JM, Goodman JM, Koren D, Turner WM, Faraone SV, Tsuang MT. Sex differences in olfactory identification and Wisconsin Card Sorting performance in schizophrenia: relationship to attention and verbal ability. Biol Psychiatry. 1997;42:104–115. doi: 10.1016/S0006-3223(96)00300-9. [DOI] [PubMed] [Google Scholar]
  50. Spinella M. A relationship between smell identification and empathy. Int J Neurosci. 2002;112:605–612. doi: 10.1080/00207450290025680. [DOI] [PubMed] [Google Scholar]
  51. Spreen O, Strauss E. A Compendium of Neuropsychological Tests: Administration, Norms and Commentary. 2. Oxford University Press; New York: 1998. [Google Scholar]
  52. Stedman TJ, Clair AL. Neuropsychological, neurological and symptom correlates of impaired olfactory identification in schizophrenia. Schizophr Res. 1998;32:23–30. doi: 10.1016/s0920-9964(98)00021-8. [DOI] [PubMed] [Google Scholar]
  53. Striebel K, Beyerstein B, Remick RA, Kopala L, Honer W. Olfactory identification and psychosis. Biol Psychiatry. 1999;45:1419–1425. doi: 10.1016/s0006-3223(98)00245-5. [DOI] [PubMed] [Google Scholar]
  54. Suzuki Y, Critchley HD, Rowe A, Howlin P, Murphy DGM. Impaired olfactory identification in Asperger’s Syndrome. J Neuropsychiatry Clin Neurosci. 2003;15:105–107. doi: 10.1176/jnp.15.1.105. [DOI] [PubMed] [Google Scholar]
  55. Tamminga CA, Buchanan RW, Gold JM. The role of negative symptoms and cognitive dysfunction in schizophrenia outcome. Int Clin Psychopharmacol. 1998;13 (Suppl 3):S21–S26. doi: 10.1097/00004850-199803003-00004. [DOI] [PubMed] [Google Scholar]
  56. Tanabe T, Iino M, Takagi SF. Discrimination of odors in olfactory bulb, pyriform-amygdaloid areas and orbitofrontal cortex of the monkey. J Neurophysiol. 1975;38:1284–1296. doi: 10.1152/jn.1975.38.5.1284. [DOI] [PubMed] [Google Scholar]
  57. Vasterling JJ, Brailey K, Sutker PB. Olfactory identification in combat-related posttraumatic stress disorder. J Trauma Stress. 2000;13:241–253. doi: 10.1023/A:1007754611030. [DOI] [PubMed] [Google Scholar]
  58. Wechsler D. Wechsler Adult Intelligence Scale-Revised. Manual. Psychological Corporation; New York: 1981. [Google Scholar]
  59. Westervelt HJ, Ruffolo JS, Tremont G. Assessing olfaction in the neuropsychological exam: the relationship between odor identification and cognition in older adults. Arch Clin Neuropsychol. 2005;20 (6):761–769. doi: 10.1016/j.acn.2005.04.010. [DOI] [PubMed] [Google Scholar]
  60. Zatorre RJ, Jones-Gotman M. Human olfactory discrimination after unilateral frontal or temporal lobotomy. Brain. 1991;114:71–84. [PubMed] [Google Scholar]

RESOURCES