Skip to main content
Schizophrenia Bulletin logoLink to Schizophrenia Bulletin
. 2020 Mar 27;46(5):1144–1154. doi: 10.1093/schbul/sbaa043

Contingent Negative Variation Blunting and Psychomotor Dysfunction in Schizophrenia: A Systematic Review

K Juston Osborne 1,, Brian Kraus 1, Phoebe H Lam 1, Teresa Vargas 1, Vijay A Mittal 1,2
PMCID: PMC7505188  PMID: 32221557

Abstract

The contingent negative variation (CNV) is an event-related potential that provides a neural index of psychomotor processes (eg, attention and motor planning) well known to be dysfunctional in schizophrenia. Although evidence suggests that CNV amplitude is blunted in patients with schizophrenia (SZ) compared to healthy controls (HCs), there is currently no meta-analytic evidence for the size of the effect. Further, it is unknown how CNV blunting compares to closely related measures of psychomotor dysfunction, such as reaction time slowing. We used random-effects models to calculate the pooled effect size (ES) across 30 studies investigating CNV amplitude differences between patients and HCs (NSZ = 685, NHC = 714). Effect sizes for reaction time slowing across the studies were also quantified. Potential moderators, including sample characteristics and aspects of the CNV measurement, were examined. There was robust blunting of CNV activity in patients compared to HCs (ES = −0.79). The magnitude of this effect did not differ from reaction time slowing. Notably, CNV blunting in patients was significantly greater at central sites (ES = −0.87) compared to frontal sites (ES = −0.48). No other assessed methodological characteristics significantly moderated the magnitude of CNV differences. There is a large effect for CNV blunting in SZ that appears robust to potential confounds or methodological moderators. In addition, reduced CNV activity was statistically comparable to that of reaction time slowing. Blunting was the largest at central electrodes, which has been implicated in motor preparation. These findings speak to the complexity of psychomotor dysfunction in SZ and suggest significant promise for a biomarker.

Keywords: electrophysiology, motor, psychosis, attention, event-related potential, biomarker

Introduction

Schizophrenia (SZ) is a severe disorder that results in chronic impairment caused by a progressive disassociation with reality and neurocognitive decline. Given that SZ is an illness characterized by deficits in both cognitive and motor function,1–3 it is perhaps unsurprising that an abundance of research has examined whether event-related potentials (ERPs) related to psychomotor processes are impaired in psychosis.4–7 First discovered by Walter et al,8 the contingent negative variation (CNV) is a sustained negativity that gradually develops in response to a warning stimulus that signals that a response is required for a subsequent stimulus (ie, an “imperative” stimulus). It is considered to be a neural index of several psychomotor processes involved in transforming sensory information into goal-directed action, including anticipatory attention, motivation, and overall readiness to respond.9,10 Although evidence suggests that CNV amplitude is blunted in patients with SZ compared to healthy controls (HCs), there has not yet been a meta-analysis to determine the size of this effect. It is also currently unknown how CNV blunting in psychosis compares to behavioral markers of psychomotor dysfunction (eg, reaction time slowing) and other potential ERP biomarkers in psychosis (eg, mismatch negativity, P300), limiting the field’s understanding of its usefulness as a putative biomarker of disease processes. The present meta-analysis aimed to determine the overall effect size (ES) of CNV blunting in patients with SZ compared to HCs. Because the CNV is strongly associated with behavioral psychomotor slowing (ie, reaction time),11,12 which has been called “the closest thing to a north star in schizophrenia research” 13 due to its large ES and consistent replication, we also compared the overall magnitude of CNV blunting to reaction time slowing in order to further clarify its usefulness for understanding disease mechanisms in SZ.

Consistent with electrophysiology studies of the CNV, evidence from functional magnetic resonance imaging, single-unit recording, and lesion studies suggest that the CNV is generated by a network of brain regions involved in psychomotor functions such as attention and higher-order cognitive processes (eg, dorsolateral prefrontal and parietal cortices),10,14–16 as well as motor planning and preparation (eg, cerebellum, basal ganglia, supplementary motor area [SMA], and premotor cortex).10,17,18 Notably, because there is a strong overlap between these putative neural generators of the CNV and those implicated in both psychomotor dysfunction and prominent etiological models of SZ,10,15,19,20 CNV blunting in patients may be a promising neural marker of psychomotor dysfunction in psychosis. This is important as psychomotor abnormalities have been recognized as core features of SZ since its earliest conceptualizations,21,22 and recent evidence has indicated that psychomotor dysfunction is present in high-risk populations and is sensitive to illness progression.23–25 Further, it has been shown that CNV blunting in patients is associated with both positive (ie, disorganized) and negative symptoms (eg, avolition, attention, and blunted affect).26–29 Given the strong association between negative symptoms, cognitive deficits, and motor abnormalities in SZ,30,31 CNV amplitude may provide a useful marker for tracking distinct deficits across these domains.

A number of studies have examined the CNV in patients with SZ. Although occasionally blunting is not observed in patients (see 32–34), the most consistent finding is that CNV amplitude is reduced in patients compared to controls (eg, 27, 35–38). However, studies examining the CNV in SZ have varied considerably with regards to patient characteristics, task design, and CNV measurement. Specifically, studies have varied in both the duration of illness of their patient sample (eg, 36, 39–41) and the complexity of the task used to elicit the CNV (eg, 35, 39, 42), both of which may influence the CNV.43 In addition to differences in task design, several studies have limited measurement electrodes to frontal (eg, 33, 38) or central (eg, 36, 44) electrode sites, many without distinction between whether measurement reflects the early or late CNV (eg, 35,44), leading to difficulty contextualizing findings within the broader CNV literature. For example, the more frontal early CNV component is thought to reflect the alerting and orienting response to the warning stimulus, whereas the prominent central late CNV component largely reflects anticipatory attention and motor preparation.11,45,46 Consistent with efforts to determine potential electrophysiology markers of SZ,6,47 elucidating the exact size and potential moderators of this effect stands to inform future work and aid in determining the overall promise of the CNV as a potential neural marker of SZ.

In the present meta-analysis, these issues were addressed through 2 main aims. First, we estimated the weighted mean ES of the difference in CNV amplitude between patients with SZ and HCs. Given that reaction time slowing is a robust finding in SZ that is closely associated with CNV amplitude (ie, greater CNV amplitude is related to faster reaction times12,48), we also compared CNV blunting to reaction time slowing in order to clarify its relative importance as a marker of disease mechanisms in psychosis. Indeed, reaction time slowing only provides a single metric that does not distinguish amongst the different dysfunctional cognitive and motor processes contributing to slowing. Because CNV amplitude reflects specific cognitive (alerting, attention) and motor function (planning), comparing CNV blunting and reaction time slowing serves the additional purpose of demonstrating the utility of using markers of distinct processes with more specific neural substrates. Second, we examined the effect(s) of sample characteristics (ie, duration of illness, mean age, percentage of males, years of education, medication dosage, and clinical status of the patient sample) and characteristics of CNV measurement (ie, task complexity, interstimulus interval, mean vs peak amplitude, and electrode site) as potential moderators on the magnitude of CNV blunting.

Methods and Materials

Literature Search

The present study was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for transparent and replicable systematic reviews and meta-analyses.49 A systematic search was performed across multiple stages (see figure 1 for flowchart). First, relevant articles were identified through PubMed and Google Scholar online databases (conducted on May 15, 2019) using the following search terms: (schizo*) AND (“contingent negative variation” OR CNV) AND (EEG OR ERP). The title and abstract of each article from the resultant search were then reviewed to determine if the study was appropriate for further consideration and their references were reviewed to identify additional potential studies. Second, relevant articles were then closely evaluated against study inclusion criteria (see below for details).

Fig. 1.

Fig. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart diagram.

Study Selection

Studies were considered eligible for inclusion if they met the following criteria: (1) were from a peer-reviewed journal article written in English, (2) had both a control and SZ group comprised of adult subjects (≥18 years of age), (3) reported descriptive statistics (mean and SD/SE) and/or parametric statistics (t or F values) that allowed for calculations of Hedge’s g comparing CNV amplitude (mean or peak measures) in controls and patients, (4) had nonoverlapping samples, (5) used established criteria for SZ diagnoses (DSM-III, DSM-III-R, DSM-IV, and/or ICD-10), and (6) measured CNV amplitudes from fronto-central electrode sites (Fz, F3, F4, FCz, Cz, C3, and C4) according to the 10–20 system. These electrode sites were chosen due to the well-established fronto-central scalp topography of the CNV (see 48). Two reviewers (J.O. and B.K.) independently reviewed each article to determine if the study met inclusion criteria and discrepancies were resolved through consensus meetings.

A total of 86 articles were initially identified as relevant from titles and abstracts and 32 of these studies were ultimately included in subsequent analyses (see table 1 for included studies and figure 1 for reasons for exclusion of k = 54 studies). Each of the 32 identified studies was coded for quality and risk of bias by 2 reviewers (J.O. and B.K.) using the Risk of Bias Assessment Tool for Nonrandomized Studies (RoBANS50). Interrater agreement was high (Cohen’s Kappa = 0.83) with the majority of included studies having overall low risk of bias (see supplementary table 1 for RoBANS ratings).

Table 1.

Mean sample characteristics (k = 32).

First author (year) Mean age (years) Education (years) % Male Clinical status Diagnosis criteria Illness duration (years) Medication (mg CPZ equivalent) Symptom ratings
Patient Control Patient Control Patient Control
Cameron (2003) 37.6 38.1 NR NR 79.2 0.78 Outpatient ICD-10 14.0 400.0 NR
Chaillou (2015) 35.9 34.5 12.9 13.0 68.4 68.4 NR DSM-IV 13.5 242.0 PANSS (pos./neg.) = 14.0/16.9; SAPS = 19.6; SANS = 35.0
Debruille (2010) 31.0 29.8 11.3 10.9 77.1 80.0 Outpatient DSM-IV 6.5 586.0 BRPS = 45.8
Dias (2011) 33.3 32.5 NR NR 80.0 100.0 Inpatient/outpatient DSM-IV 14.5 1241.3 BRPS = 40.0; SANS = 33.7
Eikmeier (1992) 31.8 30.6 ≥ 9 ≥ 9 81.3 60.0 Inpatient DSM-III-R 7.9 398.0 BRPS = 32.0; SANS = 30.0
Ford (2010) 39.3 37.3 13.8 16.2 81.3 59.0 Outpatient DSM-IV 17.9 NR PANSS (pos./neg.) = 17.5/15.6
Guterman (1996) 32.3 27.8 NR NR 78.6 57.1 Outpatient DSM-III-R 9.8 711.3 NR
Kathmann (2000) 31.8 27.8 13.7 14.5 62.5 45.0 Inpatient DSM-III-R 6.4 NR PANSS (pos./neg.) = 14.5/19.2
Kirenskaya (2011) 31.0 26.8 NR NR 100.0 100.0 Unmedicated inpatient ICD-10 NR 0.0 PANSS = NR
Kirenskaya (2013) 31.2 28.2 11.3 13.8 100.0 100.0 Unmedicated inpatient ICD-10 NR 0.0 NR
Kirenskaya (2017) 34.2 27.1 11.6 13.2 100.0 100.0 Unmedicated inpatient ICD-10 > 5.0 0.0 PANSS (pos./neg.) = 21.7/22.8
Klein (1994) 29.4 30.6 9.76 9.76 70.6 75.0 Inpatient DSM-III-R > 2.0 187.3 BPRS = 38.2
Klein (1996) 29.9 30.9 NR NR 79.9 73.7 Inpatient DSM-III-R NR 185.9 BPRS = 41.3
Klein (2000) 29.9 31.3 10.3 10.4 70.6 72.2 Inpatient/outpatient DSM-III-R 5.1 416.0 PANADSS = NR
Koyama (1994) 27.9 28.7 12.7 NR 67.9 73.1 NR DSM-III-R NR 561.3 BPRS = 30.7; SANS = 30.7
Li (2013) 24.8 26.1 11.5 11.9 45.8 60.0 Inpatient/outpatient DSM-IV 1.4 NR PANSS (pos./neg.) = 24.0/22.2
Li (2015) 26.2 25.7 10.9 11.5 46.7 53.0 Inpatient/outpatient DSM-IV 1.5 NR PANSS (pos./neg.) = 22.1/26.2
Mattes (1991) 27.9 29.7 11.5 12.6 66.7 66.7 Inpatient/outpatient DSM-III-R > 0.5 NR NR
Oke (1994) 34.0 40.0 NR NR 75.0 75.0 Inpatient/outpatient DSM-III-R 13.7 1178.0 SANS = NR; SOPS = NR
Reuter (2006) 30.8 32.5 12.0 12.0 65.2 65.2 Inpatient DSM-IV NR NR SANS = 7.7; SAPS = 5.4
Rockstroh (1997) 30.4 29.3 11.0 12.0 84.6 84.6 NR DSM-III-R 7.3 267.7 BPRS = 42.8
Simlai (2010) 30.0 30.0 NR NR 100.0 100.0 Unmedicated inpatient ICD-10 6.0 0.0 PANSS (pos./neg.) = 13.0/22.0; BPRS = 57.0
Smid (2009) 25.4 26.7 NR NR 88.0 63.0 NR DSM-IV ≤ 5.0 425.0 NR
Smid (2013) 24.3 26.4 NR NR 66.7 66.7 Inpatient/outpatient DSM-IV ≤ 2.0 278.9 PANSS (pos./neg.) = 10.9/14.8
Strandburg (1994) 26.3 26.1 12.6 13.4 94.4 94.7 Outpatient DSM-III-R 2.8 NR BPRS = 25.4; SANS = NR-overall
Strandburg (1997) 26.0 26.1 12.4 13.4 94.1 94.7 Outpatient DSM-III-R 2.8 NR BPRS = 24.8; SANS = NR-overall
Taguchi (2003) 64.7 68.9 NR NR 22.7 29.4 NR DSM-IV NR NR NR
Van Den Bosch (1983) 36.5 33.1 NR NR 60.0 60.0 NR DSM-III NR 6.4 (haloperidol equiv) NR
Verleger (1999) 33.7 31.4 10.8 12.0 50.0 52.2 Inpatient/outpatient DSM-III-R 8.4 601.5 PANSS (pos./neg.) = 23.2/26.2
Wagner (1996) NR NR NR NR 70.0 70.0 Inpatient DSM-III-R 8.0 346.0 BPRS = 42.0
Wynn (2010) 44.6 38.6 12.8 14.6 79.4 66.7 Outpatient DSM-IV 24.3 NR BPRS = NR-overall; SANS = NR-overall
Zhang (2016) 31.3 33.6 13.4 14.2 57.4 56.5 Inpatient DSM-IV 11.73 584.7 PANSS (pos./neg.) = 15.1/16.2
Note: PANSS: Positive and Negative Syndrome Scale; SAPS: Scale for the Assessment of Positive Symptoms; SANS: Scale for the Assessment of Negative Symptoms; BRPS: Brief Psychiatric Rating Scale; NR: No report.

Moderators

Patient and study design characteristics were examined as potential moderators. For patient characteristics, duration of illness, mean age, percentage of males, years of education, medication dosage (chlorpromazine [CPZ] equivalents), and clinical status of the patient sample (inpatient vs outpatient) were coded for each study. Clinical status was coded as either inpatient (k = 11) or outpatient (k = 7) but, because a fair number of studies (k = 8) used SZ samples comprised of both inpatients and outpatients, potential moderating effects of combined inpatient and outpatient samples were also examined. For characteristics of the study design, we examined task complexity, interstimulus interval (ISI), mean vs peak amplitude measurement, and topographical location of the CNV (frontal vs central). Given that some research has suggested that blunting in patients with SZ is greatest at central sites but has also been observed at frontal sites (eg, 38, 51), we examined electrode site (ie, frontal vs central) as a potential moderator in a subset of studies (k = 21) that measured CNV amplitude differences between patients and HCs at frontal and/or central electrode sites. We were not able to examine CNV measurement window as a potential moderator because there were too few studies (k = 4) that reported ES for the early CNV component. See supplementary table 2 for descriptive statistics of study design characteristics.

Analyses

The primary variable of interest was the amplitude of the CNV for patients with SZ compared to HC subjects. For each study (k = 32), Hedge’s g was calculated based on reported means and SDs/SEs or parametric statistics (t or F values) comparing CNV amplitudes for patients and HCs. For each study, a single ES collapsed across conditions, electrode measurement sites, and groups (eg, inpatient/outpatient) was used for the primary analyses.

Random-effects models with restricted-information maximum likelihood estimation were used to calculate the overall weighted mean ES, SE, and 95% CI of CNV amplitude differences between patients and HCs across the studies. We employed z tests to examine if the overall effect was significantly different from zero. Heterogeneity in the distribution of ES was examined using the I2 statistic and Q test. Visual inspection of funnel plots and Egger’s regression test of funnel plot asymmetry were used to determine if publication bias was present. Studentized residuals were used to identify potential outlier and influential cases. The same procedure was used to estimate the weighted mean ES for group differences in reaction time, which were available in 22 of the 32 studies. Statistical comparisons between ES were made by examining whether CIs overlapped between the obtained ES for reaction time slowing and CNV blunting within the same subset of 22 studies.

To examine the impact of potential moderators, meta-regression analyses were employed using random-effects modeling. A single effect was included from each study in the random-effect models testing for moderation by duration of illness (k = 18), mean age of patients (k = 32), percentage of males (k = 32), years of education of patients (k = 18), medication dosage (k = 19), clinical status of sample (k = 26), task complexity (k = 32), ISI (k = 32), and amplitude measurement (k = 32). Because 10 of the 21 studies measured CNV amplitude group differences at both frontal and central electrode sites, robust variance estimation (RVE) was employed to account for within-study statistical dependencies. Thus, a total of 31 ES (19 central) were included in RVE analyses examining the effect of measurement electrode on the weighted mean ES of CNV amplitude group differences. Forest plots are provided to visualize the distribution of ES across the included studies.

We also quantified whether CNV blunting and reaction time slowing are distinct measures by calculating an estimate of the meta-analytic correlation between these 2 variables. Then, to determine if CNV blunting provided unique variance in the combined ES of the 2 variables above and beyond reaction time slowing alone, we entered the meta-analytic correlation into the following equation: C1+C22×(1+r), where C1 and C2 are the ES of reaction time slowing and CNV amplitude and r is the correlation between these 2 measures (see supplementary materials for details).

Results

Across the studies, the mean age of patients and controls was 32.35 (SD = 7.47) and 31.80 (SD = 7.91), respectively. Years of education was comparable between patients and controls, with patients having on average 11.89 (SD = 1.15) years of education and controls having 12.71 years (SD = 1.65). The mean distribution of sex was 74% males (SD = 17.71) within patient samples and 72% males (SD = 17.83) within control samples. The mean duration of illness was 9.17 years (SD = 5.86). The mean medication dosage was 422.36 mg (CPZ equivalent; SD = 353.73).

Mean Weighted ES

The weighted mean ES across the 32 included studies revealed a significant overall blunting of CNV amplitude in SZ patients compared to controls [ES = −0.80, 95% CI: −0.98, −0.62, P < 0.0001]. However, the distribution of ES was highly heterogenous [Q(31) = 79.23, I^2 = 64.17%, P < 0.0001]. Measures of influences (ie, studentized residuals) suggested that 2 studies were contributing a considerable amount to the observed heterogeneity.28,33 Specifically, a study by Strandburg et al33 found opposite CNV effects in patients and controls, with controls exhibiting “blunting” compared to patients with SZ (ES = 0.82). In contrast, the ES from the study by Oke et al28 was substantially larger than the rest of the included studies (ES = 3.49). Data were reanalyzed after removal of the 2 outlier articles, and the resultant weighted mean ES of the 30 remaining studies was large [ES = −0.79, 95% CI: −0.90, −0.68, P < 0.0001] with heterogeneity across the distribution of ES substantially reduced [Q(29) = 28.99, I^2 = 5.77%, P = 0.47] (see figure 2 for forest plot). Further, the examination of funnel plots and Egger’s coefficient bias test did not reveal a small study publication bias [intercept = −0.15, 95% CI: −1.42, −1.11, P = 0.82].

Fig. 2.

Fig. 2.

Forest plot of CNV effect sizes. *Note: RE = random effects, square size = weight, diamond size = effect size and CI.

Comparing CNV Blunting to Reaction Time Slowing

The weighted mean ES for reaction time across 22 studies revealed significant reaction time slowing in SZ patients compared to controls [ES = −1.20, 95% CI: −1.43, −0.96, P < 0.0001]. Similar to a recent meta-analysis of 76 studies examining reaction time slowing in SZ,52 the distribution of ES for reaction time slowing was highly heterogenous [Q(21) = 54.31, I^2 = 60.70%, P < 0.0001]. Within the same 22 studies from which reaction time data were obtained, the mean weighted ES for CNV blunting in SZ patients compared to controls was [ES = −0.87, 95% CI: −1.00, −0.74, P < .0001], which was homogenous across the observed ES [Q(21) = 20.82, I^2 = 4.60%, P = .47]. Although the ES estimate for reaction time slowing was somewhat larger than the ES estimate for CNV blunting, the CIs overlapped, indicating no significant difference between the 2 meta-analytic ES. In addition, the pooled ES of CNV blunting and reaction time slowing combined (r = 0.57) was found to be greater than the ES of reaction time slowing alone (r = 0.52; see supplementary materials for complete correlation coefficients). This provides evidence that CNV blunting accounts for unshared variance between reaction time slowing and psychomotor deficits in SZ.

Moderating Variables

Meta-regression analyses did not indicate that illness duration, patient age, patient years of education, percent male, medication dosage, clinical status, task complexity, amplitude measurement (peak and mean), or ISI were statistically significant moderators (all Ps > 0.10).

Regarding electrode sites, robust variance estimation indicated that the weighted mean ES of CNV amplitudes at central electrode sites was large [ES = −0.87, 95% CI: −1.01, −0.73, P < 0.0001] (see figure 3 for forest plot). By contrast, the weighted mean ES of CNV amplitudes at frontal electrode sites was medium [ES = −0.48, 95% CI: −0.75, −0.22, P < 0.01] (see figure 4 for forest plot). When comparing weighted mean ES of central and frontal electrode sites, analyses revealed that blunting in patients was significantly larger at central sites than those measured at frontal electrode sites [ΔES = −0.39, 95% CI: −0.62, −0.15, P < 0.01]. Further, these findings remained when conducting sensitivity analyses removing the 2 ES with inverted effects at frontal electrode sites (ie, controls having blunted CNV amplitude).34,41 Specifically, these analyses again revealed that blunting in patients was significantly larger at central sites than those measured at frontal electrode sites [ΔES = −0.27, 95% CI: −0.48, −0.07, P = 0.01].

Fig. 3.

Fig. 3.

Forest plot of CNV effect sizes at central electrode sites. *Note: RE = random effects, square size = weight, diamond size = effect size and CI.

Fig. 4.

Fig. 4.

Forest plot of CNV effect sizes at frontal electrode sites. *Note: RE = random effects, square size = weight, diamond size = effect size and CI.

Discussion

The CNV serves as a continuous measure of psychomotor processes that are well evidenced to be impaired in SZ.53,54 The current meta-analytic review revealed a reliable and robust blunting of CNV activity in patients compared to HCs (ES = −0.79). This finding is consistent with the broader literature demonstrating that deficits in attention and motor function are core features of SZ.1,3,53 The magnitude of CNV blunting was not statistically different than that of psychomotor slowing. Notably, the CI for CNV blunting from the current study (CI: −0.90, −0.68) overlapped with a large recent meta-analysis that observed an overall ES of 0.99 (CI: −1.12, −.86) for reaction time slowing in patients,52 providing further support that CNV blunting in SZ is comparable to that of reaction time slowing. Similarly, the magnitude of CNV blunting in patients is also comparable to other proposed ERP biomarkers, including the error-related negativity (ES = −0.96; CI: −1.09, −0.82),55 mismatch negativity (ES = −0.95; CI: −1.04, −0.85),56 P300 (ES = −0.74; CI: −0.78, −0.69),57 and N170 (ES = −0.64; CI: −0.78, −0.49).58 Notably, whereas both reaction time slowing and the aforementioned ERP components tend to have a wide degree of heterogeneity across ES,52,55–57 after removal of the 2 outlier studies, the magnitude of CNV blunting was highly consistent across studies. Given that the CNV is believed to be a neural index of several psychomotor processes (eg, alerting, anticipatory attention, and motor planning) involved in translating sensory information into a response (ie, reaction time), it is not surprising that the majority of variance explained is shared with reaction time slowing. It is important to note that CNV blunting reflects anticipatory attention and motor planning processes that contribute to reaction time slowing.59–61 CNV blunting and reaction time slowing are not analogs and it is important that there remains a modest unique effect of CNV blunting above and beyond reaction time. Furthermore, consistent with past work in other disorders,62 identifying additional measures that increase the predictive utility of psychomotor deficits in patients with SZ may allow for better classification of individuals based on these diagnostic measures. Because reaction time slowing is the observable endpoint of several partially distinct psychomotor processes, having biomarkers that index specific processes and neural generators (ie, CNV) may be particularly suited for this purpose.

There is a strong overlap between the putative neural generators of the CNV and brain regions implicated in prominent pathophysiological models of SZ, suggesting the CNV may serve as a valuable biomarker for the illness. Evidence from neuroimaging and lesion research indicates the putative cortical and subcortical generators of the CNV, including the prefrontal cortex, primary motor cortex, parietal cortex, SMA, anterior cingulate cortex, thalamus, basal ganglia, and cerebellum.15,63–66 Critically, all of these brain regions comprise neural networks that are well documented to be disturbed in SZ. For example, it has been argued that the CNV primarily reflects neuronal activity from a dopamine-mediated cortico-thalamo-striatal network (see 17,67), which is highly consistent with the dopamine theory of psychosis68,69 and the large body of evidence demonstrating disturbances in cortico-striatal-thalamic-cortical networks in SZ.70–74 Given that these networks are thought to be responsible for mediating the psychomotor processes considered to be indexed by the CNV, such as attention and motor control, it is possible that CNV blunting in SZ is a marker of disturbances within distinct regions implicated in this network or connectivity across different regions of this system when engaging in anticipatory response selection.

We also examined the effect(s) of potential moderators on the magnitude of CNV blunting. Moderation analyses indicated that blunted CNV amplitude in patients with SZ was not moderated by illness duration, patient age, sex distribution of sample, years of patient education, medication dosage, clinical status of the patient sample, task complexity, ISI, or amplitude measurement. Although medication dosage did not moderate the magnitude of CNV blunting, it is important to note that serval lines of evidence suggests that CNV amplitude may normalize after the onset of neuroleptics and symptom remission.75–77 Notably, electrode site (frontal vs central) was a significant moderator of the magnitude of CNV blunting. There has been considerable discussion within the literature regarding whether blunting of the CNV in SZ is primarily located at central or frontal electrode sites. Specifically, it has been argued that reduction of the CNV at central electrode sites is a stable marker of SZ and largely reflects motor preparation, whereas blunting at frontal sites is state dependent.27,29,78,79 Evidence from the present study supports previous impressions that CNV blunting is greatest at central sites (ES = −0.87) and may primarily reflect deficits in motor preparation.27 However, a medium ES was also observed for frontal sites in the current study (ES = −0.48). The discrepancy between past impressions and the current findings may suggest that reductions at frontal sites are present but between-group differences may not always be found, possibly due to small sample sizes used in the majority of studies.

Although there are clear strengths to current meta-analysis (ie, no evidence for publication bias), there are also limitations that warrant discussion. First, we were unable to directly compare ES for early and late components of the CNV in SZ. This was largely due to studies either not directly assessing the early CNV component or only reporting statistics for the late component when there were no significant group differences observed in the early measurement window. Second, patient samples were highly similar across the included studies. Specifically, the majority of included studies reflect CNV blunting in middle-aged patients with SZ that have a fairly chronic illness course, calling into question the generalizability of the current meta-analytic findings to first episode and other subpopulations of SZ.

The current meta-analysis highlights several key areas for future research. First, it will be important for future well-powered research to compare the CNV blunting in the early vs late window to better elucidate the psychomotor processes that are impaired in psychosis. Similarly, future work should focus on examining the CNV in youth at clinical high risk for the illness, which would provide valuable confirmatory evidence for the CNV as a potential biomarker for the progression of the disorder. Lastly, given that the present findings indicate that CNV blunting and reaction time slowing are measuring partially distinct processes, it will be important to determine the relative predictive utility of these measures for distinguishing between patients with SZ and HCs in a high-powered study. Investigating these questions may provide valuable insight into the underlying neural mechanisms of attention and response preparation deficits in SZ, as well as provide a novel biomarker of the integrity of cortico-striatal-thalamic-cortical networks in psychosis.

Funding

This work was supported by the National Institutes of Health (V.A.M., grant numbers R01MH094650, R21/R33MH103231; K.J.O. and B.K., grant number T32NS047987).

Supplementary Material

sbaa043_suppl_Supplementary_Materials
sbaa043_suppl_Supplementary_Tables

Acknowledgments

None.

Conflict of Interest

The authors have declared that there are no conflicts of interest in relation to the subject of this study.

References

  • 1. Abboud R, Noronha C, Diwadkar VA. Motor system dysfunction in the schizophrenia diathesis: neural systems to neurotransmitters. Eur Psychiatry. 2017;44:125–133. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Bortolato B, Miskowiak KW, Köhler CA, Vieta E, Carvalho AF. Cognitive dysfunction in bipolar disorder and schizophrenia: a systematic review of meta-analyses. Neuropsychiatr Dis Treat. 2015;11:3111–3125. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Walther S, Mittal VA. Motor system pathology in psychosis. Curr Psychiatry Rep. 2017;19(12):97. [DOI] [PubMed] [Google Scholar]
  • 4. Hughes ME, Fulham WR, Johnston PJ, Michie PT. Stop-signal response inhibition in schizophrenia: behavioural, event-related potential and functional neuroimaging data. Biol Psychol. 2012;89(1):220–231. [DOI] [PubMed] [Google Scholar]
  • 5. Kappenman ES, Luck SJ, Kring AM, et al. Electrophysiological evidence for impaired control of motor output in schizophrenia. Cereb Cortex. 2016;26(5):1891–1899. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Luck SJ, Mathalon DH, O’Donnell BF, et al. A roadmap for the development and validation of event-related potential biomarkers in schizophrenia research. Biol Psychiatry. 2011;70(1):28–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Qiu YQ, Tang YX, Chan RC, Sun XY, He J. P300 aberration in first-episode schizophrenia patients: a meta-analysis. PLoS One. 2014;9(6):e97794. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Walter WG. Contingent negative variation: an electric sign of sensorimotor association and expectancy in the human brain. Nature. 1964;230:380–384. [DOI] [PubMed] [Google Scholar]
  • 9. Tecce JJ. Contingent negative variation (CNV) and psychological processes in man. Psychol Bull. 1972;77(2):73–108. [DOI] [PubMed] [Google Scholar]
  • 10. Nagai Y, Critchley HD, Featherstone E, Fenwick PB, Trimble MR, Dolan RJ. Brain activity relating to the contingent negative variation: an fMRI investigation. Neuroimage. 2004;21(4):1232–1241. [DOI] [PubMed] [Google Scholar]
  • 11. Verleger R, Wauschkuhn B, van der Lubbe R, Jaśkowski P, Trillenberg P. Posterior and anterior contribution of hand-movement preparation to late CNV. J Psychophysiol. 2000;14(2):69–86. [Google Scholar]
  • 12. Hillyard SA. Relationships between the contingent negative variation (CNV) and reaction time. Physiol Behav. 1969;4(3):351–357. [Google Scholar]
  • 13. Cancro R, Sutton S, Kerr J, Sugerman AA. Reaction time and prognosis in acute schizophrenia. J Nerv Ment Dis. 1971;153(5):351–359. [DOI] [PubMed] [Google Scholar]
  • 14. Gómez CM, Flores A, Ledesma A. Fronto-parietal networks activation during the contingent negative variation period. Brain Res Bull. 2007;73(1–3):40–47. [DOI] [PubMed] [Google Scholar]
  • 15. Rosahl SK, Knight RT. Role of prefrontal cortex in generation of the contingent negative variation. Cereb Cortex. 1995;5(2):123–134. [DOI] [PubMed] [Google Scholar]
  • 16. Rebert CS. Cortical and subcortical slow potentials in the monkey’s brain during a preparatory interval. Electroencephalogr Clin Neurophysiol. 1972;33(4):389–402. [DOI] [PubMed] [Google Scholar]
  • 17. Brunia CH, van Boxtel GJ. Wait and see. Int J Psychophysiol. 2001;43(1):59–75. [DOI] [PubMed] [Google Scholar]
  • 18. Leuthold H, Jentzsch I. Neural correlates of advance movement preparation: a dipole source analysis approach. Brain Res Cogn Brain Res. 2001;12(2):207–224. [DOI] [PubMed] [Google Scholar]
  • 19. Andreasen NC, Pierson R. The role of the cerebellum in schizophrenia. Biol Psychiatry. 2008;64(2):81–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Grace AA. Dysregulation of the dopamine system in the pathophysiology of schizophrenia and depression. Nat Rev Neurosci. 2016;17(8):524–532. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Bleuler E. Dementia Praecox or the Group of Schizophrenias. New York, NY: International Universities Press;1950. [Google Scholar]
  • 22. Ungvari GS. The Wernicke-Kleist-Leonhard school of psychiatry. Biol Psychiatry. 1993;34(11):749–752. [DOI] [PubMed] [Google Scholar]
  • 23. Giuliano AJ, Li H, Mesholam-Gately RI, Sorenson SM, Woodberry KA, Seidman LJ. Neurocognition in the psychosis risk syndrome: a quantitative and qualitative review. Curr Pharm Des. 2012;18(4):399–415. [DOI] [PubMed] [Google Scholar]
  • 24. Seidman LJ, Shapiro DI, Stone WS, et al. Association of neurocognition with transition to psychosis: baseline functioning in the second phase of the North American prodrome longitudinal study. JAMA Psychiatry. 2016;73(12):1239–1248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Jahshan C, Heaton RK, Golshan S, Cadenhead KS. Course of neurocognitive deficits in the prodrome and first episode of schizophrenia. Neuropsychology. 2010;24(1):109–120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. van den Bosch RJ. Contingent negative variation and psychopathology: frontal-central distribution, and association with performance measures. Biol Psychiatry. 1983;18(6):615–634. [PubMed] [Google Scholar]
  • 27. Verleger R, Wascher E, Arolt V, Daase C, Strohm A, Kömpf D. Slow EEG potentials (contingent negative variation and post-imperative negative variation) in schizophrenia: their association to the present state and to Parkinsonian medication effects. Clin Neurophysiol. 1999;110(7):1175–1192. [DOI] [PubMed] [Google Scholar]
  • 28. Oke S, Saatchi R, Allen E, Hudson NR, Jervis BW. The contingent negative variation in positive and negative types of schizophrenia. Am J Psychiatry. 1994;151(3):432–433. [DOI] [PubMed] [Google Scholar]
  • 29. Wagner M, Rendtorff N, Kathmann N, Engel RR. CNV, PINV and probe-evoked potentials in schizophrenics. Electroencephalogr Clin Neurophysiol. 1996;98(2):130–143. [DOI] [PubMed] [Google Scholar]
  • 30. Walther S, Strik W. Motor symptoms and schizophrenia. Neuropsychobiology. 2012;66(2):77–92. [DOI] [PubMed] [Google Scholar]
  • 31. Hovington CL, Lepage M. Neurocognition and neuroimaging of persistent negative symptoms of schizophrenia. Expert Rev Neurother. 2012;12(1):53–69. [DOI] [PubMed] [Google Scholar]
  • 32. Rockstroh B, Cohen R, Berg P, Klein C. The postimperative negative variation following ambiguous matching of auditory stimuli. Int J Psychophysiol. 1997;25(2):155–167. [DOI] [PubMed] [Google Scholar]
  • 33. Strandburg RJ, Marsh JT, Brown WS, et al. Reduced attention-related negative potentials in schizophrenic adults. Psychophysiology. 1994;31(3):272–281. [DOI] [PubMed] [Google Scholar]
  • 34. Klein C, Müller M, Heinz A, Rockstroh B. Slow cortical potentials in schizophrenic patients during association formation. J Psychophysiol. 1994;. 8(4),314–327. [Google Scholar]
  • 35. Dias EC, Butler PD, Hoptman MJ, Javitt DC. Early sensory contributions to contextual encoding deficits in schizophrenia. Arch Gen Psychiatry. 2011;68(7):654–664. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Smid HG, Martens S, de Witte MR, Bruggeman R. Inflexible minds: impaired attention switching in recent-onset schizophrenia. PLoS One. 2013;8(10):e78062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Wynn JK, Horan WP, Kring AM, Simons RF, Green MF. Impaired anticipatory event-related potentials in schizophrenia. Int J Psychophysiol. 2010;77(2):141–149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Zhang D, Zhao Y, Liu Y, Tan S. Perception of the duration of emotional faces in schizophrenic patients. Sci Rep. 2016;6:22280. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Li Z, Deng W, Liu X, et al. Contingent negative variation in patients with deficit schizophrenia or bipolar I disorder with psychotic features: measurement and correlation with clinical characteristics. Nord J Psychiatry. 2015;69(3):196–203. [DOI] [PubMed] [Google Scholar]
  • 40. Chaillou AC, Giersch A, Bonnefond A, Custers R, Capa RL. Influence of positive subliminal and supraliminal affective cues on goal pursuit in schizophrenia. Schizophr Res. 2015;161(2–3):291–298. [DOI] [PubMed] [Google Scholar]
  • 41. Eikmeier G, Lodemann E, Olbrich HM, Pach J, Zerbin D, Gastpar M. Altered fronto-central PINV topography and the primary negative syndrome in schizophrenia. Schizophr Res. 1993;8(3):251–256. [DOI] [PubMed] [Google Scholar]
  • 42. Ford JM, Roach BJ, Miller RM, Duncan CC, Hoffman RE, Mathalon DH. When it’s time for a change: failures to track context in schizophrenia. Int J Psychophysiol. 2010;78(1):3–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Ruchkin DS, Canoune HL, Johnson R Jr, Ritter W. Working memory and preparation elicit different patterns of slow wave event-related brain potentials. Psychophysiology. 1995;32(4):399–410. [DOI] [PubMed] [Google Scholar]
  • 44. Guterman Y, Josiassen RC, Bashore TE, Johnson M, Lubow RE. Latent inhibition effects reflected in event-related brain potentials in healthy controls and schizophrenics. Schizophr Res. 1996;20(3):315–326. [DOI] [PubMed] [Google Scholar]
  • 45. Leuthold H, Sommer W, Ulrich R. Preparing for action: inferences from CNV and LRP. J Psychophysiol. 2004;18(2/3):77–88. [Google Scholar]
  • 46. van Boxtel GJ, Brunia CH. Motor and non-motor aspects of slow brain potentials. Biol Psychol. 1994;38(1):37–51. [DOI] [PubMed] [Google Scholar]
  • 47. Turetsky BI, Dress EM, Braff DL, et al. The utility of P300 as a schizophrenia endophenotype and predictive biomarker: clinical and socio-demographic modulators in COGS-2. Schizophr Res. 2015;163(1–3):53–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Brunia CH, van Boxtel GJ, Böcker KB, Kappenman E, Luck S. Negative slow waves as indices of anticipation: the Bereitschaftspotential, the contingent negative variation, and the stimulus-preceding negativity. In: Luck S, Kappenman E, eds. The Oxford Handbook of Event-related Potential Components. New York, NY: Oxford University Press, Inc.; 2012:196–199. [Google Scholar]
  • 49. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–269. [DOI] [PubMed] [Google Scholar]
  • 50. Kim SY, Park JE, Lee YJ, et al. Testing a tool for assessing the risk of bias for nonrandomized studies showed moderate reliability and promising validity. J Clin Epidemiol. 2013;66(4):408–414. [DOI] [PubMed] [Google Scholar]
  • 51. Kirenskaya AV, Kamenskov MY, Myamlin VV, Novototsky-Vlasov VY, Tkachenko AA. The antisaccade task performance deficit and specific CNV abnormalities in patients with stereotyped paraphilia and schizophrenia. J Forensic Sci. 2013;58(5):1219–1226. [DOI] [PubMed] [Google Scholar]
  • 52. Fioravanti M, Bianchi V, Cinti ME. Cognitive deficits in schizophrenia: an updated metanalysis of the scientific evidence. BMC Psychiatry. 2012;12:64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Nuechterlein KH, Green MF, Calkins ME, et al. Attention/vigilance in schizophrenia: performance results from a large multi-site study of the consortium on the genetics of schizophrenia (COGS). Schizophr Res. 2015;163(1–3):38–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Kappenman ES, Kaiser ST, Robinson BM, et al. Response activation impairments in schizophrenia: evidence from the lateralized readiness potential. Psychophysiology. 2012;49(1):73–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. Martin EA, McCleery A, Moore MM, Wynn JK, Green MF, Horan WP. ERP indices of performance monitoring and feedback processing in psychosis: a meta-analysis. Int J Psychophysiol. 2018;132(Pt B):365–378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Erickson MA, Ruffle A, Gold JM. A meta-analysis of mismatch negativity in schizophrenia: from clinical risk to disease specificity and progression. Biol Psychiatry. 2016;79(12):980–987. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Jeon YW, Polich J. Meta-analysis of P300 and schizophrenia: patients, paradigms, and practical implications. Psychophysiology. 2003;40(5):684–701. [DOI] [PubMed] [Google Scholar]
  • 58. McCleery A, Lee J, Joshi A, Wynn JK, Hellemann GS, Green MF. Meta-analysis of face processing event-related potentials in schizophrenia. Biol Psychiatry. 2015;77(2): 116–126. [DOI] [PubMed] [Google Scholar]
  • 59. Bervoets C, Docx L, Sabbe B, et al. The nature of the relationship of psychomotor slowing with negative symptomatology in schizophrenia. Cogn Neuropsychiatry. 2014;19(1):36–46. [DOI] [PubMed] [Google Scholar]
  • 60. Morrens M, Hulstijn W, Van Hecke J, Peuskens J, Sabbe BG. Sensorimotor and cognitive slowing in schizophrenia as measured by the symbol digit substitution test. J Psychiatr Res. 2006;40(3):200–206. [DOI] [PubMed] [Google Scholar]
  • 61. Luck SJ, Gold JM. The construct of attention in schizophrenia. Biol Psychiatry. 2008;64(1):34–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62. Cavanagh JF, Meyer A, Hajcak G. Error-specific cognitive control alterations in generalized anxiety disorder. Biol Psychiatry Cogn Neurosci Neuroimaging. 2017;2(5):413–420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63. Hultin L, Rossini P, Romani GL, Högstedt P, Tecchio F, Pizzella V. Neuromagnetic localization of the late component of the contingent negative variation. Electroencephalogr Clin Neurophysiol. 1996;98(6):435–448. [DOI] [PubMed] [Google Scholar]
  • 64. Ioannides AA, Fenwick PB, Lumsden J, et al. Activation sequence of discrete brain areas during cognitive processes: results from magnetic field tomography. Electroencephalogr Clin Neurophysiol. 1994;91(5):399–402. [DOI] [PubMed] [Google Scholar]
  • 65. Liu MJ, Fenwick PB, Lumsden J, Lever C, Stephan KM, Ioannides AA. Averaged and single-trial analysis of cortical activation sequences in movement preparation, initiation, and inhibition. Hum Brain Mapp. 1996;4(4):254–264. [DOI] [PubMed] [Google Scholar]
  • 66. Hamano T, Lüders HO, Ikeda A, Collura TF, Comair YG, Shibasaki H. The cortical generators of the contingent negative variation in humans: a study with subdural electrodes. Electroencephalogr Clin Neurophysiol. 1997;104(3):257–268. [DOI] [PubMed] [Google Scholar]
  • 67. Fan J, Kolster R, Ghajar J, et al. Response anticipation and response conflict: an event-related potential and functional magnetic resonance imaging study. J Neurosci. 2007;27(9):2272–2282. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68. Howes OD, Nour MM. Dopamine and the aberrant salience hypothesis of schizophrenia. World Psychiatry. 2016;15(1):3–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69. Maia TV, Frank MJ. An integrative perspective on the role of dopamine in schizophrenia. Biol Psychiatry. 2017;81(1):52–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70. Anticevic A, Cole MW, Repovs G, et al. Characterizing thalamo-cortical disturbances in schizophrenia and bipolar illness. Cereb Cortex. 2014;24(12):3116–3130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71. Bracht T, Schnell S, Federspiel A, et al. Altered cortico-basal ganglia motor pathways reflect reduced volitional motor activity in schizophrenia. Schizophr Res. 2013;143(2-3):269–276. [DOI] [PubMed] [Google Scholar]
  • 72. Martino M, Magioncalda P, Yu H, et al. Abnormal resting-state connectivity in a substantia nigra-related striato-thalamo-cortical network in a large sample of first-episode drug-naive patients with schizophrenia. Schizophr Bull. 2017;44(2):419–431. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73. Mittal VA, Bernard JA, Northoff G. What can different motor circuits tell us about psychosis? An RDoC perspective. Schizophr Bull. 2017;43(5):949–955. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74. Walther S, Stegmayer K, Federspiel A, Bohlhalter S, Wiest R, Viher PV. Aberrant hyperconnectivity in the motor system at rest is linked to motor abnormalities in schizophrenia spectrum disorders. Schizophr Bull. 2017;43(5):982–992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75. Knott J, Peters J, Robinson M, Smith A, Andreasen N. Contingent negative variation (CNV) in schizophrenic and depressed-patients. Electroencephalogr Clin Neurophysiol. 1976;40(3):329–330. [Google Scholar]
  • 76. Rizzo PA, Albani GF, Spadaro M, Morocutti C. Brain slow potentials (CNV), prolactin, and schizophrenia. Biol Psychiatry. 1983;18(2):175–183. [PubMed] [Google Scholar]
  • 77. Simlai J, Nizamie SH, Khess CR. A study of contingent negative variation and post-imperative negative variation: search for state and trait electrophysiological markers in schizophrenia. East Asian Arch Psychiatry. 2010;20(2):62–68. [PubMed] [Google Scholar]
  • 78. Heimberg DR, Naber G, Hemmeter U, et al. Contingent negative variation and attention in schizophrenic and depressed patients. Neuropsychobiology. 1999;39(3):131–140. [DOI] [PubMed] [Google Scholar]
  • 79. Smid HG, Westenbroek JM, Bruggeman R, Knegtering H, Van den Bosch RJ. Abnormal externally guided movement preparation in recent-onset schizophrenia is associated with impaired selective attention to external input. Psychiatry Res. 2009;170(1):75–81. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

sbaa043_suppl_Supplementary_Materials
sbaa043_suppl_Supplementary_Tables

Articles from Schizophrenia Bulletin are provided here courtesy of Oxford University Press

RESOURCES