Intranasal Oxytocin May Improve High-Level Social Cognition in Schizophrenia, But Not Social Cognition or Neurocognition in General: A Multilevel Bayesian Meta-analysis

Abstract

While there is growing interest in the potential for intranasal oxytocin (IN-OT) to improve social cognition and neurocognition (ie, nonsocial cognition) in schizophrenia, the extant literature has been mixed. Here, we perform a Bayesian meta-analysis of the efficacy of IN-OT to improve areas of social and neurocognition in schizophrenia. A systematic search of original research publications identified randomized controlled trials (RCTs) of IN-OT as a treatment for social and neurocognitive deficits in schizophrenia for inclusion. Standardized mean differences (SMD) and corresponding variances were used in multilevel Bayesian models to obtain meta-analytic effect-size estimates. Across a total of 12 studies (N = 273), IN-OT did not improve social cognition (SMD = 0.07, 95% credible interval [CI] = [−0.06, 0.17]) or neurocognition (SMD = 0.12, 95% CI = [−0.12, 0.34]). There was moderate between study heterogeneity for social cognition outcomes$({\tau }_s=0.12)$. Moderator analyses revealed that IN-OT had a significantly larger effect on high-level social cognition (ie, mentalizing and theory of mind) compared to low-level social cognition (ie, social cue perception) (b = 0.19, 95% CI = [0.05, 0.33]). When restricting our analysis to outcomes for high-level social cognition, there was a significant effect of IN-OT (SMD = 0.20, 95 % CI = [0.05, 0.33]) but the effect was not robust to sensitivity analyses. The present analysis indicates that IN-OT may have selective effects on high-level social cognition, which provides a more focused target for future studies of IN-OT.

Introduction

Due to converging evidence in animals and healthy human populations, oxytocin (OT) has been identified as potentially having therapeutic properties in schizophrenia.^1,2 As a result, several randomized controlled trials (RCTs) have investigated the efficacy of intranasal oxytocin (IN-OT) on reducing psychiatric symptoms in schizophrenia.^3–10 However, results have been mixed. Indeed, our recent Bayesian meta-analysis (n [total participants] = 238, P-C.B. and D.R.W.) indicated that IN-OT was not effective for treating the positive, negative, or general symptoms of individuals with schizophrenia.¹¹ In fact, we found moderate evidence in favor of the null hypothesis of no effect (via Bayes factors) for negative symptoms, for which IN-OT was hypothesized as having potential therapeutic properties.¹ Nonetheless, optimism remains that IN-OT may ameliorate the social and neurocognitive impairments frequently seen in schizophrenia.¹² The present meta-analysis thus investigates the effects of IN-OT on these deficits in individuals with schizophrenia.

Cognitive deficits in schizophrenia were once attributed to antipsychotic drugs or thought of as relatively unimportant.¹³ However, cognitive deficits are now considered essential features of schizophrenia:¹⁴ they are predictive of developing schizophrenia in healthy individuals,¹⁵ are present in attenuated forms in unaffected family members,¹⁶ and likely have a genetic component.¹⁷ These cognitive deficits include deficits in both social cognition¹⁸ and neurocognition.¹⁹ Although both areas of cognition overlap,²⁰ they have differential effects on a several outcomes²¹ and can be considered distinct constructs.²⁰ Impaired social cognitive domains in schizophrenia include emotion recognition²² and theory of mind.²³ Deficits in social cognition in general, and theory of mind in particular,²¹ have been linked to worse functional outcomes (eg, community functioning). The neurocognitive deficits include well-described deficits in working memory,²⁴ executive function,²⁵ and verbal memory,²⁶ with verbal proficiency being strongly associated with functional outcomes.²¹ Despite their clinical importance, no currently available pharmacotherapies are effective at ameliorating any of these deficits.^27,28

Several lines of evidence support the role of OT in cognition, including in animal models of schizophrenia^29,30 and autism.³¹ Notably, OT has been shown to have restorative effects on cognition in these rodent models.³² Furthermore, there is substantial evidence supporting the role of the OT system in social behaviors,³³ which are core deficits observed in individuals with schizophrenia³⁴ and other disorders.³⁵ Accordingly, IN-OT has been tested as a potential treatment for various neuropsychiatric disorders in which symptomology includes cognitive deficits.³⁶ In schizophrenia, RCTs have investigated the effectiveness of IN-OT on a range of cognitive deficits and improvement has been reported in several areas including verbal memory,³⁷ recognition of certain emotions,³⁸ and theory of mind.⁹ However, several outcomes were measured within each study and positive effects have been inconsistent between studies. In other words, studies that reported at least 1 positive effect also included several nonsignificant findings and in some cases a majority of the outcomes showed no significant effect.

In addition, 3 studies have explicitly investigated the differential effects of IN-OT on low- vs high-levels of social cognition.^39–41 Low-level domains rely on reflexive responses, including motor resonance and affect sharing, both of which allow for shared experiences.³⁴ In contrast, high-level social cognition involves facial and voice perception, as well as mentalizing (eg, theory of mind).^34,41 In support of this distinction, the levels appear to be modulated by related but distinct neural mechanisms and are differentially impaired in individuals with schizophrenia.^42,43 For example, high-level domains show impairment, while low-level processes are relatively less affected.³⁴ Each of the 3 studies that investigated IN-OT and social cognitive levels reported limited effects on low-level, but there was a more consistent positive effect on high-level social cognition.

Considering these divergent results, we conducted a meta-analysis investigating IN-OTs effects on cognition in schizophrenia to provide: (1) overall meta-analytic effect size estimates for social cognition and neurocognition; (2) an indication of efficacy for low-level and high-level social cognition; and (3) a measure of heterogeneity between studies and areas of cognition. We used a Bayesian framework for 2 reasons. First, we wanted to incorporate some prior information into our models (specifically for the variances between and within studies), in order to facilitate hierarchical shrinkage of study estimates which helps in improving precision of the obtained overall estimates.⁴⁴ Second, a Bayesian framework allows assessment of the evidence for the null hypothesis (ie, no effect of OT) via Bayes-factors,^45,46 an approach that is not available when fitting models in a frequentist framework. Additionally, we attempted to explain heterogeneity between outcomes through conducting moderator analyses, assessed publication bias, and performed sensitivity analyses via leave-one-out methods.

Methods and Materials

Inclusion Criteria and Search Strategy

The current study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PRISMA).⁴⁷ As a guide for exclusion/inclusion criteria, we used the Patient, Intervention, Comparator, and Outcome (PICO) strategy (patients: diagnosed with schizophrenia or closely related disorder (eg, schizoaffective disorder); intervention: IN-OT administration; comparator: placebo; outcome: psychometric assessments of social cognition or neurocognition. Additionally, only randomized and double-blinded RCTs were included.

The initial search was conducted using the Web of Science and PubMed for peer-reviewed research articles that were published up to July 28, 2016. The search term “oxytocin AND schizophrenia AND intranasal” produced 143 articles from both databases. Of these, 39 were excluded as duplicates leaving 104 original articles. Of those, 94 were then excluded for reasons including: nonhuman animal subjects, noncognitive outcome, review or meta-analysis, or different intervention (not IN-OT). While Davis et al⁵ investigated IN-OT in relation to cognition in schizophrenia, it was excluded because IN-OT was not administered on the day of testing. An additional 2 articles were identified through searching reference lists in relevant reviews (figure 1 for flow chart). In total, we identified 12 original research articles that met the inclusion criteria. The meta-analytic estimates for social cognition were computed from 10 studies and 67 outcomes, while the estimates for neurocognition were obtained from 3 studies and 10 outcomes. Summary statistics and data from these studies were initially extracted by D.R.W., and then verified by P-C.B. When sufficient information was not provided, we used the application WebPlotDigizier⁴⁸ to extract the necessary data from figures. We did not observe any inconsistencies in the retrieved data, thus investigators of the primary studies were not contacted.

Fig. 1.

Flowchart illustrating the literature search process in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Risk of bias in individual studies was assessed with the Delphi method.⁴⁹ This assessment was constructed based on agreement between experts in RCT evaluation. The 8-item scale used in the present study included randomization, blinding procedures, homogeneity of baseline characteristics, eligibility criteria, and reporting of statistical information. Included items were given one point, whereas a definitive no or insufficient information was scored as zero. The highest score possible was 8, which indicated that a primary study included all items. P-C.B. and D.R.W. independently performed the coding, and disagreement was settled through discussion until consensus was reached.

Outcomes and Data Extraction

The primary studies used a variety of psychometric assessments. Due to this variation, consensus about coding outcomes was reached by P-C.B., D.R.W., and J.D.W. Together, we determined the constructs being measured were generally consistent between studies that measured social cognition. As implemented in 3 of the primary studies,^39–41 we categorized social cognitive outcomes as either low- or high-level. Outcomes that required rapid inference and limited integration of contextual information were labeled as low-level cognition (eg, social cue detection). The most commonly used assessments for measuring low-level social cognition were the Reading the Mind in the Eyes Test⁵⁰ and variants of the Facial Affect Recognition test (table C1 in supplementary appendix C).⁵¹ In contrast, high-level social cognition required participants to combine multilayered social information into reflective judgments.⁴¹ Measures of high-level social cognition included the Theory of Mind Picture Stories Task⁵² and The Awareness of Social Inference Test⁵³ (see table C1 in supplementary appendix C). In the current analysis, only performance-based data were included. We thus excluded self-report measures including the Empathy Quotient.⁵⁴ the Interpersonal Reactivity Index,⁵⁵ and the Ambiguous Intention Hostility Questionnaire.⁵⁶

In contrast to social cognition, the domains assessed in neurocognition were more variable. Neurocognitive outcomes were reported in 3 studies,^4,37,57 each of which used different assessment tools: The Repeatable Battery for Assessment of Neuropsychological Status,⁵⁸ the California Verbal Learning Task,⁵⁹ the Letter Number Sequence,⁶⁰ Digit Span, and Digit Symbol Coding⁶¹ (see table C2 in supplementary appendix C).

We extracted relevant data and demographic information from all of the primary studies. Additionally, we obtained study characteristics for moderator analyses that were of theoretical interest, most notably of which was low- vs high-level social cognition. Both animal and human studies have indicated that OT may have sexually dimorphic effects.^62,63 Furthermore, OT levels are predictive of negative symptoms in males with schizophrenia.⁶⁴ As such, the proportion of males in each study was used as a potential moderator. The OT system has been associated with fear processing,^65,66 such as recognition⁶⁷ and neural responses^68,69 in healthy subjects, as well as schizophrenic individuals.³⁸ We thus investigated whether IN-OT has selective effects on fear recognition when compared to all other emotions combined. As OT dosage varied between studies, this was included as a potential moderator. Some studies administered IN-OT daily for a given period of time, whereas others administered treatment exclusively on the day of testing. We thus considered administration interval (daily vs day of testing) as a moderator. Demographic variables, including mean age of participants and study country, were also obtained for moderator analyses. All quality scores were at least 6, indicating that all studies were of high quality. Due to this lack of variability, we did not explore the possibility that higher quality studies produced larger effects.

Statistical Analysis

To estimate the influence of IN-OT on cognition in individuals with schizophrenia, we used 2 types of effect sizes. First, the standardized mean difference (SMD; also known as Hedges’ g) was computed for post-treatment scores. Second, the standardized mean change using raw score standardization (SMCR) was computed.⁷⁰ The SMCR, unlike SMD, not only contrasts treatment and control group, but also controls for possible differences in the pre-treatment values (when present, see supplementary appendix A for computational details). The SMCR requires knowledge of the correlations of outcomes across time points to compute its variance. As these correlations were rarely reported in the primary studies, they were set to r = .5 in order to be similar to correlations typically obtained in schizophrenia studies (eg, see the Schizo_PANSS data in the R package Surrogate⁷¹). Because only 6 out of 12 studies reported pre-treatment values, SMD was our primary effect size. SMCR estimates were also reported to demonstrate the robustness of our results.

As studies differ more or less in their experimental design, outcome assessment, and treatment properties, there will likely be some heterogeneity between outcomes of different studies and samples.⁷² Additionally, because we extracted multiple outcomes from a single study, within study dependency must be considered.⁷³ Therefore, a 3-level hierarchical model was assumed allowing us to estimate the pooled effect sizes and corresponding credible (ie, Bayesian confidence) intervals, as well as 2 additional sources of variation: (1) the variance ${\mathit{\tau }}_{\mathit{s}}^2$ between outcomes of different studies/samples; and (2) the residual variance ${\mathit{ô}}_{\mathit{e}}^2$ between different outcomes within the same study/sample.⁷⁴ In contrast, other meta-analytic approaches for dealing with multiple outcomes from the same study would strategically select one outcome per study or combine outcomes within a particular study.⁷⁵ Using the current approach, however, allows for richer inferences as information is obtained from multiple levels⁷⁶ and reduces researcher degrees of freedom such that our meta-analytic estimates are not dependent upon possible selection bias.⁷⁷ To analyze the influence of potential moderators on between study heterogeneity, meta-regression models⁷² were applied separately for each moderator.

For mathematical details on the applied Bayesian meta-analytic models as well as specification of priors see supplementary appendix B. Priors were chosen to be only weakly informative so that their influence on the meta-analytic estimates were relatively small.⁷⁸ We report 95% credible intervals (CIs) and Bayes factors. Given the observed data, there is a 95% probability that the parameter is contained within a 95% CI.⁷⁹ In the present situation, the Bayes factors (BF₀₁) quantified evidence in favor of the null hypothesis of no effect of IN-OT⁸⁰ and provides the ratio of likelihood between the null (H₀) to alternative hypotheses (H₁). A BF₀₁ equal to 3 indicates the null is 3 times more likely than the alternative hypothesis (interpretation of evidence: 1–3 = anecdotal; 3–10 = moderate; 10–30 = strong).⁸¹ Because Bayesian methods are less commonly used and understood, we also report P values in order to reach a broader audience.

To explore the possibility of publication bias,^82,83 potential for funnel plot asymmetry was examined visually and also statistically using Egger’s test.⁸⁴ Sensitivity of the estimates was examined with leave-one-out analyses. The alpha-level of all statistical tests was set to á = .05. All computation was done in R.⁸⁵ The package metafor⁸⁶ was used for the effect size computation, while the package brms⁸⁷—allowing to fit Bayesian multilevel models (including meta-analytic models) using Stan⁸⁸—was used for the actual analysis.

Results

Study Characteristics

Across all 13 patient samples reported in 12 RCTs, adult individuals with schizophrenia or schizoaffective disorder (N = 273) were randomized into experimental groups, including: between subjects or crossover designs. Dosages of OT ranged between 10–80 IU per day and sample sizes ranged from 5 to 52 patients. Summarized over all samples, the mean age was 37 ± 0.3 years and 88% of the participants were male. All studies were published in English. Detailed study characteristics can be found in table C1 in supplementary appendix C.

Meta-analysis

The obtained outcomes for social cognition and neurocognition are visualized in the forest plots in figures 2 and 3, respectively. Hierarchical Bayesian meta-analyses revealed no significant effect of IN-OT on either social cognition (SMD = 0.07, CI = [−0.06, 0.17], P = .238) or neurocognition (SMD = 0.12, CI = [−0.12, 0.34], P = .209). Moreover, Bayes factors indicated moderate evidence in favor of the null hypothesis of no effect for social cognition (BF₀₁ = 3.85), while evidence was inconclusive for neurocognition (BF₀₁ = 1.77). Heterogeneity between studies was moderate for both social cognition $({ô}_s=0.12)$ and neurocognition $({ô}_s=0.14)$, both of which are on the scale of the data (SMD). Results obtained for the SMCR estimates, which control for pre-treatment values, were similar (table 1).

Fig. 2.

Forest plot of SMD effect sizes for social cognition. Square sizes represent study weights. Filled diamonds represent summary effect sizes. Goldman et al (2011a) and (2011b) denote outcomes of their nonpolydipsic and polydipsic group, respectively. Ekman, variation of Facial Affect Recognition test; RMET, Reading Mind in Eyes Test; FEEST, Facial Expression of Emotions Task; TFPT, The Faux Pas Task; TMST, The Movie Stills Task; TFBPST, The False Belief Picture Sequencing Task; TASIT I, The Awareness of Social Inference Test (part 1); TASIT II, The Awareness of Social Inference Test (part 2); TASIT III, The Awareness of Social Inference Test (part 3); EPTT, Emotional Perspective Taking Task; half-PONS, Half Profile of Nonverbal Sensitivity; FAR, Facial Affect Recognition; IPT, Interpersonal Perception Task; ER-40, Emotion Recognition 40; TMPST, Theory of Mind Picture Stories Task; DANVA, Diagnostic Analysis of Non-Verbal Accuracy; HT, Hinting Task; SMT, standardized mean difference.

Fig. 3.

Forest plot of SMD effect sizes for neurocognition. Square sizes represent study weights. Filled diamonds represent summary effect sizes. RBANS, Repeatable Battery for the Assessment of Neuropsychological Status; CVLT, California Verbal Learning Task; LNS, Letter Number Sequence; DS, Digit Span; DSC, Digit Symbol Coding.

Table 1.

Main Results of the Meta-analysis

Cognition Type	Effect size	# Samples	# Outcomes	Estimate	95% CI	P Value	BF01	${\tau }_{\mathit{s}}$	95% CI of ${\tau }_{\mathit{s}}$	${\tau }_{\mathit{e}}$	95% CI of ${\tau }_{\mathit{e}}$
Social cognition	SMD	11	67	0.07	[−0.06, 0.17]	.238	3.85	0.12	[0.01, 0.27]	0.12	[0.02, 0.21]
	SMCR	5	35	0.06	[−0.13, 0.27]	.545	4.91	0.13	[0.01, 0.39]	0.08	[0.00, 0.22]
Neurocognition	SMD	3	10	0.12	[−0.12, 0.34]	.209	1.77	0.14	[0.01, 0.48]	0.09	[0.01, 0.22]
	SMCR	2	7	0.12	[−0.36, 0.53]	.503	1.99	0.23	[0.01, 0.90]	0.13	[0.01, 0.42]

Note: SMD, standardized mean difference; SMCR, standardized mean change with raw score standardization; CI, credible interval; ${\tau }_{\mathit{s}}$, between-sample standard deviation; ${\tau }_{\mathit{e}}$, residual standard deviation; P values are 2-tailed; BF₀₁ indicates the evidence in favor of the null hypothesis that oxytocin has no effect on social cognition / neurocognition.

Moderator Analyses

The following variables were analyzed as potential moderators of social cognition outcomes: social cognition level, recognition of fear vs other emotions, single administration vs chronic administration, study country, mean age of patients in the study, percentage of male patients in the study, and OT dosage per ingestion/day. To achieve an acceptable amount of statistical power, moderator variables were analyzed separately. As shown in table 2, only social cognition level (low vs high) explained a significant amount of heterogeneity between outcomes. The effect of IN-OT on low-level social cognition was essentially zero (SMD = 0.01, CI = [−0.11, 0.11]), while there was a small effect on high-level social cognition (SMD = 0.20, CI = [0.05, 0.33]). However, there was still a large amount of variability within high-level social cognition outcomes (figure 4). Moderator analyses of neurocognition outcomes were not performed, as neurocognition was assessed in only 3 studies.

Table 2.

Moderator Analysis of SMD Effect Sizes for Social Cognition

Moderator	Contrast	Estimate	95% CI	P Value
Social cognition level	High-level vs low-level	0.19	[0.05, 0.33]	.010
Recognition of specific emotions	Fear vs other	0.20	[−0.08, 0.46]	.169
Administration schedule	Daily vs day of testing	−0.08	[−0.34, 0.22]	.516
Country	United States vs other	−0.07	[−0.33, 0.17]	.597
Mean age EG (y)		0.00	[−0.01, 0.01]	.971
Mean age CG (y)		0.00	[−0.01, 0.01]	.997
% Males EG		0.36	[−0.21, 0.88]	.181
% Males CG		0.34	[−0.28, 0.91]	.257
OT dose per ingestion (IU)		0.00	[−0.01, 0.01]	.834
OT doses per day (IU)		0.00	[−0.01, 0.01]	.882

Note: SMD, standardized mean difference; CI, credible interval; EG, experimental group receiving oxytocin; CG, control grouping receiving placebo; OT, oxytocin; P values are 2-tailed. Categorical moderators are dummy coded.

Fig. 4.

Forest plot of SMD effect sizes for social cognition separated after cognition level. Square sizes represent study weights. Filled diamonds represent summary effect sizes. Goldman et al (2011a) and (2011b) denote outcomes of their nonpolydipsic and polydipsic group, respectively. Abbreviations of outcomes are explained in figure 2. The filled in diamonds equal the summary effect size. The squares correspond to how much a given effect contributes the summary effect size.

Sensitivity Analysis

In order to investigate the robustness of the meta-analytic estimates, additional analyses were performed. Egger’s test indicated funnel plot asymmetry in the SMD effect sizes for social cognition (t(65) = −2.04, P = .048). However, according to the funnel plots displayed in supplementary appendix D, the amount of asymmetry appears to be rather small especially when compared to the amount of overall heterogeneity between outcomes. No evidence could be found for significant funnel plot asymmetry in the SMD effect sizes for neurocognition (t(8) = 0.287, P = .781).

To investigate the influence of single studies on the obtained meta-analytic effects, leave-one-out analyses were conducted. The differences between SMD effects obtained by the leave-one-out analysis and the complete analysis were within (lower SMD estimate = 0.04, upper SMD estimate = 0.09) for social cognition and within [0.11, 0.16] for neurocognition (see supplementary appendix D for details). When restricting the outcomes to high-level social cognition, removal of Woolley et al⁴¹ reduced the estimate by 0.08 (SMD = 0.12, CI = [−0.05, 0.28]) which was no longer significant (see table D2 in supplementary appendix D).

Discussion

The present study is the first to provide meta-analytic estimates of IN-OTs effects on various aspects of cognition in schizophrenia. We used a fully Bayesian framework to determine that IN-OT does not reliably improve most aspects of cognition and there was moderate evidence in favor of the null hypothesis (ie, no effect of IN-OT) for social cognition. We did find that the level of social cognition was a significant moderator of the effects of IN-OT and, when assessing the effects of IN-OT on high-level social cognition alone, we determined that IN-OT may have beneficial effects. Through using a multilevel approach, variance within studies (eg, same sample but different outcomes) and between studies was shown to be moderate and similar in magnitude.

Social Cognition

When looking at the overall estimate for social cognition, the Bayes factor indicated that there was moderate evidence in favor of the null hypothesis (ie, no effect of IN-OT). Notably, the SMD and SMCR meta-analytic estimates were consistent and the overall SMD estimates were robust to leave-one-out analyses. However, given previous work in schizophrenia and in healthy individuals, there is growing realization that social cognition is not a singular construct. Indeed, several researchers have put forth and tested this very hypothesis.^39–41 We thus categorized social cognitive domains into low-level and high-level social cognition in order to evaluate whether IN-OT was especially effective for improving high-level social cognition. While there was an effect in the hypothesized direction, the effect was small in size (SMD = 0.20) and the amount of heterogeneity present suggests that interpreting the effect should be done with caution. Additionally, we conducted sensitivity analyses and determined the effect was primarily attributed to 1 study.⁴¹ This suggests the evidence is inconsistent for IN-OT improving high-level social cognition in schizophrenia.

Neurocognition

While social cognition was reported most often, 3 of the primary studies measured aspects of neurocognition.^4,37,57 In each of these studies different assessments and treatment protocols were used. Furthermore, one study was a brief report to the editors,⁵⁷ and another examined neurocognitive outcomes to ensure they were not adversely affected by IN-OT.³⁷ Indeed, previous evidence showed that OT had amnestic effects on neurocognition, in which performance in memory tasks worsened compared to controls.^89,90 In the present analysis, the Bayes factor for the overall estimate indicated that the data were inconclusive. Although both SMD and SMCR meta-analytic estimates were consistently nonsignificant, having few outcomes resulted in low precision (ie, wide intervals). Indeed, the 3 included studies were small and tested the effects of IN-OT on many different aspects of neurocognition. These limitations in combination with an inconclusive Bayes factor and previous reports of reductions in neurocognitive ability suggests that more studies are needed to thoroughly characterize IN-OTs effects on these deficits in individuals with schizophrenia.

Moderator Analyses

In an attempt to explain between study variance, we conducted several moderator analyses. Despite extensive data suggesting sexual dimorphic effects of OT in animals and human,^62,63 the proportion of males in a given sample did not moderate the effects of IN-OT. Based on animal and human studies showing OT may have selective effects on fear processing,^65,66 we compared recognition of fear to all other emotions combined. This comparison indicated that IN-OT did not have selective effects on fear recognition. It should be noted that additional contrasts between emotions could have been made. However, other emotions (eg, happy) were not assessed often, which limited our ability to make further comparisons. While there was variability in dosage, administration interval (daily vs day of testing), ages of participants, and country in which the study was performed, none of these variables explained a significant amount of between study variance. However, the sample sizes to test these moderators are necessarily smaller than the main analysis, which resulted in a reduction in statistical power.⁹¹ As such, it would be premature to draw definitive conclusions from these analyses.

Limitations

Our findings suggest that IN-OT does not produce a consistent effect on cognition in schizophrenia, which may be due to patient heterogeneity such as differing medications or genetic predispositions. It should also be noted that the exact mechanisms of how and if IN-OT reaches cerebrospinal fluid and brain tissue remain unclear.^92,93 In support of this notion, Quintana et al⁹⁴ put forth a variety of potential delivery routes. Additionally, absorption by olfactory and trigeminal fibers can only occur if the treatment is able to travel beyond the nasal valve. Further complicating matters is the differing dosages and treatment protocols used in clinical trials.⁹⁵ For instance, IN-OT was administered by technicians in some studies,^40,96 whereas in others the patients were responsible for administration.^4,41,97 As a result, each of the primary studies likely had a differential ability for IN-OT to enter the CSF. Indeed, Quintana et al⁹⁴ noted this variability and suggested to work towards standardizing protocols, which would allow for more thoroughly characterizing the effects of IN-OT. Together, these sources of heterogeneity may explain the absence of findings.

There are many reasons why IN-OT may not have large effects on cognition in schizophrenia.⁹⁸ For instance, while animal research provides a theoretical rationale for investigating the therapeutic properties of OT,^99–101 invasive manipulations are often used in animal research. Based on invasive approaches, the estimated effect of OT administration on social behavior in rodents is much larger (ie, SMD = 0.74) than the effects of IN-OT reported in the nonclinical human literature (SMD = 0.28).¹⁰² Additionally, whether and to what extent IN-OT reaches the brain in humans is unclear,⁹³ whereas direct routes of administration are typically used in animals. Furthermore, the location OT receptors are well characterized in many species,^103–105 whereas in humans the precise locations remain unclear.¹⁰⁶ OT also has complex relationships with neurotransmitters,^107,108 which may further limit translation from animals to individuals with schizophrenia. Commonly used antipsychotic drugs inhibit dopamine and/or serotonin transmission by acting as receptor antagonists.^109,110 However, access to dopamine D2 receptors is necessary for OT-induced pro-social behavior in animals.^107,111 Additionally, several subtypes of serotonin receptors modulate OT secretion, with antagonists reducing OT secretion.¹⁰⁸ These possibilities should be investigated in future studies.

Because studies with small sample sizes typically have inadequate power to detect small effects, significant findings are at risk of being a false positive¹¹² or of reflecting greatly inflated effect sizes.¹¹³ Thus, when considering the estimate for high-level social cognition (SMD = 0.20), it is important to keep in mind that large samples will be needed in future trials to detect this size of an effect. To have 80% power, between-subject designs need 393 per group and within-subject designs need 199 participants. Based on the primary (within subjects) study that had the largest sample size per group (n = 35),⁹⁶ statistical power to detect an effect size of 0.20 was 21%. Because this was computed based on the largest sample, actual power for the other studies was much lower. Using this power estimate, however, indicates that conducting many such experiments would result in only 21% successfully rejecting the null hypothesis if there really is an effect. As such, the vast majority would not produce a positive effect, which demonstrates why replicability has proven to be a difficult task. The current meta-analysis allowed for increased power,¹¹⁴ in which we detected a possible effect of IN-OT in high-level social cognition.

The present study has several limitations. First, our meta-analytic estimates are dependent upon published findings, which makes our analysis sensitive to publication bias.¹¹⁵ However, several of the primary studies reported nonsignificant results and some reported only nonsignificant results.⁴ In combination with a visually symmetric funnel plot (supplementary appendix figure D1), this suggests that the literature does not demonstrate substantial publication bias for positive findings.⁸⁴ Second, there was not one instance in which dosage and administration (daily vs day of testing) were consistent. We conducted moderator analyses on these variables, each of which produced a negligible effect (P value > .52). However, it might also be the case that we had insufficient power, which is difficult to assess in a multilevel framework.¹¹⁶ Third, schizophrenia is a heterogeneous disorder in which a spectrum of symptoms exists^117,118 and we were unable to analyze differing treatment effects among individuals with certain symptom profiles. This could be especially important for future research to consider because cognitive deficits are associated with negative symptoms.²⁰ Fourth, OT likely has complex relationships with antipsychotic drugs. We were not able to investigate this possibility, however, because descriptions of medication regimens were too varied in the primary studies. Fifth, although the constructs of low- and high-level social cognition are not new,¹¹⁹ evidence for distinct neural mechanisms and neuropsychological assessments refined enough to differentiate between these constructs are just emerging.^42,43 As such, this might provide a theoretical foundation from which future research can be built. Sixth, it should be noted that we did not pre-register a meta-analysis protocol. This would have reduced research degrees of freedom such as our coding scheme for levels of social cognition. However, we clearly stated our rationale and conducted leave-one-out analyses, both of which added validity to our conclusions. Seventh, the present meta-analysis was based on only 12 RCTs with a combined sample size of 273 patients, which is likely too small to allow for a final evaluation of the effectiveness of IN-OT on improving cognition in schizophrenia.

Implications

While the effect of IN-OT on high-level social cognition is considered small, it is comparable to effect sizes reported in meta-analyses on healthy individuals. Furthermore, the practical implications should be considered. For instance, high-level social cognition (eg, theory of mind) has been linked to real world social outcomes^120,121 and is connected to functional outcomes in schizophrenia.²⁰ As such, even a small effect might have clinical significance.¹²² It should be noted that the effect was not only significant, but was also significantly larger than that of IN-OT for low-level social cognition. This suggests IN-OT may have selective effects. There was also heterogeneity between studies that measured high-level domains, including symptom profiles, dosage size, and assessments used. While some studies used still images of cartoons,^7,9 for instance, others used video clips of actual interactions between humans.^39,41 The fact that we were able to detect a signal through the noise is promising, particularly because high-level cognitive deficits are both predictive of outcomes²⁰ and resistant to available treatments.³⁶ Furthermore, by elucidating the variability in treatment protocols used in the primary studies, it is clear that efforts should be made towards standardization.⁹⁴ This would not only allow for more thoroughly characterizing IN-OTs therapeutic properties in empirical studies, but would also facilitate higher powered moderator analyses in quantitative reviews.

Conclusion

In conclusion, our results suggest that OT does not reliably improve broadly defined areas of social cognition and neurocognition. It remains unclear whether these findings are just due to absence of evidence or really indicate evidence of absence. In support for the latter, there was moderate evidence in favor of the null hypothesis of no effect for the overall estimate of social cognition, which—according to animal and human research—had strong rationale for producing a positive effect. In contrast, when specifying social cognitive domains as either low- or high-level, there was a small effect of IN-OT on high-level social cognition. The divergent results between high-level social cognition and social cognition in general may be reason for concern. Alternatively, this might be considered further evidence for social cognitive domains being modulated by distinct neural mechanisms and IN-OT having selective effects on certain of these mechanisms. The present analysis indicates that IN-OT may have selective effects on high-level social cognition, which provides a more focused target for future studies of IN-OT.

Supplementary Material

Supplementary material is available at Schizophrenia Bulletin online.

Funding

This work was supported by the Career Development Award # CX000758 from the United States Department of Veterans Affairs, Office of Research and Development, Clinical Science Research and Development program. The funders did not play any role in study design, data collection and analysis, decision to publish, or preparation of the article.