Cognitive deficits in problematic internet use: a meta-analysis of 40 studies

Abstract

Background and aims

Excessive use of the internet is increasingly recognized as a global public health concern. Individual studies have reported cognitive impairment in problematic internet use (PIU), but have suffered from various methodological limitations. Confirmation of cognitive deficits in PIU would support the neurobiological plausibility of this disorder. The aim of this study was to conduct a rigorous meta-analysis of cognitive performance in PIU from case-control studies; and to assess the impact of study quality, main type of online behaviour (e.g. gaming), and other parameters on the findings.

Methods

Systematic literature review was conducted of peer-reviewed case-controlled studies comparing cognition in PIU (broadly defined) to healthy controls. Findings were extracted and subjected to a meta-analysis where at least four publications existed for a given cognitive domain of interest.

Results

The meta-analysis comprised 2922 participants across 40 studies. Compared to controls, PIU was associated with significant impairment in inhibitory control (Stroop task Hedge’s g = 0.53[SE 0.19-0.87], Stop-signal task g = 0.42[0.17-0.66], Go/No-Go task g = 0.51[0.26-0.75]), decision-making (g=0.49[0.28-0.70]), and working memory (g=0.40[0.20-0.82]). Whether or not gaming was the predominant type of online behavior did not significantly moderate the observed cognitive effects; nor did age, gender, geographical area of reporting, or the presence of co-morbidities.

Conclusions

Problematic internet use (PIU) is associated with decrements across a range of neuropsychological domains, irrespective of geographical location, supporting its cross-cultural and biological validity. These findings also suggest a common neurobiological vulnerability across PIU behaviors, including gaming, rather than a dissimilar neurocognitive profile for internet gaming disorder.

Introduction

Since its inception in the 1980s, the Internet has become a global phenomenon (1–3). Some adolescents and adults develop a problem controlling their use of the internet, leading to marked functional impairment (e.g. lower quality of life, worse scholastic outcomes, and occupational difficulties) (4). Historically, the term ‘Internet addiction disorder’ started appearing in the mid-nineties (1–3) to describe a maladaptive pattern of use of online resources that shared the characteristics of an addictive or compulsive disorder. Since then, the diagnostic criteria, assessment tools, and conceptual formulation of internet addiction have been controversial (5,6). Theoretically different views on problematic use of the internet exist, as exemplified by the terms referred to, e.g. compulsive internet use, problematic internet use, internet addiction, etc. The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (7) featured Internet Gaming Disorder (IGD) in Section III, as a condition in need of further study, but did not include the more general disorder of problematic internet use. The DSM-5 highlighted that Internet Gaming Disorder appeared to be most common in male adolescents, aged 12-20 years (7).

The concept of problematic internet use (PIU) was coined to avoid classification with addictions until more about the disorder was understood (8,9). It has been noted that a broad range of excessive online behaviours are associated with marked functional impairment as well as with profound psychiatric sequalae, including in adolescents (10), adults (11), and mixed samples of both (12). Based on empirical evidence, we define PIU as excessive online activities likely to be associated with marked functional impairment, including compulsive online buying, gambling, cybersex, as well as excessive use of online streaming and social media that have addictive, impulsive and/or compulsive elements (11,13). Age may influence the presentation of PIU and its comorbidities. For example, one study found that attention-deficit hyperactivity disorder (ADHD) and social anxiety were associated with PIU in young adults; whereas generalized anxiety disorder and obsessive-compulsive disorder were associated with PIU in older adults (14). Thus, PIU can occur in younger and older individuals but may present differently as a function of age. The debate is still ongoing as to whether PIU should be classified as an addictive, impulse control (5) or obsessive-compulsive related disorder (15,16).

Understanding of the neurobiological underpinnings of a given mental disorder is vital for optimising disease models, classification, and treatment approaches; as well as in understanding how it may relate to other disorders. In the case of excessive use of the internet, research in this area has the additional utility of helping to confirm or refute its validity. Currently, little is known about the neurocognitive determinants of PIU. Examining the cognitive performance of people suffering from PIU to identify deficits (i.e. significantly worse performance compared against matched healthy controls) can provide insights into the neuropsychological mechanisms underpinning the disorder, and possible overlap with other psychiatric conditions. Conceptually, as noted above, PIU may share parallels with behavioural addiction, incorporating features such as escalating use over time, loss of control, concealing excessive use from others, failed attempts to cut back, and psychological distress when/if prevented from using the Internet (3,17). In integrating research on PIU phenomena, the Interaction of Person-Affect-Cognition-Execution (I-PACE) model was developed by Brand and colleagues (18). Within this conceptual framework, reductions in executive functioning and inhibitory control contribute to engagement in online behaviours, leading to gratification, and ultimately contributing to the emergence and persistence of PIU.

Despite growing numbers of published case-control studies examining cognition in this context, there is a paucity of rigorous meta-analyses, from which to draw firm conclusions and examine potential moderators. In a meta-analysis restricted to internet gaming disorder and one cognitive domain, a significant decrement was found for response inhibition compared to controls (19). Current models of PIU suggest that a broader range of cognitive failures may contribute including top–down inhibitory control, working memory and decision-making (20). The aim of the current study was to conduct a rigorous systematic review and meta-analysis of cognitive findings in PIU from case-control studies, including in adolescents and adults, reported in the peer-reviewed literature. We hypothesized, based on findings from individual studies and parallels between PIU and other related disorders, like problematic gambling, that the condition would be associated with marked impairments across the above cognitive domains.

Methods

Our meta-analysis protocol followed the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) guidelines (21) and was pre-registered electronically and published online on the PROSPERO International prospective register of systematic reviews [Available from: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42017080405].

Search strategy

Our search and screening strategy is outlined in figure 1. The search string was determined by consensus amongst the co-authors. Pubmed search was conducted with the following string: ["cognitive" OR "cognition" OR "memory" OR "executive" OR "attention" OR "decision-making" OR "gambling task" OR "inhibition" OR "stroop" OR "stop-signal" OR "go no go" OR "go/no-go" OR "gng" AND "internet use" OR "internet addiction" OR "gaming addiction" OR "PIU" OR "PUI" OR "internet gaming disorder"]. The initial search yielded 2908 results. The majority of these were excluded based on reading of the title and abstract, due to being out of scope (e.g. papers not measuring cognition, without a suitable control group, or unrelated to problematic internet use). This yielded 138 possibly eligible papers for inclusion. We then undertook a consensus meeting involving three members of the study team and examined full texts to exclude papers that were out-of-scope; references of full text documents were also screened for further papers within scope.

Figure 1.

Inclusion criteria

We included all studies that a) were published in scholarly peer-reviewed journals between 1995 and October 2017; b) were written in English or provided an English translation; c) examined a cognitive domain that was also measured in at least three other studies (i.e. sufficient N for valid meta-analysis); d) examined cognitive measures of participants with PIU (used in its wider meaning to include the full spectrum of ‘addictive use of the internet’, ‘problematic internet use’ and ‘internet gaming disorder’) versus healthy controls and d) included necessary information to calculate effect sizes. Where a given paper had not reported necessary information to calculate effect sizes, the study team contacted the paper’s authors via email to request this information.

Exclusion criteria

We excluded studies that a) did not report cognitive measures; b) used non-standard cognitive tasks (those tailored to a particular study where independent replication would not have been possible; and/or those not focusing on a recognized cognitive domain); c) did not have a healthy comparison group; d) lacked the required measures for meta-analysis (and such information was not provided within 4 weeks by the paper’s authors); and e) were published only in the grey literature (including conference papers, non-peer reviewed publications, doctoral theses etc.; due to these sources not necessarily being subject to the same journal level rigorous peer-review procedures as non-grey literature).

Data collection and analysis

Data were extracted from the original papers or were provided by the authors of each study. Information from the included studies was recorded in an electronic spreadsheet and different types of data were extracted from each study including (a) a geographical determinant in which the data collection occurred; (b) key demographics of the participants (age as categorized by mean age reported in the sample: Children 0-12, Youth 12-24, Adults 24-55, Older people ≥55; gender distribution in the sample as “male only”, “female only”, or “mixed”); (c) operationalization of PIU including instrument used and cut-off variant; (d) reported psychiatric co-morbidities in the sample; (e) effects of PIU on cognitive measures; f) quality scores. The quality assurance control was performed independently by two psychiatrists (KI, SRC; Cohen's kappa 0.96), who then met together to arrive at a consensus. All papers in scope were assessed against the quality standard individually and received a score between 0-10 (for quality scoring details see Supplement TABLE S4).

The full list and references of studies that entered the meta-analysis are reported in supplemental TABLE S1. Data were analysed using statistical software R version 3.4.2. Meta-analysis was performed using packages of “robumeta” and “metafor” (22). To provide a more generalizable model estimate, a random-effects model (REM) was used in all cases. The R code used for this analysis is shared in the supplement, to support reproducible research. To compare PIU and control groups in terms of quantitative measures of cognitive performance we used mean scores and standard deviation to calculate standardized mean difference measures, which were utilized to produce random-effects models for each different cognitive domain under investigation. Statistical significance was defined as p<0.05 two-tailed throughout, and standard effect sizes were also reported. Moderator analysis was conducted to examine potential effects of the following on the results: age, gender (i.e. ‘males only’ vs ‘mixed’), presence of co-morbidities (i.e. psychiatric co-morbidities in the sample versus not), quality of study, whether or not online gaming was the predominant type of online activity (IGD vs PIU) and geographical area of reported study. Publication bias was assessed using Regression tests for Funnel Plot Asymmetry (23) and, where appropriate, the trim and fill method (24). Heterogeneity was quantified using Tau^2 and Q-tests.

Results

The number of data studies and total pooled sample sizes used in the meta-analysis are summarized in Table 1. Sufficient suitable data were found for meta-analysis of the following cognitive domains (tasks): motor inhibition (go/no go), pre-potent motor inhibition (stop-signal), decision-making (Cambridge Gambling Task, Iowa Gambling Task, Game of Dice, and Balloon Analogue Risk), working memory (Digit Span, Spatial Working Memory), and discounting. The mean (standard deviation) [range] of quality scores for the included studies, expressed as percentage of maximum, was: 68% (21%) [2-9] (see supplementary Table S4 for full details). Effects of scores in moderation analysis are reported later. Most studies (approximately 80%) screened for affective disorders and substance misuse using validated instruments, whereas relatively few (<10%) screened for impulse control disorders and gambling disorder. Another limitation of the extant data was that most studies were conducted in relatively young adults hence the association between PIU and cognition in older age groups was not addressable.

Table 1.

Domain	N Studies	PIU Total N	Control Total N	Model Estimate (SE)	Sig.
Attentional Inhibition (Stroop)	16	362	361	0.53 (0.175)	**
Motor Inhibitory Control (GNG)	14	330	333	0.51 (0.167)	***
Motor Inhibitory Control (SST)	5	149	279	0.42 (0.12)	***
Decision-Making	7	188	349	0.54 (0.14)	***
Working Memory	4	126	254	0.40 (0.17) †	*
Discounting	4	98	93	1.03 (0.26)	††
Total	40	1248	1674	-	-

^†Adjusted model estimate after trim and fill method was applied due to publication bias

^††Not further analysed due to publication bias and other methodological limitations

Figure 2a shows results from the meta-analysis of motor inhibitory control domain, where it can be seen that PIU was associated with significant impairment on go/no go and stop-signal tasks, versus controls, with small-medium effect sizes (Hedge’s g=0.51 and 0.42 respectively). Figure 2b shows meta-analytic results for the domains of attentional inhibition (Colour-word Stroop), decision-making, and working memory. PIU was associated with significant impairment versus controls across all three domains with small-medium effect sizes (Hedge’s g=0.53, 0.49, and 0.51 respectively). The discounting domain was excluded and not considered further due to methodological limitations (see supplement).

Figure 2.

Evidence of publication bias was observed for the working memory domain, but the finding retained statistical significance when the trim and fill approach was used. Homogeneity metrics are presented in full in supplemental TABLE S2. High heterogeneity was identified in Stroop studies and low to moderate heterogeneity was found for the other examined cognitive domains.

Age, gender, presence of co-morbidities, whether or not gaming was the predominant online activity, and geographical area were not significant moderating factors in any of the cognitive domains examined (all p>0.05 non-corrected). In some cases analysis was not possible due to lack of comparison groups. For example Stroop and Stop-Signal studies had only been performed in youth (adolescents and young adults) and Stroop studies were only performed in populations lacking co-morbidities. Quality of study was a significant moderating variable in SST (p = 0.032) with all higher quality studies (20,25,26) [Quality mean = 9/10] reporting smaller and non-statistically significant effects, and the two relatively lower quality studies (27,28) [Quality mean = 7/10] reporting higher and statistically significant effects. Study quality was not a significant moderator for the other cognitive domains. More details on moderator analysis results are presented in the supplement Table S5.

Discussion

This is the first study to amass all available information from case-control studies of cognitive performance in people with problematic internet use (PIU). We defined PIU as excessive online activities likely to be associated with marked functional impairment, including compulsive online buying, gambling, cybersex, as well as excessive use of online streaming and social media that have addictive, impulsive and/or compulsive elements. In meta-analysis, PIU was associated with significant cognitive deficits in attentional inhibition, motor inhibition (and pre-potent motor inhibition), decision-making, and working memory, in line with our a priori hypothesis and supporting recent conceptualizations of PIU that implicate cognitive dysfunction in its pathophysiology (17,18). These findings were not significantly moderated by whether or not online gaming was the predominant form of online behaviour, nor by geographical site, age, gender, or co-morbidities. Study quality did not significantly moderate the results, except for evidence of lesser stop-signal impairment for studies that were of higher quality. These neurocognitive results support the existence of underlying fronto-striatal dysfunction in PIU, and highlight the need for international collaborations using standardized measures to further elucidate its precise neurobiological underpinnings and the specificity of deficits in given domains. These findings also suggest a common neurobiological vulnerability across PIU behaviors, including gaming.

Two previous systematic reviews examined ‘higher order’ meta-cognitive constructs that are relevant for internet gaming disorder, including escapism, social identity and acceptance and beliefs about game reward (29,30), without providing a quantitative measure of cognition nor covering in detail neurocognitive performance. Therefore, in the wider context of existing literature, our study advances our knowledge of the neurocognitive aspects of PIU. One previous meta-analysis of response inhibition was conducted in gaming disorder, which reported significant impairment (19). The current study extends beyond this prior meta-analysis by also considering the impact of study quality, and including a much larger range of available data. Problematic internet users are characterized by elevated behavioural impulsivity and compulsivity (11,15), which are characteristics of a wide range of psychiatric disorders, including ADHD, OCD, impulse control, and substance use disorders. The majority of studies in this meta-analysis screened for mainstream mental disorders (such as affective disorders [78%] or substance misuse [80%]) using validated instruments. However, very few indeed used appropriate screening tools to identify co-morbid impulse control disorders (e.g. gambling disorder, ADHD) [7.5%]. As such, the current meta-analysis cannot fully assess the contribution of comorbid impulsive disorders to the observed cognitive deficits. Data elsewhere suggest that cognitive problems are more pronounced in PIU individuals with comorbid impulse control disorders (31). Nonetheless, the results of this meta-analysis demonstrate that people with PIU have measurable deficits versus controls in cognitive performance, which may have implications for day-to-day functioning, even if they partly stem from unmeasured co-morbid disorders.

Another important aspect to consider is the effects of age and symptom duration in PIU. While we did not find a moderating effect of participant age on the cognitive findings, most studies in this meta-analysis were conducted in relatively young participants. Excessive use of the internet can occur in older people (14), and this is a neglected area of research. Studies did not generally report symptom duration, so the current analysis cannot evaluate the extent to which cognitive problems may predate symptoms (perhaps reflecting vulnerability) as opposed to arising due to chronic engagement with internet-related activities. In a longitudinal (3-months) exposure of smartphone-naïve young adults to heavy smartphone use resulted in performance decrease in arithmetic accuracy and increase in concern for appropriateness (32). While these results are preliminary, they may demonstrate the capacity of PIU to cause cognitive and behavioral changes.

Furthermore, we need to highlight that ~85% of the studies included in the meta-analysis were based in centres of predominantly Asian communities. This limits the generalizability of the results to a degree, nevertheless, there was no evidence from the moderator analysis that the geographical area of study impacted the observed cognitive effects. Previous work has established that PIU is a global issue (4), and our meta-analysis supports the notion that the neurocognitive signature of PIU is not influenced by ethnicity. This is in line with previous work, which found that the profiles of PIU were similar across two separate geographical and cultural settings (USA and South Africa) (11). Finally, IQ measures are known to influence neurocognitive performance, which means that IQ is a parameter which needs to be controlled for in comparison studies. However, only 22.5% of studies included direct measures of IQ, and therefore, it is unclear whether differences between PIU participants and control participants may have been caused by differences in IQ. Robust research should include such measures in the future.

Limitations

Some studies were excluded due to use of non-standard cognitive domains, use of non-standard variants of common neuropsychological tasks (those not enabling replication by other groups); or insufficient numbers of other papers in the given domain to facilitate meta-analysis (a full list of those are presented in supplemental TABLE S3). For example, a number of studies utilized variants of the Stroop test with internet related stimuli; pooling effects of ‘Stroop’ studies and ‘Internet Stroop’ studies was not scientifically justified, because they evaluate different cognitive processes (colour-word inhibition versus attentional bias for internet-related stimuli, the latter measured via a heterogeneous spread of stimulus types and methodological approaches). By excluding these studies we do not mean to suggest that they are not extremely relevant for understanding PIU; but rather, the technique of meta-analysis is not well suited to examining non-standardized cognitive tasks, and is not suitable when few independent studies exist for a given cognitive domain. Lastly, we opted for a broad operational definition of PIU; however, we recognize that further research is needed to better define and characterize PIU and its composite behaviours.

Summary and recommendations for future studies

The current meta-analysis provides firm evidence that PIU (defined broadly and operationally) is associated with cognitive impairments in motor inhibitory control, working memory, Stroop attentional inhibition, and decision-making. These findings were not moderated by age, gender, geographical location, or by whether the predominant online activity was gaming or not. This analysis constitutes a vital first step towards a better understanding of PIU, supporting its existence as a biological plausible entity associated with dysfunction of fronto-striatal brain circuitry, and with clinical implications for people affected by PIU. The extent to which the identified cognitive deficits were present prior to PIU, or rather stemmed from engaging in such problematic behaviors, cannot be addressed within the confines of this cross-sectional data analysis. Longitudinal studies are needed to address the issue of direction of effect and causality. Based on cognitive findings in other settings, such as in the context of substance use and behavioural addiction (gambling), we theorize that some cognitive problems associated with PIU may constitute vulnerability markers; whereas others may be more associated with chronicity (17).

This analysis also serves to highlight vital next steps needed in future data papers, to further elucidate the specificity of the findings and their nature. This should include clarification of the role of IQ, the specific problematic behaviors involved beyond gaming, comorbid disorders that were seldom screened for (ADHD, impulse control disorders including gambling disorder), examining a broader range of ages and other cultural settings, and employing optimized designs to maximise study quality. The review also identifies several cognitive domains that have yet to be extensively or adequately examined in PIU, such as facial processing, set-shifting, verbal recall, sustained attention, discounting, reflection-impulsivity, and executive planning.

Box 1. Recommendations for Future Cognitive Investigations of PIU.

• Salient demographic characteristics of the sample (each study group) should be described including age, gender, education levels, and ethnicity.

• Specific problematic behaviors on the internet should be included, as this enables diagnostic specification of type of PIU, e.g. gaming, gambling, sex, shopping, social networking, streaming media.

• Group differences in general intelligence should be ruled out using a suitable IQ test.

• When considering cognitive tests to include in a study, due consideration should be given to validation of tests in other settings and how easy it would be for other groups to attempt to replicate the findings.

• When describing cognitive results, inclusion of mean, standard deviation, and sample size in each group, is extremely valuable. For example, when using graphs, this information should also be included in a footer in precise numerical form.

• Co-occurring comorbidities should be identified including mainstream mental disorders but also impulse control disorders using suitable screening and diagnostic methods.

Figure 3.

Meta-analysis funnels plots by cognitive domain; ‘z’ and ‘p’ values reported from Regression Test for Funnel Plot Asymmetry (mixed-effects meta-regression model). Evidence of publication bias identified in the domains of Discounting and working memory. Trim and fill method was used although effect size changed only for working memory (as indicated by the blue dotted line [non-corrected effect size 0.51])