Abstract
INTRODUCTION:
Problematic Internet Use (PIU) is the inability to control the amount of time spent on the Internet. Research indicates that abnormalities in reward sensitivity, sensitivity to punishment, and impulse control drive addictive behaviors such as substance abuse and gambling disorders, but it is unclear whether this is also the case in PIU.
METHODS:
Behavioral tasks and scales were completed by 62 participants (32 PIU individuals and 30 no-PIU individuals) to assess reward sensitivity, sensitivity to punishment, as well as inhibitory function and impulse control. Measures administered included Go/No-Go, delay discounting, Behavioral Inhibition/Activation (BIS/BAS) scales and the Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ).
RESULTS:
The PIU group endorsed greater reward sensitivity and punishment sensitivity as indexed by the SPSRQ. However, there were no group differences with regards to delay discounting, performance in the Go/No-Go task, or endorsement in the BIS/BAS scales.
DISCUSSION:
The present study found increased reward sensitivity and sensitivity to punishment in PIU individuals, though impulse control was not observably affected. Future experimental studies are needed to inform our conceptualization of the etiology of addictive behavior as it pertains to PIU. Further investigation will aid in informing prevention and intervention efforts.
Keywords: PIU, Reward Sensitivity, Inhibitory Function, Impulse control, Delay Discounting, SPSRQ, BIS/BAS scales, Behavioral Inhibition System, Behavioral Activation System, Addiction
1. Introduction
Addictive behaviors contribute to grave personal and societal costs. Evidence suggests that Problematic Internet Use (PIU) shares some key features of addictive behaviors. Whereas the etiology of addictive behaviors such as gambling and substance abuse has been studied extensively for decades (Ha et al., 2007; Young, 1998b), PIU received less attention. Perhaps this lack of attention is due to the Internet being a relatively young technological advancement (Christakis, 2010). Despite this paucity of research, converging research found that PIU interferes with aspects that maintain quality of life, including academics, sleep, psychological well-being, finances, and social relationships (Casale et al., 2016; Gámez-Guadix et al., 2013; Griffiths, 2000; Kuss et al., 2014; Kuss and Lopez-Fernandez, 2016a; Kuss and Lopez-Fernandez, 2016b; Mei et al., 2016; Pontes et al., 2015a; Pontes et al., 2015b; Shapira et al., 2000). Research in the broader addiction field hypothesized addictive behaviors are related to aberrant reward sensitivity, inhibitory function, and impulse control (Dong and Potenza, 2014; Ko et al., 2010; Moretta and Buodo, 2018; Pontes et al., 2015a). Investigations of these traits in PIU and Internet Gaming Disorder (IGD) are inconsistent. Additionally, the specific facets that are affected in PIU for each domain is unclear. To address this gap in knowledge, the current study assessed specific facets of reward, inhibition, and impulse control in individuals with versus without PIU. Further understanding the cognitive and affective mechanisms underlying PIU is crucial for clarifying etiological mechanisms and targeting preventive interventions.
Broadly, PIU is an inability to control the amount of time spent on the Internet (Shapiraet al., 2000) or psychological dependence irrespective of activity (Kandell, 1998; Yellowlees and Marks, 2007) that results in psychological distress and impaired daily functioning (Beard and Wolf, 2001; Kuss and Lopez-Fernandez, 2016b; Shapira et al., 2000). Symptoms of PIU include a general preoccupation with the Internet, an inability to stop using the Internet, and isolation from life outside the Internet (Kuss et al., 2014; Kuss and Lopez-Fernandez, 2016b; Kuss and Lopez-Fernandez, 2016a; Pontes et al., 2015a; Young, 1999). Existing research has largely focused on Internet Gaming Disorder (IGD), which is specific to excessive gaming on the Internet. This focus may be due to the DSM-V listing IGD as a “condition for future study” (Dong and Potenza, 2014). However, broader internet use contains distinct components of equal relevance that may differentiate PIU from IGD (Davis, 2001; Dong and Potenza, 2014; Moretta and Buodo, 2018; Orchard, 2010). Though both disorders focus on problematic Internet usage, PIU most conceptually differs from IGD in that it incorporates all facets of the internet, rather than only focusing on online/offline computer gaming (Dong and Potenza, 2014). Thus, PIU may be of broad interest to psychiatrists, as it captures a wider net of problematic behaviors centered around internet use (Pontes et al., 2015b). Given the increasing ubiquity of the Internet, it will become increasingly important for research to focus on broader Internet use (i.e. PIU) (Cash et al., 2012; Davis, 2001; Dong and Potenza, 2014; Kuss et al., 2014; Orchard, 2010).
Addictive behaviors have been related to abnormalities in reward sensitivity, inhibitory function, and impulse control (Brand et al., 2016; Dong and Potenza, 2014; Goodman, 1990). The IGD literature suggests that PIU could also be related to abnormalities in reward processing, inhibition, and impulse control, though it is unclear which specific facets are most affected in PIU (Brand et al., 2016; Dong et al., 2015; Kuss and Lopez-Fernandez, 2016b; Kuss and Lopez-Fernandez, 2016a; Nalwa, 2003; Younes et al., 2016). For example, a study found that PIU individuals exhibited stronger neural responses to monetary wins vs losses in a gambling task compared to healthy controls (Dong, 2013). Likewise, there is accumulating evidence indicating that deficits in inhibition are key features of PIU, which is similar to other addictive behaviors. Studies have reported that those with PIU report altered inhibition on behavioral measures (Cao, 2007; Mottram, 2009). Finally, the literature has been limited and inconsistent with regard to sensitivity to punishment (SP), which also indexes the behavioral inhibitory system (He, 2017; Nikolaidou et al., 2016).
Though the literature is promising, it is unclear which specific facets of reward and inhibitory function are implicated in PIU. Models of addiction have often conceptualized reward and inhibition according to two basic brain and behavioral systems that respond to punishing and reinforcing stimuli: the Behavioral Activation System (BAS), and the Behavioral Inhibition System (BIS) (Carver, 1994). The BAS includes sensitivity to reward (SR) and approach motivation, whereas the BIS relates to SP and avoidance motivation. Two common scales for measuring function in these systems are the BIS/BAS scales and the Sensitivity to Punishment and Reward Questionnaire (SPSRQ) (Carver, 1994; Torrubia, 2001). While these are both meant to index BIS/BAS function, studies suggest they may tap into distinct facets, and there is evidence that the scales can correlate with different outcomes (Carver, 1994; Segarra et al., 2014; Torrubia, 2001). For example, the SPSRQ was developed after the BIS/BAS scales, with the aim of tapping into concrete reward and punishment behaviors (Torrubia, 2001). Thus, it includes items that incorporate specific situations (O’Connor et al., 2004). In contrast, the BIS/BAS scales capture reactions to general, more abstract, cues of punishment and reward (Vandeweghe et al., 2016). To date, studies in the PIU/IGD literature have yet to report separate measurements of behavioral inhibition and reward sensitivity using both scales. This could be greatly beneficial in specifying exactly which facets of these domains are affected (Torrubia, 2001).
Finally, it is important to consider the role that inhibition may have in PIU. In fact, studies have reported that PIU is related to higher scores on self-report measures of impulsivity, which is theoretically orthogonal to BIS/BAS function (Cao, 2007; Chen et al., 2017; Franken and Muris, 2006; Mottram, 2009). Impulsivity is a key feature of many addictive behaviors, and yet it does not always relate to BAS learning of reward expectancies or behavioral responses to rewarding stimuli (Corr, 2004; Ding et al., 2014; Lee et al., 2012; Shapira et al., 2000). Thus, it is important to explore impulsivity separately from traditional BIS/BAS measures. The literature paints a complicated picture of impulsivity in PIU and related disorders. Studies assessing impulse control in the face of neutral stimuli have not observed impairment among those with IGD (Ding et al., 2014). Studies on IGD focusing on reward and impulsivity control have had inconsistent conclusions. One of these studies used an EEG gambling task, finding stronger SR in Internet-addicted youth (He, 2017). Two recent studies of IGD and one of PIU found enhanced SR and decreased ability to control impulsivity in the face of reward stimuli (Li et al., 2016; Tian, 2018; Wang et al., 2017). Other studies using self-report measures, however, have found a lack of association between risk taking and IGD/PIU (Ko et al., 2010; Sariyska et al., 2017). Studies testing impulse control in the face of both neutral and reward stimuli have been sparse in the PIU literature. Teasing out whether these relationships differ based on the nature of the stimuli would add nuance to interpretations of how impulse control interacts with reward in the context of PIU.
The present study recruited an undergraduate sample to build a psychological profile of individuals with PIU. Consistent with previous addiction and PIU/IGD research, (Brand et al., 2016; Cao, 2007; Dong, 2013; Meerkerk et al., 2010; Young, 1998a) we predicted that PIU individuals would exhibit greater sensitivity to reward, lower inhibition, and lower impulse control across measures. Finally, exploratory correlations with PIU symptoms were conducted to explore whether differences between cognitive domains are observable within high PIU individuals, and follow-up exploratory analyses were run examining PIU subdomains.
2. Method
2.1. Participants.
A total of 62 participants were recruited from the Northwestern University Undergraduate Research Study Pool and the paid registry (see Table 1). Participants were screened into the study based on their scores on the 18-item PIU questionnaire (PIUQ) (Demetrovics, 2008). Participants were included if English was their first language and if they endorsed PIU Questionnaire scores in either the highest 25% (PIU group) of scores or lowest 25% of scores (no-PIU group). Due to missing data, the resulting sample size for SPSRQ was 41 participants, 23 for PIU and 18 for no-PIU individuals. One PIU individual was excluded from the Go/NoGo task due to missing data, comprising a total of 31 PIU (Ding et al., 2014) and 30 no-PIU individuals. The total sample size of 62 people, 32 PIU and 30 no-PIU, was used for demographics, and BIS/BAS data.
Table 1.
Demographic characteristics.
PIU | No-PIU | Total | Statistic | P | |
---|---|---|---|---|---|
Gender | 14/33 male | 12/29 male | 26/62 male | χ2(1) = 0.007 | .93 |
Agea | 18.76 (1.06) | 19.17 (1.2) | 18.95 (1.14) | t(60) = 1.45 | .15 |
Parental Educationb | 16.39 (1.96) | 15.65 (2.01) | 16.05 (2.00) | t(60) = −1.462 | .15 |
WRATa | 111.58 (10.9) | 110.83 (11.2) | 111.26 (10.9) | t(42) =−.218 | .83 |
PIUQ Totala | 49.76 (8.65) | 23.79 (7.46) | 37.61 (15.34) | t(60) = −12.57 | <.001* |
Note.
Mean (SD),
Average of mother and father years of education.
2.2. PIU.
The PIUQ is an 18-item self-report questionnaire assessing problematic Internet behaviors (see Table 2 for items and subdomains). The test-retest reliability is 0.9 for the PIU scale (Demetrovics et al., 2008). In terms of internal validity, PIU has been found not to have a close association with total internet use, but rather with time spent online for non-work purposes (Demetrovics et al., 2008). PIU as measured by this scale has also been significantly associated with multiple indices of psychological and physical health (Kelley and Gruber, 2010). Scores range from 18 to 90 where higher scores reflect more PIU. There are 3 subdomains: Obsession constitutes daydreaming, fantasizing and longing for Internet use, while Neglect constitutes neglecting everyday roles and activities, and Control disorder relates to difficulties controlling Internet use (Demetrovics et al., 2008).The PIUQ was validated in a sample of 1,037 community members and identified four categories of participants: those with no problems (<22.7), those with few/average problems (22.7 to 42.4), those with problems present (42.4 to 52.2), and those with significant problems (>52.2) (Demetrovics, 2008). The 25th percentile cutoff for the current sample was 35, with an average score in the PIU group of 49.76, which falls in the range of moderate problems, nearing the cutoff for significant problems with internet usage. For no-PIU participants in the lowest 25th percentile, the average score was 23.79, within the few to average problems range, close to the “no problems” range.
Table 2.
Queries indexing overall PIU divided by subsets of Obsession, Neglect, and Control disorder.
PIU questionnaire queries |
---|
Obsession (0–30) |
How often do you fantasize about the Internet, or think about what it would be like to be online when you are not on the Internet? |
How often do you daydream about the Internet? |
How often do you feel tense, irritated, or stressed if you cannot use the Internet for several days? |
How often does it happen to you that you feel depressed, moody, or nervous when you are not on the Internet and these feelings stop once you are back online? |
How often do you dream about the Internet? |
Neglect (0–30) |
How often do you neglect household chores to spend more time online? |
How often do you spend time online when you’d rather sleep? |
How often do you choose the Internet rather than being with your partner? |
How often does the use of Internet impair your work or your efficacy? |
How often do people in your life complain about spending too much time online? |
How often do you choose the Internet rather than going out with somebody to have fun? |
Control Disorder (0–30) |
How often do you feel that you should decrease the amount of time spent online? |
How often does it happen to you that you wish to decrease the amount of time spent online but you do not succeed? |
How often do you try to conceal the amount of time spent online? |
How often do you feel that your Internet usage causes problems for you? |
How often do you realize saying when you are online, “just a couple of more minutes and I will stop”? |
How often do you think that you should ask for help in relation to your internet use? |
2.3. Reward Sensitivity/ Behavioral Activation System.
The BIS/BAS scales measure BIS and BAS sensitivities (Carver, 1994). These scales are widely used and have been validated in young adult populations (Demianczyk et al., 2014). Participants responded to BIS/BAS items using a scale from 1 (very false for me) to 4 (very true for me). The BAS scale included 13 items assessing desire for reward, positive responses to real or anticipated reward, and persistence in pursuing desired reward. Sample items include “I go out of my way to get things I want,” and “I often act on the spur of the moment.” The test-retest reliability is 0.66 for Drive, 0.59 for Reward Responsiveness, and 0.69 for Fun Seeking (Carver, 1994). For BAS, correlations have been observed between Reward and MMPI hypomania, PANAS positive affectivity, positive temperament, and TPQ reward dependence (Carver, 1994).
The SPSRQ measures reward and punishment sensitivities (Torrubia, 2001), and has been validated for use in a wide variety of populations (Dufey et al., 2011; Luman et al., 2012; Torrubia, 2001; Vandeweghe et al., 2016). The SPSRQ contains 48 questions which participants endorse using a binary Yes/No response scale. The 24-item SR scale contains items probing responses to specific rewards like money (“Does the good prospect of obtaining money motivate you strongly to do some things?”), sex (“Do you often take the opportunity to pick up people you find attractive?”), and power (“does the possibility of social advancement, move you to action even if this involves not playing fair?”). The test-retest reliability for SR is 0.87 (Torrubia, 2001). In terms of internal validity, intercorrelations are present with the Impulsiveness Scale and Sensation Seeking Scales, in line with Gray’s RST theoretical model of BAS function (Dufey et al., 2011; Torrubia, 2001).
2.4. Inhibitory function/Behavioral Inhibition System.
The seven-item scale from the above-mentioned BIS/BAS scales assessed BIS function, including threat sensitivity (“Criticism or scolding hurts me quite a bit”). The test-retest reliability is 0.66 for the BIS scale (Carver, 1994). Correlations have been observed between the BIS and the Manifest Anxiety Scale, The CPI Socialization Scale, the Life Orientation Test Optimism Scale, as well as PANAS negative affectivity, GTS Negative Temperament, Susceptibility to Punishment, and TPQ Harm Avoidance and Reward Dependence (Carver, 1994). The 24-item SP scale from the above mentioned SPSRQ contains items which reflect behavioral inhibition system function (Are you often afraid of new or unexpected situations?”), introversion (“Are you a shy person?”), and risk aversion (“Do you often refrain from doing something because of your fear of being praised?”). The test-retest reliability for SP is 0.89 (Torrubia, 2001). With regards to internal validity, negative intercorrelations have been observed between the SP scale and SSS Experience Seeking, Thrill and adventure seeking (Torrubia, 2001).
2.5. Impulse Control: neutral and in the face of reward stimuli.
Participants first completed the Go/No-Go Task to measure impulse control in the face of neutral stimuli, whereby a motor response was executed or inhibited. This task was used extensively by Telzer et al. (2013) and was implemented using the Psychology Building Experiment Battery (PEBL; see Figure 1a for a sample trial) (Mueller and Piper, 2014; Telzer et al., 2013). During the task, participants observed letters presented sequentially, and responded to target letters by pressing a button. The display was in a 2 × 2 format with 4 stars (one in each square of the array). One letter (P or R) was presented in one of the squares for 500 milliseconds, with a 1500 millisecond inter-stimulus interval. There were two conditions with 160 trials each. For the first condition (P-Go), participants pressed a button in response to the target letter P, and withheld from responding to the non-target letter R. The second condition was reversed (R-Go). The ratio of targets to non-targets was 80:20. A practice trial was completed to ensure comprehension of the task design. Number of correct responses and reaction time (RT) were calculated. Errors in responding correctly to the target letter are interpreted as indicators of inattention to the task, while responding incorrectly to the non-target letter as well as reaction time for target letter responses are indicators of impulsivity, or lack of inhibition (Bezdjian, 2009).
Figure 1.
(a) Sample trial for PEBL Go/No-Go task, (b) sample trial for delay discounting task.
We operationalized impulse control in the face of reward stimuli/inhibition of rewards as performance on a delay discounting task (DDT), see Figure 1b for a sample trial (Pornpattananangkul et al., 2017). DDT is a temporal discounting task that assesses willingness to wait for hypothetical rewards (Ahn et al., 2011; Rachlin et al., 1991). For each trial in the DDT, participants made a series of choices between a smaller immediate reward and a future reward of $800 for six delay period conditions that continued for six trials for a total of 36 trials. After their choice was made, a square was presented on the screen reflecting their choice for 1,500 ms, followed by a 3,000 ms fixation. Within each trial participants made hypothetical choices pitting an immediate reward against a delayed but more valuable reward. Specifically, participants made a series of choices between a smaller hypothetical immediate reward and a hypothetical future reward of $800. The future $800 choice was set to one of six delay periods: two weeks, one month, two months, six months, one year, three years, or ten years into the future (Ahn et al., 2011; Pornpattananangkul and Nusslock, 2016; Rachlin et al., 1991). At each delay period, choices began by pitting $400 now against $800 in the future. If the participant chose the immediate reward, then the immediate reward amount on the subsequent trial decreased by half the distance between that amount and $0 (Ahn et al., 2011; Du, 2002). All decisions were for hypothetical rewards and delays, as previous research has found similar response patterns whether the DDT is played for real or hypothetical reward amounts (Johnson and Bickel, 2002; Lagorio and Madden, 2005). See supplementary material for more information on DDT task design.
2.6. Data analyses.
Group differences in continuous and categorical demographic variables were evaluated with independent samples t-tests and Chi-square tests using SPSS Statistics 25. Independent t-tests were employed to examine group differences on domain measure performance. Due to expected skew in PIU domain measures (reflecting the recruitment strategy which focused on upper quartile for the high PIU grouping), exploratory Spearman correlations were conducted between domain measures and PIU score within the high PIU group. Lastly, as follow-up exploratory analyses, Spearman correlations were run between PIU subdomains of Obsession, Neglect, and Control Disorder, and SPSRQ scales showing group differences. Domain variables were normally distributed except for DDT and Go/No-Go. Running analyses log transforming to account for skew did not alter results. Two-tailed tests were used for all analyses.
3. Results
3.1. Demographic characteristics.
There were no significant group differences in demographic characteristics (see Table 1). As expected, the PIU group endorsed significantly greater Internet use compared to no-PIU individuals.
3.2. Group differences.
There were group differences on SPSRQ SR (see Table 3, Figure 2), but not in BIS/BAS scales measures. There were also group differences on SP as indexed by the SPSRQ, but not in BIS/BAS scales measures. There were no group differences in the total amount of errors from the Go/No-Go task, or in DDT performance. Follow up exploratory analyses showed a significant correlation between SR and PIU Neglect (r = 0.45, p = 0.003), a trending association with PIU Control Disorder (r = 0.31, p = 0.05), and no significant association with PIU Obsession (r = 0.23, p = 0.15). For SP, there were significant associations with all PIU domains: PIU Neglect (r = 0.53, p < 0.001), PIU Control Disorder (r = 0.42, p = 0.006), and PIU Obsession (r = 0.51, p = 0.001).
Table 3.
Group differences between no-PIU and PIU group and cognitive domains.
Task | t | p | Cohen’s d |
---|---|---|---|
Impulse control in the face of reward stimuli | |||
DDT k Hyperbolic | −1.0 | 0.32 | 0.32 |
DDT AUC | −0.20 | 0.84 | 0.06 |
Impulse control in the face of neutral stimuli | |||
Go/No-Go RT | 1.71 | 0.09 | 0.26 |
Go/No-Go total correct | 1.11 | 0.27 | 0.17 |
Behavioral Activation System Function | |||
SPSRQ Sensitivity to Reward | −2.73 | 0.01* | 0.87 |
BIS/BAS scales BAS Reward Responsivity | −0.86 | 0.39 | 0.22 |
BAS Drive | 0.33 | 0.74 | 0.05 |
BAS Fun Seeking | −1.32 | 0.19 | 0.20 |
Behavioral Inhibitory System Function | |||
BIS/BAS scales BIS | −1.26 | 0.21 | 0.20 |
SPSRQ Sensitivity to Punishment | −4.48 | <0.001** | 1.43 |
Note.
p < 0.05,
p < 0.01
Figure 2.
Sensitivity to Punishment (SP) by group (no-PIU vs. PIU)
3.3. Exploratory correlations within PIU group.
There were no significant differences excepting Go/No-Go reaction time (see Table 4).
Table 4.
Exploratory correlations within the PIU group between cognitive domain measures and PIU score.
Task | r | p |
---|---|---|
Impulse control in the face of reward stimuli | ||
DDT k Hyperbolic | 0.28 | 0.18 |
DDT AUC | −0.30 | 0.16 |
Impulse control in the face of neutral stimuli | ||
Go/No-Go RT | 0.37 | 0.04* |
Go/No-Go total correct | 0.04 | 0.85 |
Behavioral Activation System Function | ||
SPSRQ Sensitivity to Reward | −0.01 | 0.97 |
BIS/BAS scales BAS Reward Responsivity | −0.06 | 0.75 |
BAS Drive | 0.16 | 0.36 |
BAS Fun Seeking | −0.23 | 0.20 |
Behavioral Inhibitory System Function | ||
BIS/BAS scales BIS | −0.04 | 0.82 |
SPSRQ Sensitivity to Punishment | 0.13 | 0.56 |
Note.
p < 0.05,
p < 0.01
4. Discussion
The present study measured possible reward, inhibitory and impulse-control processes underlying PIU. In contrast to the previous literature that has largely focused on IGD, we indexed overall Internet use, which is more in line with the constantly increasing variety of ways in which individuals currently interact with the Internet (Kuss et al., 2014). Thus, the present study allows for an extension of the literature, as well as a cross-measure validation of previous findings. Among those with PIU, there was increased SR, as well as increased SP as assessed by the SPSRQ. However, there were no group differences in reward sensitivity and inhibitory function as assessed by the BIS/BAS scale. In addition, there were no group differences in impulse control in the face of neutral stimuli (Go/No-Go task) or in the face of reward stimuli (DDT). Results are complementary and partially consistent to the previous literature, suggesting that reward sensitivity and inhibitory function are compromised in PIU, which may render it similar to other types of addiction (Demetrovics et al., 2008; Tokunaga and Rains, 2016). Results suggest that in PIU individuals, the BIS/BAS scales and the SPSRQ may tap into at least somewhat distinct facets of reward sensitivity and inhibitory function (Torrubia, 2001), which will be important for future studies to consider and replicate. Findings are relevant for understanding possible risk factors for PIU and have the potential of informing future prevention/intervention efforts.
Differences were found in both SR and SP, such that PIU individuals reported both SR and greater SP, as assessed by the SPSRQ. These results support the notion that reward sensitivity and inhibitory systems are implicated in PIU, which is supported by a wide body of literature implicating these systems in other types of addiction (Brand et al., 2016; Cash et al., 2012; Chou, 2005; Dong, 2013; Dong et al., 2015; Ha et al., 2007; He, 2017; Lee et al., 2012; MacKillop et al., 2011; Sariyska et al., 2017; Smith et al., 2014; Wang et al., 2017). Greater SR and inhibitory function (as indexed by greater SP on the SPSRQ) may be predisposing factors for PIU. This could occur through multiple avenues. Internet use operates on a variable ratio reinforcement schedule (VRRS), similar to gambling and video games (Cash et al., 2012). That is to say, Internet activities, including blogging, surfing, message boards, social networking, video games, e-mail, etc. provide unpredictable and variable rewards, which supports addictive behavior. The rewarding nature of Internet use could likewise be intensified by mood enhancing/stimulating content encountered online (Cash et al., 2012). This mood enhancing/stimulating could in turn include aesthetically pleasing graphics, identification with a goal or motivation as can be encountered in online games, a sense of belonging and community found on blogs and message boards, or even a feeling of productivity from checking e-mail. Thus, many innate features of Internet usage may be conducive to developing addictive behavior. These addictive features may be particularly problematic for individuals with predisposing vulnerabilities in SR and SP (Cash et al., 2012; Kubey et al., 2001; Kuss et al., 2014; Meerkerk et al., 2010). Indeed, the current longitudinal literature supports the notion that PIU could be predicted by other predisposing vulnerability factors such as cyberbullying and depressive symptoms (Gámez-Guadix et al., 2013; Van den Eijnden et al., 2008). Given the cross-sectional nature of our sample, however, it could also be the case that engaging in these behaviors over time results in increased SR and SP.
Importantly, this investigation’s results examining reward responsiveness/behavioral inhibition using the BIS/BAS scale and SR/SP as assessed by the SPSRQ were not consistent. The BIS/BAS scales did not detect group differences, as was hypothesized. It is possible that no effect was found due to limited power to detect a smaller effect. However, despite both the BIS/BAS scales and the SPSRQ index BIS/BAS function, there are possibly important differences between the two measures (Carver, 1994; Torrubia, 2001). As mentioned earlier, items on the BIS/BAS scales measure reactions to general cues of punishment and reward, while the items on the SPSRQ index more specific cues of punishment and reward (Vandeweghe et al., 2016). Thus, in measuring reward sensitivity, the SPSRQ both employed more items, and probed for a wider variety of more specific rewards. Indeed, previous studies have found a lack of convergence between the BIS/BAS scales and related behavioral measures (Pickering, 1997); in some cases convergence has been found between behavioral indices and SPSRQ, but not with BIS/BAS scales (Smillie and Jackson, 2005).
SP has not often been directly investigated with regards to PIU. One previous investigation found results consistent with ours, though with a different measure of SP (Nikolaidou et al., 2016). An EEG investigation in individuals with internet addiction that found that those with Internet addiction exhibited weaker punishment sensitivity during an EEG gambling task (He, 2017). Diverging results may be due to differing measurement methods. Since the aforementioned investigation did not utilize self-report methods of SP, results are not directly comparable. The current study did not observe differences in inhibition as assessed by the BIS scales. Given solid evidence that the BIS/BAS scales and SPSRQ do not always converge (Pickering, 1997; Vandeweghe et al., 2016), future studies should further distinguish the relevant facets of BIS/BAS function. As partially mentioned above, many items on the SP scale as assessed by the SPSRQ probe for specific punishments. Results with regards to inhibition mirror results with regards to SR, suggesting that the SPSRQ may be capturing a facet of reward sensitivity and behavioral inhibition system function that was not observable with the BIS/BAS scales. Nonetheless, given the limited number of studies investigating SP in the context of PIU, future studies will be needed to replicate this result.
Results investigating relationships between impulse control in the face of neutral and reward stimuli add nuance to our interpretation. Impulse control deficits in the face of neutral stimuli were not observed in PIU individuals (Bezdjian et al., 2009; Lee et al., 2012). Previous IGD investigations using the Go/No-Go task with neutral stimuli have reported similar results (Ding et al., 2014). More surprisingly, PIU individuals in the present sample did not exhibit differences in impulse control in the face of reward stimuli. This finding is partially inconsistent with the literature, which has found differences in delay discounting in IGD and PIU individuals (Li et al., 2016; Tian, 2018; Wang et al., 2017). It is worth noting that the majority of these studies focused on gaming disorder, whereas the current study assessed PIU more broadly, not specific to gaming. It is also possible that because the rewards were hypothetical rather than actual, the effect was blunted, though previous studies have found this not to alter response patterns (Johnson and Bickel, 2002; Lagorio and Madden, 2005). Sample size may likewise have precluded our ability to detect an existing effect. Nonetheless, inconsistency in the literature signals to a need for further replication, including multiple measures cross-validating the same construct of interest. It will be paramount for future studies to further test competing models of impulse control in the face of neutral and/or reward stimuli versus reward sensitivity in this regard (Brand et al., 2016; Dong et al., 2015; Lee et al., 2012; Sariyska et al., 2017; Wang et al., 2017).
Finally, exploratory correlations within the high PIU group yielded differences in Go-No-Go RT, indicating that higher PIU was associated with increasing reaction times. Increased RT suggests greater difficulty completing the impulse control task in the face of neutral stimuli. This finding is interesting given that group differences were not found with regards to performance or RT between the low PIU and high PIU group. Notably, there were trending level correlations with DDT as well, such that higher PIU scores were trending with greater preference for smaller-but-immediate rewards. Perhaps the effect is observable only at moderate to high PIU levels, though future research is necessary before any definitive conclusion can be made. Associations were not detected among other domain measures. These results ought to be interpreted carefully given the limited sample size, and possible restriction of range issues in the high PIU group. Relatedly, follow-up analyses with PIU subdomains showed significant associations between PIU Control Disorder and SR, while all PIU subdomains were associated with SP. Difficulty controlling internet use could be especially related to increased SR, though these exploratory follow-up analyses ought to be interpreted as preliminary. Future research exploring PIU subdomains and differential associations with psychiatric symptoms are needed in order to understand underlying phenomenology.
This investigation provides important findings that can serve to guide future research, but it should be viewed as preliminary. To this point, there are several important caveats to consider when interpreting the results. First, in order to maximize signal with regards to PIU, participants were selected based on PIU scores in the top and bottom quartile. Future investigations with larger samples would benefit from incorporating participants with a broader range of PIU scores. Though power was adequate in the present study, future studies recruiting larger, more representative, samples will be paramount moving forward. Given the present sample size, results should be carefully interpreted as preliminary. This caution is particularly applicable to the SPSRQ data, which unfortunately had some missing data that further limited the overall sample size. Most notably, as above mentioned, the sample is cross-sectional, which precludes inferring causality. Future investigations will additionally benefit from mapping out specific patterns of Internet behaviors (i.e. time spent online gaming versus social media, etc.) and testing subgroups accordingly. Finally, studies examining symptoms related to PIU would be informative, as the literature suggests PIU can be both distressing and impairing, often impacting daily functioning (Cash et al., 2012; Kuss et al., 2014; Kuss and Lopez-Fernandez, 2016b; Younes et al., 2016; Young, 1999).
In conclusion, the current study has the potential of being highly informative for future preventive and intervention efforts. Understanding the cognitive underpinnings of PIU could provide us with targets for treatment and increased etiological understanding. For example, if it is known that this is related to SR, intervention treatments could focus on developing reward-focused interventions. If SR is a risk factor, this could inform preventive efforts in those that are highly sensitive to reward. The present study suggests that specific cues of reward and punishment relate to PIU. Effects were not detected for impulse control in the face of neutral or reward stimuli, nor were they detected by the BIS/BAS scales, which index trait level features of BIS/BAS functions. Thus, results suggest that incorporating measures of multiple facets of impulse control, behavioral activation and behavioral inhibitory systems is a crucial future direction for PIU research. Though results ought to be interpreted as preliminary due to limited sample size, the present investigation contributes a pilot examination of multiple facets of BIS/BAS function and impulse control.
Supplementary Material
Figure 3.
Sensitivity to Reward (SR) by group (no-PIU vs. PIU).
Figure 4.
Sensitivity to Punishment (SP) and Sensitivity to Reward (SR) by group (no-PIU vs. PIU)
Highlights.
Problematic Internet Use (PIU) individuals showed greater sensitivity to reward.
PIU individuals exhibited greater sensitivity to punishment.
PIU individuals did not show differences in impulse control (for neutral or reward stimuli).
Acknowledgements
There are no acknowledgements.
Role of funding source
This work was supported by a Northwestern University Society Biology and Health Cluster fellowship (T.V.) and by grants R01MH112545, R21/R33MH103231 and R21MH110374 (V.A.M).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflicts of Interest.
The authors report no conflicts of interest.
References
- Ahn WY, Rass O, Fridberg DJ, Bishara AJ, Forsyth JK, Breier A, Busemeyer JR, Hetrick WP, Bolbecker AR, O’Donnell BF, 2011. Temporal discounting of rewards in patients with bipolar disorder and schizophrenia. J Abnorm Psychol 120 (4), 911–921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beard KW, Wolf EM, 2001. Modification in the proposed diagnostic criteria for Internet addiction. Cyberpsychol Behav 4 (3), 377–383. [DOI] [PubMed] [Google Scholar]
- Bezdjian S, Baker LA, Lozano DI, Raine A, 2009. Assessing inattention and impulsivity in children during the Go/NoGo task. Br J Dev Psychol 27 (Pt 2), 365–383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bezdjian S, Baker LA, Lozano DI, Raine A, 2009. Assessing inattention and impulsivity in children during the Go/NoGo task. British Journal of Developmental Psychology 27, 365–383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brand M, Young KS, Laier C, Wolfling K, Potenza MN, 2016. Integrating psychological and neurobiological considerations regarding the development and maintenance of specific Internet-use disorders: An Interaction of Person-Affect-Cognition-Execution (I-PACE) model. Neurosci Biobehav Rev 71, 252–266. [DOI] [PubMed] [Google Scholar]
- Cao F, Su L, Liu T, Gao X, 2007. The relationship between impulsivity and Internet addiction in a sample of Chinese adolescents. . European Psychiatry 22 (7), 466–471. [DOI] [PubMed] [Google Scholar]
- Carver CS, White TL, 1994. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales. Journal of personality and social psychology. 67 (2), 319. [Google Scholar]
- Casale S, Caplan SE, Fioravanti G, 2016. Positive metacognitions about Internet use: The mediating role in the relationship between emotional dysregulation and problematic use. Addictive behaviors 59, 84–88. [DOI] [PubMed] [Google Scholar]
- Cash H, Rae CD, Steel AH, Winkler A, 2012. Internet Addiction: A Brief Summary of Research and Practice. Curr Psychiatry Rev 8 (4), 292–298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen SK, Lo MT, Lin SS, 2017. Impulsivity as a precedent factor for problematic Internet use: How can we be sure? International Journal of Psychology 52 (5), 389–397. [DOI] [PubMed] [Google Scholar]
- Chou C, Condron L, Belland JC, 2005. A review of the research on Internet addiction. Educational Psychology Review 17 (4), 363–388. [Google Scholar]
- Christakis DA, 2010. Internet addiction: a 21 st century epidemic? BMC medicine 8 (1), 61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corr PJ, 2004. Reinforcement sensitivity theory and personality. Neurosci Biobehav Rev 28 (3), 317–332. [DOI] [PubMed] [Google Scholar]
- Davis RA, 2001. A cognitive-behavioral model of pathological Internet use. Computers in human behavior 17 (2), 187–195. [Google Scholar]
- Demetrovics Z, Szeredi B, Rózsa S, 2008. The three-factor model of Internet addiction: The development of the Problematic Internet Use Questionnaire. Behavior Research Methods 40 (2), 563–574. [DOI] [PubMed] [Google Scholar]
- Demetrovics Z, Szeredi B, Rozsa S, 2008. The three-factor model of Internet addiction: The development of the Problematic Internet Use Questionnaire. Behavior Research Methods. 40, 563–574. [DOI] [PubMed] [Google Scholar]
- Demianczyk AC, Jenkins AL, Henson JM, Conner BT, 2014. Psychometric evaluation and revision of Carver and White’s BIS/BAS scales in a diverse sample of young adults. J Pers Assess 96 (5), 485–494. [DOI] [PubMed] [Google Scholar]
- Ding WN, Sun JH, Sun YW, Chen X, Zhou Y, Zhuang ZG, Li L, Zhang Y, Xu JR, Du YS, 2014. Trait impulsivity and impaired prefrontal impulse inhibition function in adolescents with internet gaming addiction revealed by a Go/No-Go fMRI study. Behav Brain Funct 10, 20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dong G, Hu Y, Lin X, 2013. Reward/Punishment sensitivities among internet addicts: Implications for their addictive behaviors. Progress in Neuro-Psychopharmacology and Biological Psychiatry 46 (139–145). [DOI] [PubMed] [Google Scholar]
- Dong G, Lin X, Hu Y, Xie C, Du X, 2015. Imbalanced functional link between executive control network and reward network explain the online-game seeking behaviors in Internet gaming disorder. Sci Rep 5, 9197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dong G, Potenza MN, 2014. A cognitive-behavioral model of Internet gaming disorder: theoretical underpinnings and clinical implications. J Psychiatr Res 58, 7–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Du W, Green L, Myerson J, 2002. Cross-Cultural Comparisons of Discounting Delayed and Probabilistic Rewards. The Psychological Record 52 (4), 479–492. [Google Scholar]
- Dufey M, Fernandez AM, Mourgues C, 2011. Assessment of the behavioral inhibition system and the behavioral approach system: adaptation and validation of the Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) in a Chilean sample. Span J Psychol 14 (1), 432–440. [DOI] [PubMed] [Google Scholar]
- Franken IH, Muris P, 2006. Gray’s impulsivity dimension: A distinction between reward sensitivity versus rash impulsiveness. Personality and individual differences 40 (7), 1337–1347. [Google Scholar]
- Gámez-Guadix M, Orue I, Smith PK, Calvete E, 2013. Longitudinal and reciprocal relations of cyberbullying with depression, substance use, and problematic internet use among adolescents. Journal of Adolescent Health 53 (4), 446–452. [DOI] [PubMed] [Google Scholar]
- Goodman A, 1990. Addiction: definition and implications. Addiction 85 (11), 413–418. [DOI] [PubMed] [Google Scholar]
- Griffiths MD, 2000. Internet addiction- time to be taken seriously? Addiction research 8 (5), 413–418. [Google Scholar]
- Ha JH, Kim SY, Bae SC, Bae S, Kim H, Sim M, Lyoo IK, Cho SC, 2007. Depression and Internet addiction in adolescents. Psychopathology 40 (6), 424–430. [DOI] [PubMed] [Google Scholar]
- He W, Qi A, Wang Q, Wu H, Zhang Z, Gu R, Luo W, 2017. Abnormal reward and punishment sensitivity associated with Internet addicts. Computers in human behavior 75, 678–683. [Google Scholar]
- Johnson MW, Bickel WK, 2002. Within-subject comparison of real and hypothetical money rewards in delay discounting. J Exp Anal Behav 77 (2), 129–146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kandell J, 1998. Internet addiction on campus: The vulnerability of college students. Cyberpsychology & behavior 1 (1), 11–17. [Google Scholar]
- Kelley KJ, Gruber EM, 2010. Psychometric properties of the problematic internet use questionnaire. Computers in human behavior 26 (6), 1838–1845. [Google Scholar]
- Ko CH, Hsiao S, Liu GC, Yen JY, Yang MJ, Yen CF, 2010. The characteristics of decision making, potential to take risks, and personality of college students with Internet addiction. Psychiatry Res 175 (1–2), 121–125. [DOI] [PubMed] [Google Scholar]
- Kubey RW, Lavin MJ, Barrows JR, 2001. Internet use and collegiate academic performance decrements: Early findings. Journal of communication 51 (2), 366–382. [Google Scholar]
- Kuss DJ, Griffiths MD, Karila L, Billieux J, 2014. Internet addiction: a systematic review of epidemiological research for the last decade. Curr Pharm Des 20 (25), 4026–4052. [DOI] [PubMed] [Google Scholar]
- Kuss DJ, Lopez-Fernandez O, 2016a. Internet addiction and problematic Internet use: A systematic review of clinical research. World journal of psychiatry 6 (1), 143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kuss DJ, Lopez-Fernandez O, 2016b. Internet addiction and problematic Internet use: A systematic review of clinical research. World J Psychiatry 6 (1), 143–176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lagorio CH, Madden GJ, 2005. Delay discounting of real and hypothetical rewards III: steady-state assessments, forced-choice trials, and all real rewards. Behav Processes 69 (2), 173–187. [DOI] [PubMed] [Google Scholar]
- Lee HW, Choi JS, Shin YC, Lee JY, Jung HY, Kwon JS, 2012. Impulsivity in internet addiction: a comparison with pathological gambling. Cyberpsychol Behav Soc Netw 15 (7), 373–377. [DOI] [PubMed] [Google Scholar]
- Li Q, Tian M, Taxer J, Zheng Y, Wu H, Sun S, Liu X, 2016. Problematic internet users’ discounting behaviors reflect an inability to delay gratification, not risk taking. Cyberpsychology, Behavior, and Social Networking 19 (3), 172–178. [DOI] [PubMed] [Google Scholar]
- Luman M, van Meel CS, Oosterlaan J, Geurts HM, 2012. Reward and punishment sensitivity in children with ADHD: validating the Sensitivity to Punishment and Sensitivity to Reward Questionnaire for children (SPSRQ-C). J Abnorm Child Psychol 40 (1), 145–157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, Munafo MR, 2011. Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology (Berl) 216 (3), 305–321. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meerkerk G-J, van den Eijnden RJ, Franken I, Garretsen H, 2010. Is compulsive internet use related to sensitivity to reward and punishment, and impulsivity? Computers in human behavior 26 (4), 729–735. [Google Scholar]
- Mei S, Yau YH, Chai J, Guo J, Potenza MN, 2016. Problematic Internet use, well-being, self-esteem and self-control: Data from a high-school survey in China. Addictive behaviors 61, 74–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moretta T, Buodo G, 2018. Autonomic stress reactivity and craving in individuals with problematic Internet use. PLoS One 13 (1), e0190951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mottram AJ, Fleming MJ, 2009. Extraversion, impulsivity, and online group membership as predictors of problematic Internet use. Cyberpsychology & behavior 12 (3), 319–321. [DOI] [PubMed] [Google Scholar]
- Mueller ST, Piper BJ, 2014. The Psychology Experiment Building Language (PEBL) and PEBL Test Battery. J Neurosci Methods 222, 250–259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nalwa K, Anand AP, 2003. Internet addiction in students: A cause of concern. Cyberpsychology & behavior 6 (6), 653–656. [DOI] [PubMed] [Google Scholar]
- Nikolaidou M, Fraser DS, Hinvest N, 2016. Physiological markers of biased decision-making in problematic Internet users. Journal of behavioral addictions 5 (3), 510–517. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Connor RM, Colder CR, Hawk LW Jr, 2004. Confirmatory factor analysis of the Sensitivity to Punishment and Sensitivity to Reward Questionnaire. Personality and individual differences 37 (5), 985–1002. [Google Scholar]
- Orchard LJ, Fullwood C, 2010. Current perspectives on personality and Internet use. Social Science Computer Review 28 (2), 155–169. [Google Scholar]
- Pickering AD, 1997. The conceptual nervous system and personality: From Pavlov to neural networks. European Psychologist 2 (2), 139–163. [Google Scholar]
- Pontes HM, Kuss DJ, Griffiths MD, 2015a. Clinical psychology of Internet addiction: a review of its conceptualization, prevalence, neuronal processes, and implications for treatment. Neuroscience & Neuroeconomics 4, 11–23. [Google Scholar]
- Pontes HM, Szabo A, Griffiths MD, 2015b. The impact of Internet-based specific activities on the perceptions of Internet addiction, quality of life, and excessive usage: A cross-sectional study. Addictive Behaviors Reports 1, 19–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pornpattananangkul N, Nadig A, Heidinger S, Walden K, Nusslock R, 2017. Elevated outcome-anticipation and outcome-evaluation ERPs associated with a greater preference for larger-but-delayed rewards. Cogn Affect Behav Neurosci 17 (3), 625–641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pornpattananangkul N, Nusslock R, 2016. Willing to wait: Elevated reward-processing EEG activity associated with a greater preference for larger-but-delayed rewards. Neuropsychologia 91, 141–162. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rachlin H, Raineri A, Cross D, 1991. Subjective probability and delay. J Exp Anal Behav 55 (2), 233–244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sariyska R, Lachmann B, Markett S, Reuter M, Montag C, 2017. Individual differences in implicit learning abilities and impulsive behavior in the context of Internet addiction and Internet Gaming Disorder under the consideration of gender. Addict Behav Rep 5, 19–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Segarra P, Poy R, López R, Moltó J, 2014. Characterizing Carver and White’s BIS/BAS subscales using the Five Factor Model of personality. Personality and individual differences 61, 18–23. [Google Scholar]
- Shapira NA, Goldsmith TD, Keck PE Jr., Khosla UM, McElroy SL, 2000. Psychiatric features of individuals with problematic internet use. J Affect Disord 57 (1–3), 267–272. [DOI] [PubMed] [Google Scholar]
- Shead NW, Hodgins DC, 2009. Probability discounting of gains and losses: implications for risk attitudes and impulsivity. J Exp Anal Behav 92 (1), 1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smillie LD, Jackson CJ, 2005. The appetitive motivation scale and other BAS measures in the prediction of approach and active avoidance. Personality and individual differences 38 (4), 981–994. [Google Scholar]
- Smith JL, Mattick RP, Jamadar SD, Iredale JM, 2014. Deficits in behavioural inhibition in substance abuse and addiction: a meta-analysis. Drug Alcohol Depend 145, 1–33. [DOI] [PubMed] [Google Scholar]
- Telzer EH, Fuligni AJ, Lieberman MD, Galvan A, 2013. Meaningful family relationships: neurocognitive buffers of adolescent risk taking. J Cogn Neurosci 25 (3), 374–387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tian M, Tao R, Zheng Y, Zhang H, Yang G, Li Q, Liu X, 2018. Internet gaming disoder in adolescents is linked to delay discounting but not probability discounting. Computers in human behavior 80, 59–66. [Google Scholar]
- Tokunaga RS, Rains SA, 2016. A review and meta-analysis examining conceptual and operational definitions of problematic Internet use. Human Communication Research 42 (2), 165–199. [Google Scholar]
- Torrubia R, Avila C, Molto J Caseras X, 2001. The Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) as a measure of Gray’s anxiety and impulsivity dimensions. Personality and individual differences. 31 (6), 837–862. [Google Scholar]
- Van den Eijnden RJ, Meerkerk G-J, Vermulst AA, Spijkerman R, Engels RC, 2008. Online communication, compulsive Internet use, and psychosocial well-being among adolescents: a longitudinal study. Developmental psychology 44 (3), 655. [DOI] [PubMed] [Google Scholar]
- Vandeweghe L, Matton A, Beyers W, Vervaet M, Braet C, Goossens L, 2016. Psychometric Properties of the BIS/BAS Scales and the SPSRQ in Flemish Adolescents. Psychol Belg 56 (4), 406–420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang Y, Wu L, Zhou H, Lin X, Zhang Y, Du X, Dong G, 2017. Impaired executive control and reward circuit in Internet gaming addicts under a delay discounting task: independent component analysis. Eur Arch Psychiatry Clin Neurosci 267 (3), 245–255. [DOI] [PubMed] [Google Scholar]
- Yellowlees PM, Marks S, 2007. Problematic Internet use or Internet addiction? Computers in human behavior 23 (3), 1447–1453. [Google Scholar]
- Younes F, Halawi G, Jabbour H, El Osta N, Karam L, Hajj A, Rabbaa Khabbaz L, 2016. Internet Addiction and Relationships with Insomnia, Anxiety, Depression, Stress and Self-Esteem in University Students: A Cross-Sectional Designed Study. PLoS One 11 (9), e0161126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young KS, 1998a. Internet addiction: The emergence of a new clinical disorder. Cyberpsychology & behavior 1 (3), 237–244. [Google Scholar]
- Young KS, 1999. Internet addiction: symptoms, evaluation and treatment. Innovations in clinical practice: A source book 17, 19–31. [Google Scholar]
- Young KS, Rogers RC, 1998b. The relationship between depression and Internet addiction. Cyberpsychology & behavior 1 (1), 25–28. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.