Abstract
Background
Few studies have examined the prevalence of problematic internet use (PIU) in young people undergoing inpatient treatment in child and adolescent psychiatry centers. The aims of our study were thus (a) to assess the frequency of comorbid PIU in a sample of adolescent psychiatric inpatients and compare it with a control group of nonreferred adolescents and (b) to gain insights into correlations between PIU and psychiatric comorbidities.
Methods
111 child and adolescent psychiatry inpatients (CAP-IP, mean age 15.1 ± 1.4 years; female : male 72.4% : 27.6%) undergoing routine psychodiagnostics were screened for the presence of PIU. The widely used Compulsive Internet Use Scale (CIUS) was chosen for this purpose. Prevalence rates of PIU were then compared to matched nonreferred control subjects from a school sample. Additionally, comorbidities of inpatients with PIU were compared to inpatients without PIU.
Results
Our inpatient sample showed a much higher prevalence of PIU than that found in previous populational samples of young people. Compared with a matched school sample, addictive internet use was 7.8 times higher and problematic internet use 3.3 times higher among our adolescent sample. PIU was significantly associated with characteristic patterns of psychopathology, that is, suicidality, difficulties in establishing stable and consolidated identity, and peer victimization.
Conclusion
PIU among adolescents undergoing inpatient psychiatric treatment is much more frequent than among their peers in the general population and is associated with specific patterns of psychopathology.
1. Introduction
The use of the internet and related digital media has grown exponentially worldwide in recent years. Indeed, in mid-2016, a penetration rate of almost half of the world's population was recorded. This translates into a worldwide growth rate of more than 900% from 2010 to 2016. The fastest growing region was found to be the European Union, where in 2016, nearly 80% of the overall population was using the internet [1].
While this trend has been observed in all age groups, adolescents, in particular, adopt new technologies quickly. Virtually 100% of German-speaking adolescents use the internet on a daily basis, more than 80% use it on the go, and more than 50% consider it “indispensable” [2, 3]. Access to relevant technologies has become a precondition for educational and occupational advancement; thus, internet “abstinence” is not an option for the majority of youth today [2, 4].
For the past 20 years, the rising significance of the internet and associated media has stimulated debate about dysregulated or problematic use of the internet (PIU) among young people [5, 6] and about whether such dysfunctional patterns affect somatic health and psychosocial wellbeing [7].
From a pathophysiological point of view, PIU has been understood as an impulse control disorder that is best classified within the group of behavioral addictions. Despite a growing body of scientific evidence for this taxonomy [8–17], only “Internet Gaming Disorder (IGD)”—that is, the excessive use of online computer games, leading to functional impairment and distress—was included in Section III of the DSM-5 as a “condition that requires further study” [18].
Given the diversity of diagnostic assessment methods [19, 20], it is unsurprising that PIU prevalence figures vary substantially. The only meta-analysis to date included 80 studies with nearly 90,000 participants and a mean age of 18.4 years. The authors found mean worldwide prevalence of 6% for PIU, with the European subsample showing 4.35% [21]. In recent years, studies based on large representative samples of the European adolescent population have found PIU prevalence rates of 1.2–4.4% [4, 22–24]. Müller et al. investigated the occurrence of the proposed DSM-5 criteria for Internet Gaming Disorder (IGD) [18] in a large sample of more than 10,000 European adolescents, finding a prevalence of 1.6% [25]. In a representative sample of German adolescents, criteria for IGD were found in 1.16% of the study participants [26].
Our study used the Compulsive Internet Use Scale (CIUS), which has already been applied by several other investigations of German-speaking adolescents, yielding results comparable to ours. Rumpf et al. [27] found PIU to be prevalent in 2.4% of young people aged 14–24 years and in 4.0% of adolescents aged 14–16. In a second study, the authors found a prevalence rate of 3.2% in German adolescents aged 14–17 [24]. The only Austrian study to date using the CIUS [2] showed PIU to be prevalent in 3.3% of the sample (n = 398, mean age = 15.2 years). This school sample was used in the present study as a point of reference.
A growing body of evidence indicates that PIU is associated with a broad spectrum of somatic and psychosocial problems as well as with psychiatric comorbidity. Research on the negative somatic effects of excessive internet use in adolescents has found sleeping problems [8, 28–32], overweight [29, 33–35], poor nutrition [8, 36], and back and musculoskeletal issues [28]. With respect to psychosocial behavior, there is evidence that dysregulated internet use is associated with tobacco use [8], less time spent with real-life peers [37] and physical inactivity, or sedentary lifestyle [8, 38]. PIU has also been associated with self-destructive behavior in adolescents [9]. Two meta-analyses on internet addiction and psychiatric comorbidities in adolescents and adults showed comparable results. In these studies, PIU was associated with symptoms of ADHD, alcohol abuse, depression and anxiety, hostility and aggression, and obsessive-compulsive symptoms [39, 40]. In their large-scale and cross-national study on European adolescents (n = 11,356, mean age 14.9 years), Kaess et al. demonstrated that PIU in adolescents significantly correlated with suicidal behavior, depression, anxiety, conduct problems, and symptoms of ADHD [9]. The authors thus conclude that assessing PIU could be helpful for early detection and intervention in cases of suicidal ideation and psychopathology in adolescents. A close association between suicidality and PIU has also been reported by other studies [8, 41, 42].
In light of these findings, it is certainly worth exploring why PIU seems to play a role in such a broad spectrum of internalizing and externalizing psychopathology. In the case of internalizing comorbidity, PIU might be a “digital expression” or form of “digital compensation” of classical psychopathology symptoms, such as loss of enjoyable activities, loss of engagement with real-life people, social withdrawal, escapism, avoidance, loss of life satisfaction, and suicidality. In the case of externalizing comorbidity, the impulse control domain of disorders such as ADHD, borderline personality disorder, and NSSI may contribute to the association between these entities and PIU. The association between substance-use disorders and PIU is not surprising, given that they share pathophysiological mechanisms [12, 13, 16, 17]. It is important to emphasize that a potential causal relationship between PIU and psychiatric comorbidity is not yet clear because findings have primarily been based on cross-sectional data. Thus, conclusions on cause and effect cannot yet be drawn.
In adolescence, young people increasingly begin to establish social networks outside of the core family, and psychosocial development is strongly influenced by extrafamilial peers. The internet and social networking sites (SNS) can be an important and useful tool for young people to develop and experiment with relationships and social capital [43]. On the other hand, there is some evidence that in adolescents, intensive SNS use may be associated with poor psychological functioning, poor mental health, and suicidality, especially in the presence of cyberbullying [44, 45]. Instability in interpersonal relationships and problems with identity and ego strength may play a mediating role, but to our knowledge, this hypothesis has not yet been tested in studies.
Taken together, the “complex interaction between various aetiological factors” [40] and the ubiquitous availability of relevant technologies may have led to the high prevalence of PIU and associated psychopathology in adolescent populational samples. In recent European studies, PIU and associated psychopathology cases were equally distributed among male and female adolescents, in contrast to previous findings and popular belief [8].
Most findings on the comorbidity of PIU and psychiatric diagnoses in youth have been gathered in nonclinical samples. Few studies have examined adolescents undergoing inpatient psychiatric treatment. Müller et al. screened a sample of 81 juvenile patients and found a high prevalence of PIU, 11.3%, as well as an association with internalizing disorders such as anxiety and depression [46]. In another study, a sample of 60 referred adolescents with diagnosed PIU was screened for the presence of comorbid psychiatric disorders. The most common conditions found were ADHD, social phobia, and major depressive disorder [47].
To our knowledge, no study to date has directly compared the prevalence of PIU in adolescents undergoing inpatient CAP treatment with a matched control group of nonreferred peers using the same diagnostic instrument. The present study was thus conducted to test the following two hypotheses:
(1) Adolescents undergoing inpatient psychiatric treatment will show different rates of dysfunctional internet use (addictive internet use and problematic internet use) from a control group of nonreferred peers. Null hypothesis: the groups will not differ in terms of internet use.
(2) A group comparison of adolescent inpatients with dysfunctional internet use (addictive internet use and problematic internet use) and inpatients with normal use will show differences concerning additional clinical variables. Null hypothesis: the groups will not differ statistically in terms of other psychopathology.
2. Materials and Methods
2.1. Sample and Procedure
We analyzed data from inpatients treated in the Department of Child and Adolescent Psychiatry, Medical University of Innsbruck, between 2013 and 2017. The Department of Child and Adolescent Psychiatry is a specialized facility with a public service mandate for the Austrian federal state of Tyrol. It is the only hospital in the state with inpatient services for minors with mental health problems.
As part of a routine diagnostic battery, patients completed the Structured Clinical Interview for DSM-IV Axis II Disorders (SCID-II), the Youth Self-Report (YSR), and the Assessment of Identity Development in Adolescence (AIDA). A parent or legal guardian completed the Child Behavior Checklist (CBCL). Sociodemographic data was also collected at the beginning of inpatient treatment. For the purposes of this study, the Compulsive Internet Use Scale (CIUS) was added to assess PIU. The primary psychiatric diagnosis was extracted from electronic patient records.
Inclusion criteria were (a) minimum age of 12 years, (b) undergoing inpatient treatment in the Department of Child and Adolescent Psychiatry (Innsbruck), and (c) written declaration of consent. Exclusion criteria were (a) age below 12 years and (b) diagnosis of severe mental retardation or florid psychopathological symptoms (e.g., psychosis or intoxication), which would have made the completion of the questionnaires unfeasible. All patients and their parents or legal guardians signed an informed consent form for the scientific use of data. The study was approved by the Ethics Committee of the Medical University of Innsbruck and was performed according to the Declaration of Helsinki 1995 (as revised in Edinburgh in 2000).
2.2. Measurement
2.2.1. CIUS
The Compulsive Internet Use Scale (CIUS) is a widely used self-report questionnaire with 14 items. It was developed to assess addictive internet use, on the basis of the criteria for substance dependence and pathological gambling in the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition [48]. The 14 questions are answered on a 5-point Likert scale, resulting in a sum score between 0 and 56 points. A cut-off of 28 points identifies pathological or addictive internet use [49], while a cut-off of 21 points identifies a subthreshold level of problematic internet use [50]. The 14 items of the CIUS measure five subscales: loss of control, preoccupation, conflict, withdrawal symptoms, and coping. The CIUS has been validated in several languages, including German [51]. Good external and factorial validity as well as good reliability indices were reported for the questionnaire in samples of heavy internet users (Cronbach α = 0.89–0.90) and normative samples (α = 0.90) [49].
2.2.2. Assessment of Identity Development in Adolescence (AIDA)
The Assessment of Identity Development in Adolescence (AIDA) self-report questionnaire was developed to measure fundamental subdomains of juvenile identity development. The instrument can distinguish between stable and consolidated identity at one end and identity diffusion at the other [52]. Identity diffusion is considered one of the core elements of borderline personality organization, but it is also part of the section “Criterion A: Level of Personality Functioning” of the alternative DSM-5 model for personality disorders [18]. The 58 items of the AIDA are summed up to a total score (extent of identity diffusion), which can also be divided into two subscales, discontinuity and incoherence. The instrument shows good reliability in total score (diffusion: α = .94), scale (discontinuity: α = .86; incoherence: α = .92), and subscale (α = .73–.86) as well as good validity [52–54].
2.2.3. SCID-II
The Structured Clinical Interview for DSM-IV Disorders (SCID) is the official diagnostic instrument of the American Psychiatric Association (APA) for the assessment of psychiatric disorders via clinical interview. To diagnose personality disorders, the Structured Clinical Interview for DSM-IV Axis II Disorders (SCID-II) is used, which is available in German [55]. The German version is considered appropriate for the diagnosis of personality disorders in adolescence [56]. SCID-II consists of a self-report questionnaire followed by a clinical interview. Each part is scored as 0 (absent), 2 (subclinical), or 3 (present). Our calculations were based on the mean dimensional score (ranging from 0–3) of the three personality disorder clusters A, B, and C.
2.2.4. CBCL and YSR
The Child Behavior Checklist (CBCL) and the corresponding Youth Self-Report (YSR) are among the most widely used instruments to assess mental health in minors [57]. Both instruments have been translated into more than 50 languages, and German versions have been available for approx. 20 years [58]. The CBCL features eight problem syndrome scales that can be combined into the broadband scales “Internalizing Behavior Problems” (Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints), “Externalizing Behavior Problems” (Rule-Breaking Behavior, Aggressive Behavior), and “Mixed Problems” (Social Problems, Thought Problems, Attention Problems). A total behavior problems score (TOT) can also be calculated. Significant discrepancy between youth-reported and parent-reported psychopathology is a well-known phenomenon worldwide, being also evident in studies of large German samples [59]. For the CBCL and its subscales, good reliability (r > .86) and validity indices have been reported [57], with reliability indices for the YSR-subscales reported as sufficient (r > .70) [57].
2.2.5. Primary Psychiatric Diagnosis
Electronic patient records were evaluated, extracting the primary psychiatric diagnosis according to ICD-10 [60]. For some calculations, we grouped inpatient diagnoses into the broader categories “internalizing disorders” (affective disorders and anxiety disorders) and “externalizing disorders” (hyperkinetic disorders and conduct disorders), which is common practice in child and adolescent psychiatry [61].
2.2.6. (Cyber)bullying
As part of the routine clinical examination, patients were asked whether they had been bullied online or offline, and whether they were active bullies online or offline themselves. The total number of instances (being bullied online/offline, active bullying online/offline) was assessed for this study.
2.2.7. Control Sample
To compare CAP-IP with nonreferred youth, we analyzed the data from a school sample from a previously published study [2]. This allowed us to compare two groups examined in the same state with the same instrument. The groups were matched for age and gender.
2.3. Statistical Procedures
Sample characteristics are given as frequencies, ranges, means, and standard deviations. IBM SPSS (version 22.0) was used for statistical analysis.
To compare the CIUS values of the inpatient sample with the control sample, independent sample t-tests and effect sizes (Cohen's d) were calculated. The odds ratio was estimated through binary logistic regression. Group differences were calculated for the overall samples and for age- and gender-matched comparisons. Twin-matching was conducted using the propensity score matching procedure.
The psychopathology of inpatients with PIU was compared to inpatients without PIU. Group comparisons were conducted using the chi-squared test or Fisher's exact test for nominal data and analyses of variance with sex as a covariate (ANCOVA) for interval data. Cohen's d was used to evaluate the effect sizes of the mean differences.
To investigate differences in internet use between the two groups, one-way analyses of variance (ANOVAs) with Bonferroni correction for post hoc comparison were calculated.
3. Results
3.1. Sample
Initially, 112 patients were included in this study. One patient was excluded due to missing CIUS data, thus yielding a final sample of 111 patients. The mean age was 15.1 (SD: 1.4) years, and the majority of patients were female (74.8%). 72.1% were still attending school, and about half of the sample were from families with separated or divorced parents. The majority of the patients had been diagnosed with an internalizing disorder (61.3%). About a fourth of the sample had previously tried to commit suicide, and 40.5% of the patients had had at least one episode of self-injury (see Table 1 for details).
Table 1.
Mean age (SD), range | 15.1 (1.4) | 12–17 |
Gender | ||
Male | 32 | 25.2% |
Female | 83 | 74.8% |
Occupation | ||
In school | 80 | 72.1% |
Apprenticeship | 11 | 9.9% |
Other | 20 | 18.0% |
Family status | ||
Parents together | 50 | 45.0% |
Parents separated/divorced | 55 | 49.5% |
One or both parents deceased | 4 | 3.6% |
Data missing | 2 | 1.8% |
Psychiatric main diagnosis | ||
Internalizing disorder (F3, F4, F93) | 68 | 61.3% |
Externalizing disorder (F90–F92) | 12 | 10.8% |
Eating disorder (F66, F50) | 28 | 25.2% |
Other (esp. F1, F98) | 3 | 2.7% |
History of suicide attempt | 26 | 23.4% |
History of nonsuicidal self-injury | 45 | 40.5% |
Patients reported a mean score of 61.4 (SD: 26.8) on the YSR. 73.0% of the sample reported values above the clinical cut-off, with no gender differences (female: 73.0% versus male: 73.1%; χ2 = 0.01, p = 0.99). The large majority of the patients' parents (90.0%) described their children as being significantly distressed; the mean CBCL score was 58.5 (SD: 26.0).
As to PIU, the patients had mean values of 14.8 (SD: 13.8) on the CIUS, with 28.8% reporting values above the cut-off (8.1% problematic internet use, 20.7% addictive internet use). In terms of the subscales of the CIUS, mean values were the highest on the coping subscale and the lowest on withdrawal symptoms.
3.2. Comparison of the Clinical Sample with a Sample of Nonreferred Youth (School Sample)
To test our first hypothesis, we compared our clinical sample to a previously published sample of nonreferred adolescents (school sample, n = 398, mean age 15.2 ± 2.3, 34.2% female; see [2]). Based on CIUS cut-off scores, we categorized the two samples into normal users, problematic users, and pathological users.
The results (see Table 2 and Figure 1) show that our patients reported significantly higher CIUS total values than the adolescents in the control sample (14.8 versus 11.2), with an effect size of d = 0.32. Addictive internet use (20.7% versus 3.4%; odds ratio: 7.0) and problematic internet use (28.8% versus 11.0%; odds ratio: 3.9) were significantly more prevalent in the clinical sample than in the control sample.
Table 2.
Clinical sample (n = 111) | School sample (n = 389) | Cohen's d | t value | p value | |||
---|---|---|---|---|---|---|---|
Mean | SD | Mean | SD | ||||
CIUS | 14.78 | 13.75 | 11.15 | 8.0 | 0.32 | 6.43 | <0.001 |
| |||||||
% | % | Odds Ratio | 95% CI | p | |||
| |||||||
Problematic internet use | 28.8% | 11.0% | 3.9 | 2.3–6.3 | <0.001 | ||
Pathological internet use | 20.7% | 3.4% | 7.0 | 3.7–13.3 | <0.001 |
After twin-matching (age and gender) the two groups, there was still a significant difference between patients and control group: among inpatients, the probability (OR) of addictive internet use was 7.8 times higher and the probability of problematic internet use 3.3 times higher than their adolescent peers in the community (see Figure 1).
Our results therefore indicate that the null hypothesis may be rejected, as adolescents undergoing inpatient psychiatric treatment showed significantly higher rates of dysfunctional internet use (addictive internet use and problematic internet use) than a control group of nonreferred peers.
3.3. Analysis of the Clinical Sample: PIU versus Non-PIU
To test the second hypothesis, we divided the clinical sample into normal users (non-PIU, 71.2%, n = 79) and pathological or problematic users (PIU, 28.8%, n = 32) based on CIUS cut-off scores. The PIU group consisted of users with pathological or addictive internet use (cut-off: 28 points) and a subthreshold group with problematic internet use (cut-off: 21 points). All inpatients with a CIUS value below 21 points were considered normal users.
3.3.1. Sociodemographic Data
Neither age nor family status was significantly associated with the incidence of PIU. Male patients showed higher prevalence rates of PIU (39.3% versus 25.3%), but this difference was not statistically significant (p = 0.23, see Table 3).
Table 3.
Non- PIU | PIU | χ 2/t value | p value | |
---|---|---|---|---|
Mean (SD) | Mean (SD) | |||
Age | 15.2 (1.4) | 14.8 (1.5) | 1.09 | 0.28 |
Family status | ||||
Parents together | 34 (43.6%) | 16 (51.6%) | 1.94 | 0.38 |
Parents separated/divorced | 40 (51.3%) | 15 (48.4%) | ||
One or both parents deceased | 4 (5.1%) | 0 (0.0%) | ||
Gender | ||||
Female | 62 (74.7%) | 21 (25.3%) | 1.99 | 0.23 |
Male | 17 (60.7%) | 11 (39.3%) |
3.3.2. Clinical Data
Patients with PIU showed significantly more suicide attempts than patients without PIU (46.9% versus 13.9%; p < 0.001). With regard to clinical diagnosis, patients with PIU showed more internalizing disorders (71% versus 57%), but this difference was not statistically significant. A small but statistically not significant difference was also found for NSSI, with patients with PIU showing higher rates (46.9% versus 38%; see Table 4).
Table 4.
Non-PIU (n = 79) | PIU (n = 32) | χ 2/t value | p value | |
---|---|---|---|---|
Psychiatric main diagnosis | 4.56 | 0.10 | ||
Externalizing disorder | 45 (8.9%) | 23 (15.6%) | ||
Internalizing disorder | 7 (57.0%) | 5 (71.9%) | ||
Eating disorder | 24 (30.4%) | 4 (12.5%) | ||
History of suicide attempt | 11 (13.9%) | 15 (46.9%) | 13.78 | <0.001 |
Nonsuicidal self-injury (NSSI) | 30 (38.0%) | 15 (46.9%) | 0.75 | 0.39 |
3.3.3. Psychometric Data (AIDA, SCID-II, YSR, CBCL)
Patients with PIU reported a mean score of 123.6 points on the AIDA total score, which was significantly higher than the mean score of patients without PIU (104.1; p = 0.005, d = 0.5), representing a moderate effect size. The analysis of the two subscales showed that patients with PIU had significantly higher scores on both the coherence subscale (68.1 versus 56.1; p = 0.004, d = 0.5) and the discontinuity subscale (55.5 versus 48.0; p = 0.021, see Table 5).
Table 5.
Non-PIU (n = 79) | PIU (n = 32) | Effect size | F value | p ∗ | |||
---|---|---|---|---|---|---|---|
Mean | (SD) | Mean | (SD) | ||||
AIDA | |||||||
Diffusion (total score) | 104.1 | (40.4) | 123.6 | (41.1) | 0.5 | 8.25 | 0.005 |
Discontinuity | 48.0 | (17.2) | 55.5 | (19.8) | 0.4 | 5.48 | 0.021 |
Incoherence | 56.1 | (25.3) | 68.1 | (25.1) | 0.5 | 8.62 | 0.004 |
SCID-II (dimensional scores) | |||||||
PD cluster A | 1.18 | (0.19) | 1.26 | (0.22) | 0.4 | 3.04 | 0.084 |
PD cluster B | 1.17 | (0.13) | 1.25 | (0.20) | 0.5 | 4.5 | 0.036 |
PD cluster C | 1.39 | (0.27) | 1.42 | (0.28) | 0.1 | 0.55 | 0.58 |
YSR and CBCL (scales with significant group differences shown) |
|||||||
YSR | |||||||
Attention problems (score) | 6.8 | (3.9) | 8.8 | (4.3) | 0.5 | 4.20 | 0.043 |
YSR TOT | 59.7 | (27.2) | 66.3 | (25.3) | 0.2 | 2.24 | 0.138 |
CBCL | |||||||
Withdrawn/depressed | 6.9 | (3.2) | 9.4 | (3.2) | 0.8 | 13.92 | 0.001 |
CBCL TOT | 57.9 | (27.5) | 60.6 | (20.3) | 0.1 | 0.97 | 0.329 |
p ∗: all p values after controlling for gender.
The SCID-II showed a significant group difference between PIU and non-PIU for cluster B personality disorders (p = 0.036, d = 0.5), whereas there was no significant group difference for cluster A or cluster C (see Table 5).
While patients with PIU reported significantly higher scores for attention problems than those without PIU (YSR: 8.8 versus 6.8; p = 0.043, d = 0.5), the ratings of parents of patients with PIU differed only on the withdrawn/depressed subscale (CBCL: 9.4 versus 6.9; p = 0.001, d = 0.8). All other subscales and broadband scales of the YSR and CBCL as well as the total behavior problems score (TOT) showed no statistically significant difference between patients with and without PIU (see Table 5).
3.3.4. Peer Victimization
Patients with PIU reported significantly more instances of being bullied than patients without PIU, both offline (12.8 versus 7.0, p = 0.002, d = 0.7) and online (2.9 versus 0.9, p = 0.002, d = 0.8). Only 9 patients in the total sample (7.8%) stated that they had actively bullied other people offline. In our sample, there was no group difference between PIU and non-PIU patients regarding active online or offline bullying (see Table 6).
Table 6.
Number of experiences of being bullied or of active bullying | Non-PIU | PIU | Effect size | t | p ∗ | ||
---|---|---|---|---|---|---|---|
Mean | (SD) | Mean | (SD) | ||||
Being bullied offline | 7.0 | (8.5) | 12.8 | (9.9) | 0.7 | 10.65 | 0.002 |
Being bullied online | 0.9 | (1.5) | 2.1 | (2.9) | 0.8 | 9.84 | 0.002 |
Active bullying offline | 0.05 | (0.2) | 0.2 | (0.4) | 0.3 | 2.42 | 0.123 |
Active bullying online | 0.2 | (0.8) | 0.7 | (1.3) | 0.7 | 3.29 | 0.073 |
p ∗: all p values after controlling for gender.
Our results indicate that for the second hypothesis, the null hypothesis may also be rejected: in terms of other psychopathology, we found significant differences between adolescent inpatients with dysfunctional internet use (addictive internet use and problematic internet use) and inpatients with normal use.
4. Discussion
Our study sought to address two issues: first, whether the prevalence of comorbid PIU in a naturalistic sample of inpatients in a specialized center for child and adolescent psychiatry would differ from a control group of nonreferred adolescents; second, whether adolescent inpatients with dysfunctional internet use and inpatients with normal use show differences concerning other psychopathology.
As the data show, the prevalence of addictive internet use (20.7% versus 3.4%; odds ratio: 7.0) and problematic internet use (28.8% versus 11.0%; odds ratio: 3.9) was significantly higher in the clinical sample than in the control sample, thus confirming the first hypothesis. The differences remained stable after twin-matching the two samples. As described in detail in a previous publication of our group [2], prevalence rates of addictive and problematic internet use in the school sample show high consistency with European large-scale studies [4, 22–24, 27]. This may be indicative of the representativeness of our control sample and of the broader generalizability of our findings. To our knowledge, only one other study [46] has examined addictive internet use as a comorbid disorder among adolescent psychiatry patients, finding a prevalence of 11.3%. In our study, which used a different instrument, 20.7% of the inpatients showed addictive usage patterns. The rise of mobile internet use in recent years may have contributed to this.
Nearly 30% of all adolescent inpatients in our study showed signs of dysfunctional internet use. In clinical practice, therefore, digital media use should be taken into consideration on a routine basis in inpatient treatment and assessment. Our data suggest that young people with mental health issues have difficulties acquiring balanced, competent media use skills.
As to our second hypothesis, data from this study show that within the inpatient group, adolescents showing signs of PIU differed significantly from normal users in several psychopathological features. Figure 2 shows these associations schematically.
First, patients with PIU had significantly more suicide attempts than patients with normal internet use. This finding is in line with several other studies showing a close association between suicidality and PIU [8, 9, 41, 42]. Second, patients with PIU reported more difficulties in establishing a stable and consolidated identity and especially problems with ego strength, suggestibility, awareness of a defined core and inner substance, and understanding motives and behavior. To our knowledge, this study is the first to demonstrate an association between juvenile PIU and identity problems using a structured questionnaire (AIDA). A third association was found between PIU and cluster B personality disorders, especially juvenile borderline personality disorder. To our knowledge, there are no other studies showing this association in adolescent inpatient samples. One study found similar associations in a French adult sample of outpatients [62]. The association of identity diffusion with aspects of personality disorders in our PIU sample indicates that this psychopathological cluster specifically interferes with the consolidation of competent media use. Potentially, instability in interpersonal relationships and problems with identity and ego strength are expressed or compensated in these adolescents via online outlets. Fourth, patients with PIU reported significantly more instances of being bullied than patients without PIU, both offline and online. This is in line with several studies showing this association in adolescent European [63] and Asian [64] samples. In our study, being bullied online (“cyberbullying”) was the far less frequent scenario than being bullied “face to face”, in both the normal use group and the PIU group. This supports the findings of Wolke et al. [65], who describe cyberbullying as a modern tool that supplements traditional forms. Nevertheless, the presence of cyberbullying seems to be an important risk factor for psychological distress and suicidality, especially in adolescents with intensive use of SNS [44]. As a consequence, the combination of PIU, especially the heavy use of SNS, and the presence of cyber-victimization should be specifically assessed.
To our knowledge, no study to date has directly compared the prevalence of PIU in adolescent CAP-IP with a matched control group of nonreferred peers using the same diagnostic instrument. Furthermore, relatively few studies have examined the relationship of PIU and psychiatric comorbidities in the context of inpatient treatment. Our data thus add important new perspectives to the literature. Nevertheless, some limitations should be taken into account. First, the proposed cut-off values for the CIUS [49, 50] have not been empirically determined for adolescents and have been challenged by recent publications [24]. However, using these cut-off values, prevalence figures in our school sample were very close to previous findings gathered in large populational samples from Germany [24, 27], being indicative of the representativeness of our control sample and of the broader generalizability of our findings. Second, there was a time lag of at least a year between the examination of the school sample and data collection among CAP-IP. While a rise of PIU between 2009/2010 and 2011/2012 among European adolescents is described in the literature [66], it is unlikely that this phenomenon explains the large differences between school students and patients in our assessment. Third, only a quantitative comparison between the school sample and the clinical sample was possible because additional data on the mental health of the school students could not be assessed. Fourth, the cross-sectional design of our study provides more of a snapshot than a long-term view of PIU. It is thus important to consider data that suggest that PIU is a volatile syndrome with a tendency to recover over time [67]. Moreover, conclusions on cause and effect of PIU and associated psychopathology cannot be drawn.
5. Conclusions
Problematic and addictive internet use among adolescents undergoing inpatient psychiatric treatment is much more frequent than among their peers in the general population. This suggests that young people with mental health issues have difficulties acquiring balanced, competent media use skills.
Our study and previous findings point to an association between PIU and certain problem clusters, that is, suicidality, difficulties in establishing stable and consolidated identity, and peer victimization.
As a consequence, digital media use should be taken into consideration on a routine basis in inpatient treatment and assessment in child and adolescent psychiatry. Assessing PIU and peer victimization may help to identify a high-risk group and thus be an important contribution to the prevention of PIU and to the treatment of affected adolescents in inpatient care.
Abbreviations
- PMU:
Pathological media use
- PIU:
Pathological internet use
- CAP-IP:
Child and adolescent psychiatry inpatients
- CAP:
Child and adolescent psychiatry
- NSSI:
Nonsuicidal self-injury.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
References
- 1. "InternetWorldStats," Ed., Miniwatts Marketing Group ( http://www.internetworldstats.com), 2017.
- 2.Riedl D., Stöckl A., Nussbaumer C., Rumpold G., Sevecke K., Fuchs M. Usage patterns of internet and computer games: Results of an observational study of Tyrolean adolescents. Neuropsychiatry. 2016;30(4):181–190. doi: 10.1007/s40211-016-0205-y. [DOI] [PubMed] [Google Scholar]
- 3.Frees B., Koch W. Internetnutzung: Frequenz und Vielfalt nehmen in allen Altersgruppen zu. Media Perspektiven. 2015;9(2015) [Google Scholar]
- 4.Durkee T., Kaess M., Carli V., et al. Prevalence of pathological internet use among adolescents in Europe: Demographic and social factors. Addiction. 2012;107(12):2210–2222. doi: 10.1111/j.1360-0443.2012.03946.x. [DOI] [PubMed] [Google Scholar]
- 5.Young K. S. Psychology of computer use: XL. Addictive use of the Internet - A case that breaks the stereotype. Psychological Reports. 1996;79(3):899–902. doi: 10.2466/pr0.1996.79.3.899. [DOI] [PubMed] [Google Scholar]
- 6.Musetti A., Cattivelli R., Giacobbi M., et al. Challenges in internet addiction disorder: Is a diagnosis feasible or not? Frontiers in Psychology. 2016;7, article no. 842 doi: 10.3389/fpsyg.2016.00842. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Guan S.-S. A., Subrahmanyam K. Youth internet use: Risks and opportunities. Current Opinion in Psychiatry. 2009;22(4):351–356. doi: 10.1097/YCO.0b013e32832bd7e0. [DOI] [PubMed] [Google Scholar]
- 8.Durkee T., Carli V., Floderus B., et al. Pathological internet use and risk-behaviors among european adolescents. International Journal of Environmental Research and Public Health. 2016;13(3, article no. 294) doi: 10.3390/ijerph13030294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kaess M., Durkee T., Brunner R., et al. Pathological Internet use among European adolescents: psychopathology and self-destructive behaviours. European Child and Adolescent Psychiatry. 2014;23(11):1093–1102. doi: 10.1007/s00787-014-0562-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tonioni F., Mazza M., Autullo G. Is Internet addiction a psychopathological condition distinct from pathological gambling? Addictive Behaviors. 2014;39(6):1052–1056. doi: 10.1016/j.addbeh.2014.02.016. [DOI] [PubMed] [Google Scholar]
- 11.Lee H. W., Choi J.-S., Shin Y.-C., Lee J.-Y., Jung H. Y., Kwon J. S. Impulsivity in internet addiction: a comparison with pathological gambling. Cyberpsychology, Behavior, and Social Networking. 2012;15(7):373–377. doi: 10.1089/cyber.2012.0063. [DOI] [PubMed] [Google Scholar]
- 12.Goldstein R. Z., Volkow N. D. Dysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications. Nature Reviews Neuroscience. 2011;12(11):652–669. doi: 10.1038/nrn3119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Han D. H., Bolo N., Daniels M. A., Arenella L., Lyoo I. K., Renshaw P. F. Brain activity and desire for Internet video game play. Comprehensive Psychiatry. 2011;52(1):88–95. doi: 10.1016/j.comppsych.2010.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kormas G., Critselis E., Janikian M., Kafetzis D., Tsitsika A. Risk factors and psychosocial characteristics of potential problematic and problematic internet use among adolescents: A cross-sectional study. BMC Public Health. 2011;11, article no. 595 doi: 10.1186/1471-2458-11-595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Brezing C., Derevensky J. L., Potenza M. N. Non-Substance-Addictive Behaviors in Youth: Pathological Gambling and Problematic Internet Use. Child and Adolescent Psychiatric Clinics of North America. 2010;19(3):625–641. doi: 10.1016/j.chc.2010.03.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Grant J. E., Potenza M. N., Weinstein A., Gorelick D. A. Introduction to behavioral addictions. The American Journal of Drug and Alcohol Abuse. 2010;36(5):233–241. doi: 10.3109/00952990.2010.491884. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ko C. H., Liu G. C., Hsiao S., et al. Brain activities associated with gaming urge of online gaming addiction. Journal of Psychiatric Research. 2009;43(7):739–747. doi: 10.1016/j.jpsychires.2008.09.012. [DOI] [PubMed] [Google Scholar]
- 18.American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th. Arlington, Va, USA: American Psychiatric Association; 2013. (DSM-5). [DOI] [Google Scholar]
- 19.Lortie C. L., Guitton M. J. Internet addiction assessment tools: Dimensional structure and methodological status. Addiction. 2013;108(7):1207–1216. doi: 10.1111/add.12202. [DOI] [PubMed] [Google Scholar]
- 20.Kuss D. J., Griffiths M. D., Karila L., Billieux J. Internet addiction: A systematic review of epidemiological research for the last decade. Current Pharmaceutical Design. 2014;20(25):4026–4052. doi: 10.2174/13816128113199990617. [DOI] [PubMed] [Google Scholar]
- 21.Cheng C., Li A. Y.-L. Internet addiction prevalence and quality of (real) life: A meta-Analysis of 31 nations across seven world regions. Cyberpsychology, Behavior, and Social Networking. 2014;17(12):755–760. doi: 10.1089/cyber.2014.0317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Tsitsika A., Janikian M., Schoenmakers T. M., et al. Internet addictive behavior in adolescence: a cross-sectional study in seven European countries. Cyberpsychology, Behavior, and Social Networking. 2014;17(8):528–535. doi: 10.1089/cyber.2013.0382. [DOI] [PubMed] [Google Scholar]
- 23.Blinka L., Škařupová K., Ševčíková A., Wölfling K., Müller K. W., Dreier M. Excessive internet use in european adolescents: what determines differences in severity? International Journal of Public Health. 2014;60(2):249–256. doi: 10.1007/s00038-014-0635-x. [DOI] [PubMed] [Google Scholar]
- 24.Wartberg L., Kriston L., Kammerl R., Petersen K.-U., Thomasius R. Prevalence of pathological internet use in a representative german sample of adolescents: Results of a latent profile analysis. Psychopathology. 2015;48(1):25–30. doi: 10.1159/000365095. [DOI] [PubMed] [Google Scholar]
- 25.Müller K. W., Janikian M., Dreier M., et al. Regular gaming behavior and internet gaming disorder in European adolescents: results from a cross-national representative survey of prevalence, predictors, and psychopathological correlates. European Child and Adolescent Psychiatry. 2015;24(5):565–574. doi: 10.1007/s00787-014-0611-2. [DOI] [PubMed] [Google Scholar]
- 26.Rehbein F., Kliem S., Baier D., Mößle T., Petry N. M. Prevalence of internet gaming disorder in German adolescents: Diagnostic contribution of the nine DSM-5 criteria in a state-wide representative sample. Addiction. 2015;110(5):842–851. doi: 10.1111/add.12849. [DOI] [PubMed] [Google Scholar]
- 27.Rumpf H.-J., Vermulst A. A., Bischof A., et al. Occurence of internet addiction in a general population sample: A latent class analysis. European Addiction Research. 2014;20(4):159–166. doi: 10.1159/000354321. [DOI] [PubMed] [Google Scholar]
- 28.Surís J.-C., Akre C., Piguet C., Ambresin A.-E., Zimmermann G., Berchtold A. Is internet use unhealthy? A cross-sectional study of adolescent internet overuse. Swiss Medical Weekly. 2014;144 doi: 10.4414/smw.2014.14061.w14061 [DOI] [PubMed] [Google Scholar]
- 29.Berkey C. S., Rockett H. R. H., Colditz G. A. Weight Gain in Older Adolescent Females: The Internet, Sleep, Coffee, and Alcohol. Journal of Pediatrics. 2008;153(5):635–e1. doi: 10.1016/j.jpeds.2008.04.072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Cheung L. M., Wong W. S. The effects of insomnia and internet addiction on depression in Hong Kong Chinese adolescents: An exploratory cross-sectional analysis. Journal of Sleep Research. 2011;20(2):311–317. doi: 10.1111/j.1365-2869.2010.00883.x. [DOI] [PubMed] [Google Scholar]
- 31.Do Y. K., Shin E., Bautista M. A., Foo K. The associations between self-reported sleep duration and adolescent health outcomes: What is the role of time spent on Internet use? Sleep Medicine. 2013;14(2):195–200. doi: 10.1016/j.sleep.2012.09.004. [DOI] [PubMed] [Google Scholar]
- 32.Yen C.-F., Ko C.-H., Yen J.-Y., Cheng C.-P. The multidimensional correlates associated with short nocturnal sleep duration and subjective insomnia among Taiwanese adolescents. SLEEP. 2008;31(11):1515–1525. doi: 10.1093/sleep/31.11.1515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Barrense-Dias Y., Berchtold A., Akre C., Surís J.-C. The relation between internet use and overweight among adolescents: A longitudinal study in Switzerland. International Journal of Obesity. 2016;40(1):45–50. doi: 10.1038/ijo.2015.146. [DOI] [PubMed] [Google Scholar]
- 34.Burke V., Beilin L. J., Durkin K., Stritzke W. G. K., Houghton S., Cameron C. A. Television, computer use, physical activity, diet and fatness in Australian adolescents. International Journal of Pediatric Obesity. 2006;1(4):248–255. doi: 10.1080/17477160600984975. [DOI] [PubMed] [Google Scholar]
- 35.Russ S. A., Larson K., Franke T. M., Halfon N. Associations Between Media Use and Health in US Children. Academic Pediatrics. 2009;9(5):300–306. doi: 10.1016/j.acap.2009.04.006. [DOI] [PubMed] [Google Scholar]
- 36.Gür K., Yurt S., Bulduk S., Atagöz S. Internet addiction and physical and psychosocial behavior problems among rural secondary school students. Nursing & Health Sciences. 2015;17(3):331–338. doi: 10.1111/nhs.12192. [DOI] [PubMed] [Google Scholar]
- 37.Nalwa K., Anand A. P. Internet Addiction in Students: A Cause of Concern. Cyberpsychology, Behavior, and Social Networking. 2003;6(6):653–656. doi: 10.1089/109493103322725441. [DOI] [PubMed] [Google Scholar]
- 38.Peltzer K., Pengpid S., Apidechkul T. Heavy Internet use and its associations with health risk and health-promoting behaviours among Thai university students. International Journal of Adolescent Medicine and Health. 2014;26(2):187–194. doi: 10.1515/ijamh-2013-0508. [DOI] [PubMed] [Google Scholar]
- 39.Carli V., Durkee T., Wasserman D., et al. The association between pathological internet use and comorbid psychopathology: A systematic review. Psychopathology. 2012;46(1):1–13. doi: 10.1159/000337971. [DOI] [PubMed] [Google Scholar]
- 40.Ho R. C., Zhang M. W. B., Tsang T. Y., et al. The association between internet addiction and psychiatric co-morbidity: A meta-analysis. BMC Psychiatry. 2014;14(1, article no. 183) doi: 10.1186/1471-244X-14-183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Fischer G., Brunner R., Parzer P., et al. Depression, deliberate self-harm and suicidal behaviour in adolescents engaging in risky and pathological internet use. Praxis der Kinderpsychologie und Kinderpsychiatrie. 2012;61(1):16–31. doi: 10.13109/prkk.2012.61.1.16. [DOI] [PubMed] [Google Scholar]
- 42.Wasserman D. Review of health and risk-behaviours, mental health problems and suicidal behaviours in Young europeans on the basis of the results from the eu-funded Saving and Empowering Young Lives in Europe (SEYLE) study. Psychiatria Polska. 2016;50(6):1093–1107. doi: 10.12740/PP/66954. [DOI] [PubMed] [Google Scholar]
- 43.Spies Shapiro L. A., Margolin G. Growing Up Wired: Social Networking Sites and Adolescent Psychosocial Development. Clinical Child and Family Psychology Review. 2014;17(1):1–18. doi: 10.1007/s10567-013-0135-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Sampasa-Kanyinga H., Hamilton H. A. Social networking sites and mental health problems in adolescents: The mediating role of cyberbullying victimization. European Psychiatry. 2015;30(8):1021–1027. doi: 10.1016/j.eurpsy.2015.09.011. [DOI] [PubMed] [Google Scholar]
- 45.Sampasa-Kanyinga H., Lewis R. F. Frequent Use of Social Networking Sites is Associated with Poor Psychological Functioning among Children and Adolescents. Cyberpsychology, Behavior, and Social Networking. 2015;18(7):380–385. doi: 10.1089/cyber.2015.0055. [DOI] [PubMed] [Google Scholar]
- 46.Müller K. W., Ammerschläger M., Freisleder F. J., Beutel M. E., Wölfling K. Addictive internet use as a comorbid disorder among clients of an adolescent psychiatry - Prevalence and psychopathological symptoms. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie. 2012;40(5):331–339. doi: 10.1024/1422-4917/a000190. [DOI] [PubMed] [Google Scholar]
- 47.Bozkurt H., Coskun M., Ayaydin H., Adak I., Zoroglu S. S. Prevalence and patterns of psychiatric disorders in referred adolescents with Internet addiction. Psychiatry and Clinical Neurosciences. 2013;67(5):352–359. doi: 10.1111/pcn.12065. [DOI] [PubMed] [Google Scholar]
- 48. APA, Diagnostic and statistical manual of mental disorders (4th ed., text rev.), Washington, DC: Amewrican Psychiatric Association, 2000.
- 49.Meerkerk G.-J., van Den Eijnden R. J. J. M., Vermulst A. A., Garretsen H. F. L. The Compulsive Internet Use Scale (CIUS): some psychometric properties. Cyberpsychology, Behavior, and Social Networking. 2009;12(1):1–6. doi: 10.1089/cpb.2008.0181. [DOI] [PubMed] [Google Scholar]
- 50.Guertler D., Rumpf H.-J., Bischof A., et al. Assessment of problematic internet use by the compulsive internet use scale and the internet addiction test: A sample of problematic and pathological gamblers. European Addiction Research. 2014;20(2):75–81. doi: 10.1159/000355076. [DOI] [PubMed] [Google Scholar]
- 51.Wartberg L., Petersen K. U., Kammerl R., Rosenkranz M., Thomasius R. Psychometric validation of a German version of the compulsive Internet use scale. Cyberpsychol Behav Soc Netw. 2014;17(2):99–103. doi: 10.1089/cyber.2012.0689. [DOI] [PubMed] [Google Scholar]
- 52.Goth K., Foelsch P., Schlüter-Müller S., et al. Assessment of identity development and identity diffusion in adolescence - Theoretical basis and psychometric properties of the self-report questionnaire AIDA. Child and Adolescent Psychiatry and Mental Health. 2012;6, article no. 27 doi: 10.1186/1753-2000-6-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Jung E., Pick O., Schlüter-Müller S., Schmeck K., Goth K. Identity development in adolescents with mental problems. Child and Adolescent Psychiatry and Mental Health. 2013;7(1, article no. 26) doi: 10.1186/1753-2000-7-26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Birkhölzer M., Goth K., Schrobildgen C., Schmeck K., Schlüter-Müller S. Background and practical use of the assessment of identity development in adolescence (AIDA) Praxis der Kinderpsychologie und Kinderpsychiatrie. 2015;64(8):584–600. doi: 10.13109/prkk.2015.64.8.584. [DOI] [PubMed] [Google Scholar]
- 55.Fydrich T., Renneberg B., Schmitz B., Wittchen H. Strukturiertes Klinisches Interview für DSM-IV. Achse II: Persönlichkeitsstörungen. Göttingen, Hogrefe; 1997. [Google Scholar]
- 56.Salbach-Andrae H., Bürger A., Klinkowski N., et al. Diagnostic of personality disorders in adolescence according to SCID-II. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie. 2008;36(2):117–125. doi: 10.1024/1422-4917.36.2.117. [DOI] [PubMed] [Google Scholar]
- 57.Achenbach T. Manual for the Child Behavior Checklist and. Burlington VT: Profile., University of Vermont, Department of Psychiatry; 1991. [Google Scholar]
- 58.Dopfner M., Schmeck K., Berner W., Lehmkuhl G., Poustka F. Reliability and factorial validity of the Child Behavior Checklist - An analysis of a clinical sample and a field sample. Zeitschrift fur Kinder- und Jugendpsychiatrie. 1994;22(3):189–205. [PubMed] [Google Scholar]
- 59.Salbach-Andrae H., Klinkowski N., Lenz K., Lehmkuhl U. Agreement between youth-reported and parent-reported psychopathology in a referred sample. European Child and Adolescent Psychiatry. 2009;18(3):136–143. doi: 10.1007/s00787-008-0710-z. [DOI] [PubMed] [Google Scholar]
- 60.Dilling H., Mombour W., Schmidt M., Organization W.-W. H. Internationale Klassifikation psychischer Störungen: ICD10 Kapitel V (F) Klinischdiagnostische Leitlinien. Hogrefe. 2015 [Google Scholar]
- 61.Ravens-Sieberer U., Wille N., Erhart M., et al. Prevalence of mental health problems among children and adolescents in Germany: Results of the BELLA study within the national health interview and examination survey. European Child and Adolescent Psychiatry. 2008;17(1):22–33. doi: 10.1007/s00787-008-1003-2. [DOI] [PubMed] [Google Scholar]
- 62.Laconi S., Andréoletti A., Chauchard E., Rodgers R. F., Chabrol H. Problematic Internet use, time spent online and personality traits. L'Encéphale. 2016;42(3):214–218. doi: 10.1016/j.encep.2015.12.017. [DOI] [PubMed] [Google Scholar]
- 63.Strittmatter E., Brunner R., Fischer G., Parzer P., Resch F., Kaess M. Association of peer victimization, coping, and pathological internet use among adolescents. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie. 2014;42(2):85–94. doi: 10.1024/1422-4917/a000275. [DOI] [PubMed] [Google Scholar]
- 64.Jung Y.-E., Leventhal B., Kim Y. S., et al. Cyberbullying, problematic internet use, and psychopathologic symptoms among Korean youth. Yonsei Medical Journal. 2014;55(3):826–830. doi: 10.3349/ymj.2014.55.3.826. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Wolke D., Lee K., Guy A. Cyberbullying: a storm in a teacup? European Child and Adolescent Psychiatry. 2017;26(8):899–908. doi: 10.1007/s00787-017-0954-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Kaess M., Parzer P., Brunner R., et al. Pathological Internet Use Is on the Rise Among European Adolescents. Journal of Adolescent Health. 2016;59(2):236–239. doi: 10.1016/j.jadohealth.2016.04.009. [DOI] [PubMed] [Google Scholar]
- 67.Strittmatter E., Parzer P., Brunner R., et al. A 2-year longitudinal study of prospective predictors of pathological Internet use in adolescents. European Child and Adolescent Psychiatry. 2016;25(7):725–734. doi: 10.1007/s00787-015-0779-0. [DOI] [PubMed] [Google Scholar]