Assessment of Criteria for Specific Internet-use Disorders (ACSID-11): Introduction of a new screening instrument capturing ICD-11 criteria for gaming disorder and other potential Internet-use disorders

Abstract

Background and aims

With the inclusion of gaming disorder in the ICD-11, diagnostic criteria were introduced for this relatively new disorder. These criteria may also be applied to other potential specific Internet-use disorders, which may be classified in ICD-11 as other disorders due to addictive behaviors, such as online buying-shopping disorder, online pornography-use disorder, social-networks-use disorder, and online gambling disorder. Due to the heterogeneity in existing instruments, we aimed to develop a consistent and economic measure of major types of (potential) specific Internet-use disorders based on ICD-11 criteria for gaming disorder.

Methods

The new 11-item Assessment of Criteria for Specific Internet-use Disorders (ACSID-11) measures five behavioral addictions with the same set of items by following the principles of WHO’s ASSIST. The ACSID-11 was administered to active Internet users (N = 985) together with an adaptation of the Ten-Item Internet Gaming Disorder Test (IGDT-10) and screeners for mental health. We used Confirmatory Factor Analyses to analyze the factor structure of ACSID-11.

Results

The assumed four-factorial structure was confirmed and was superior to the unidimensional solution. This applied to gaming disorder and to the other specific Internet-use disorders. ACSID-11 scores correlated with IGDT-10 as well as with the measures of psychological distress.

Discussion and Conclusions

The ACSID-11 seems to be suitable for the consistent assessment of (potential) specific Internet-use disorders based on ICD-11 diagnostic criteria for gaming disorder. The ACSID-11 may be a useful and economic instrument for studying various behavioral addictions with the same items and improving comparability.

Introduction

The distribution of and easy access to the Internet make online services particularly attractive and offer many advantages. Besides the benefits for most people, online behaviors may take an uncontrolled addictive form in some individuals (e.g., King & Potenza, 2019; Young, 2004). Especially gaming becomes more and more a public health issue (Faust & Prochaska, 2018; Rumpf et al., 2018). After the recognition of ‘Internet gaming disorder’ in the fifth revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013) as a condition of further study, gaming disorder has now been included as an official diagnosis (6C51) in the 11th revision of the International Classification of Diseases (ICD-11; World Health Organization, 2018). This is an important step in addressing the global challenges posed by the harmful use of digital technologies (Billieux, Stein, Castro-Calvo, Higushi, & King, 2021). The worldwide prevalence of gaming disorder is estimated 3.05%, which is comparable to other mental disorders such as substance-use disorders or obsessive-compulsive disorders (Stevens, Dorstyn, Delfabbro, & King, 2021). However, the prevalence estimates vary greatly depending on the screening instrument used (Stevens et al., 2021). At present, the landscape of instruments is manifold. Most measures are based on DSM-5 criteria for Internet gaming disorder and none seems clearly preferable (King et al., 2020). Similar applies to other potential addictive behaviors on the Internet, such as the problematic use of online pornography, social networks, or online shopping. These problematic online behaviors may occur together with gaming disorder (Burleigh, Griffiths, Sumich, Stavropoulos, & Kuss, 2019; Müller et al., 2021), but may also be an own entity. Recent theoretical frameworks such as the Interaction of Person-Affect-Cognition-Execution (I-PACE) model (Brand, Young, Laier, Wölfling, & Potenza, 2016; Brand et al., 2019) assume that similar psychological processes underlie the different types of (online) addictive behaviors. The assumptions are in line with earlier approaches that can be used to explain commonalities between addictive disorders, e.g., regarding neuropsychological mechanisms (Bechara, 2005; Robinson & Berridge, 1993), genetic aspects (Blum et al., 2000), or common components (Griffiths, 2005). However, a comprehensive screening tool for (potential) specific Internet-use disorders based on the same criteria does currently not exist. Uniform screenings across different types of disorders due to addictive behaviors are important to determine commonalities and differences more validly.

In the ICD-11, gaming disorder is listed beyond gambling disorder in the category ‘disorders due to addictive behaviors’. The proposed diagnostic criteria (for both) are: (1) impaired control over the behavior (e.g., onset, frequency, intensity, duration, termination, context); (2) increasing priority given to the behavior to the extent that the behavior takes precedence over other interests and everyday activities; (3) continuation or escalation of the behavior despite negative consequences. Although not directly mentioned as additional criteria, it is mandatory for the diagnosis that the behavioral pattern leads to (4) functional impairment in important areas of daily life (e.g., personal, family, educational, or social issues) and/or marked distress (World Health Organization, 2018). Therefore, both components should be included when studying potential addictive behaviors. Overall, these criteria may also be applied to the category ‘other specified disorders due to addictive behaviors’ (6C5Y), in which buying-shopping disorder, pornography-use disorder, and social-networks-use disorder may potentially be categorized (Brand et al., 2022). Online buying-shopping disorder can be defined by excessive, maladaptive online buying of consumer goods that occurs recurrently despite negative consequences and thus may constitute a specific Internet-use disorder (Müller, Laskowski, et al., 2021). Pornography-use disorder is characterized by diminished control over the consumption of (online) pornographic content, which is separable from other compulsive sexual behaviors (Kraus, Martino, & Potenza, 2016; Kraus et al., 2018). Social-networks-use disorder can be defined by excessive use of social networks (including social networking sites and other online communication applications) characterized by diminished control over the use, increasing priority given to the use, and continuation of the use of social networks despite experiencing negative consequences (Andreassen, 2015). All three potential behavioral addictions constitute clinically relevant phenomena that show similarities with other addictive behaviors (e.g., Brand et al., 2020; Griffiths, Kuss, & Demetrovics, 2014; Müller et al., 2019; Stark, Klucken, Potenza, Brand, & Strahler, 2018).

Instruments assessing specific types of Internet-use disorders are mainly based either on earlier concepts, such as modified versions of Young’s Internet Addiction Test (e.g., Laier, Pawlikowski, Pekal, Schulte, & Brand, 2013; Wegmann, Stodt, & Brand, 2015) or the “Bergen” scales based on Griffiths’ addiction components (e.g., Andreassen, Torsheim, Brunborg, & Pallesen, 2012; Andreassen et al., 2015), or they measure unidimensional constructs based on DSM-5 criteria for gaming disorder (e.g., Lemmens, Valkenburg, & Gentile, 2015; Van den Eijnden, Lemmens, & Valkenburg, 2016) or gambling disorder (for a review see Otto et al., 2020). Some earlier measures have been adopted from measures for gambling disorder, substance-use disorders or have been developed atheoretically (Laconi, Rodgers, & Chabrol, 2014). Many of these instruments show psychometric weaknesses and inconsistencies as highlighted in different reviews (King, Haagsma, Delfabbro, Gradisar, & Griffiths, 2013; Lortie & Guitton, 2013; Petry, Rehbein, Ko, & O’Brien, 2015). King et al. (2020) identified 32 different instruments assessing gaming disorder, which illustrates the inconsistency in the research field. Even the most cited and widely used instruments, such as Young’s Internet Addiction Test (Young, 1998), do not adequately represent the diagnostic criteria for gaming disorder, neither of the DSM-5 nor of the ICD-11. King et al. (2020) further point at psychometric weaknesses, for example, a lack of empirical validation and that most instruments were designed based on the assumption of a unimodal construct. It indicates that the sum of the individual symptoms is counted instead of looking at the frequency and experienced intensity individually. The Ten-Item Internet Gaming Disorder Test (IGDT-10; Király et al., 2017) currently seems to adequately capture the DSM-5 criteria but overall none of the instruments appeared to be clearly preferable (King et al., 2020). Recently, a number of scales were introduced as first screening instruments capturing the ICD-11 criteria for gaming disorder (Balhara et al., 2020; Higuchi et al., 2021; Jo et al., 2020; Paschke, Austermann, & Thomasius, 2020; Pontes et al., 2021) as well as for social-networks-use disorder (Paschke, Austermann, & Thomasius, 2021). In general, it could be assumed that not each symptom is necessarily experienced equally, for example, equally often or equally intensively. It thus seems desirable that screening instruments be able to capture both, the overall symptom experiences, and the totality of symptoms per se. Rather, a multidimensional approach may investigate which symptom contributes decisively, or in different phases, to the development and maintenance of a problematic behavior, is associated with higher level of suffering, or whether it is just a matter of even significance.

Similar problems and inconsistencies become apparent when looking at instruments assessing other types of potential specific Internet-use disorders, namely online buying-shopping disorder, online pornography-use disorder, and social-networks-use disorder. These potential specific Internet-use disorders are not formally classified in ICD-11 in contrast to gaming and gambling disorders. Especially in the case of gambling disorder, numerous screening instruments already exist, but most of them lack adequate evidence (Otto et al., 2020), and neither address the ICD-11 criteria for gambling disorder nor focus on predominantly online gambling disorder (Albrecht, Kirschner, & Grüsser, 2007; Dowling et al., 2019). The ICD-11 lists Compulsive Sexual Behavior Disorder (CSBD), for which many assume that problematic pornography use is a main behavioral symptom, as an impulse-control disorder. Compulsive buying-shopping disorder is listed as an example under the category ‘other specified impulse control disorders’ (6C7Y) but without differentiating between online and offline variants. This differentiation is also not made in the most widely used questionnaires measuring compulsive buying (Maraz et al., 2015; Müller, Mitchell, Vogel, & de Zwaan, 2017). Social-networks-use disorder has not yet been considered in the ICD-11. However, there are evidence-based arguments for each of the three disorders to be rather classified as addictive behaviors (Brand et al., 2020; Gola et al., 2017; Müller et al., 2019; Stark et al., 2018; Wegmann, Müller, Ostendorf, & Brand, 2018). Besides a lack of consensus regarding classification and definitions of these potential specific Internet-use disorders, there are also inconsistencies in the use of screening instruments (for reviews see Andreassen, 2015; Fernandez & Griffiths, 2021; Hussain & Griffiths, 2018; Müller et al., 2017). For example, there are more than 20 instruments supposed to measure problematic pornography use (Fernandez & Griffiths, 2021) but none adequately covers the ICD-11 criteria for disorders due to addictive behaviors, which are very close to the ICD-11 criteria for CSBD.

Furthermore, some specific Internet-use disorders seem likely to co-occur, especially disordered gaming and social-networks use (Burleigh et al., 2019; Müller et al., 2021). Using latent profile analysis, Charzyńska, Sussman, and Atroszko (2021) identified that disordered social-networking and shopping as well as disordered gaming and pornography use often occurred together respectively. The profile including high levels on all Internet-use disorders showed lowest well-being (Charzyńska et al., 2021). This also emphasizes the importance of a comprehensive and uniform screening across different Internet-use behaviors. There have been attempts to use similar sets of items across different Internet-use disorders, such as the Problematic Pornography Consumption Scale (Bőthe et al., 2018), the Bergen Social Media Addiction Scale (Andreassen, Pallesen, & Griffiths, 2017) or the Online Shopping Addiction Scale (Zhao, Tian, & Xin, 2017). However, these scales were designed on the basis of the components model by Griffiths (2005) and do not cover the current proposed criteria for disorders due to addictive behaviors (cf. World Health Organization, 2018).

In summary, the ICD-11 proposed diagnostic criteria for disorders due to (predominantly online) addictive behaviors, namely gambling disorder and gaming disorder. Problematic online pornography use, online buying-shopping, and social-networks use may be assigned to the ICD-11 subcategory ‘other specified disorders due to addictive behaviors’ for which the same criteria can be applied (Brand et al., 2020). To date, the landscape of screening instruments for these (potential) specific Internet-use disorders is highly inconsistent. However, consistent measurement of the different constructs is essential to advance research on commonalities and differences across different types of disorders due to addictive behaviors. Our aim was to develop a short but comprehensive screening instrument for different types of (potential) specific Internet-use disorders covering the ICD-11 criteria for gaming disorder and gambling disorder, to assist with early identification of (potential) specific problematic online behaviors.

Methods

Participants

Participants were recruited online via an access panel service provider through which they were individually remunerated. We included active Internet users from the German-speaking area. We excluded incomplete datasets and those that indicated careless responding. The latter was identified by within-measure (instructed response item and self-report measure) and post-hoc (response time, response pattern, Mahalanobis D) strategies (Godinho, Kushnir, & Cunningham, 2016; Meade & Craig, 2012). The final sample consisted of N = 958 participants (499 male, 458 female, 1 divers) between 16 and 69 years of age (M = 47.60, SD = 14.50). Most participants were full-time employed (46.3%), in (early) retirement (20.1%), or part-time employed (14.3%). The others were students, trainees, housewives/-husbands, or not employed for other reasons. The level of highest vocational education was distributed over completed vocational-in-company training (33.6%), university degree (19.0%), completed vocational-school training (14.1%), graduation from a master school/technical academy (11.8%), and polytechnic degree (10.1%). The others were in education/students or had no degree. The random convenience sample showed a similar distribution of main socio-demographic variables as the population of German Internet users (cf. Statista, 2021).

Measures

Assessment of Criteria for Specific Internet-use Disorders: ACSID-11

With the ACSID-11 we aimed to invent a tool for assessing specific Internet-use disorders in a short but comprehensive, and consistent manner. It was developed based on theory by an expert group of addiction researchers and clinicians. The items were derived in multiple discussions and consensus meetings based on ICD-11 criteria for disorders due to addictive behaviors, as they are described for gaming and gambling, assuming a multifactorial structure. Findings of a Talk-Aloud Analysis were used to optimize content validity and comprehensibility of the items (Schmidt et al., submitted).

The ACSID-11 comprises 11 items that capture the ICD-11 criteria for disorders due to addictive behaviors. The three main criteria, impaired control (IC), increased priority given to the online activity (IP), and continuation/escalation (CE) of Internet use despite negative consequences, are represented by three items each. Two additional items were created to assess functional impairment in daily life (FI) and marked distress (MD) due to the online activity. In a pre-query, participants were instructed to indicate which activities on the Internet they have used at least occasionally in the past 12 months. The activities (i.e., ‘gaming’, ‘online shopping’, ‘use of online pornography’, ‘use of social-networks’, ‘online gambling’, and ‘other’) were listed with corresponding definitions and the response options ‘yes’ or ‘no’. Participants who answered ‘yes’ only to the ‘other’ item were screened out. All others received the ACSID-11 items for all those activities that were answered with ‘yes’. This multibehavioral approach is based on the WHO’s Alcohol, Smoking and Substance Involvement Screening Test (ASSIST; WHO ASSIST Working Group, 2002), which screens for the major categories of substance use and its negative consequences as well as signs of addictive behavior in a consistent way across specific substances.

In analogy to the ASSIST, every item is formulated in a way so that it could be answered directly for the respective activity. We used a two-part response format (see Fig. 1), in which participants should indicate per item for each activity how often they had the experience in the last 12 months (0: ‚never‘, 1: ‚rarely‘, 2: ‚sometimes‘, 3: ‚often‘), and if at least “rarely”, how intense each experience was in the last 12 months (0: ‚not at all intense‘, 1: ‚rather not intense‘, 2: ‚rather intense‘, 3: ‚intense‘). By assessing the frequency as well as the intensity of each symptom, it is possible to investigate the occurrence of a symptom, but also to control for how intense symptoms are perceived beyond the frequency. The items of the ACSID-11 (proposed English translation) are shown in Table 1. The original (German) items including pre-query and instructions can be found in the Appendix (see Appendix A).

Fig. 1.

Exemplary item of the ACSID-11 (proposed English translation of the German original item) illustrating the measurement of frequency (left columns) and intensity (right columns) of situations related to specific online activities. Notes. The figure shows an exemplary item of the factor Impaired Control (IC) as displayed A) to an individual who uses all five online activities as indicated in the pre-query (see Appendix A) and B) to an individual who indicated to use online shopping and social networks only.

Table 1.

Items of the ACSID-11 screener for specific Internet-use disorders (proposed English translation).

Item	Question
IC1	In the past 12 months, have you had trouble keeping track of when you started the activity, for how long, how intensely, or in what situation you did it, or when you stopped?
IC2	In the past 12 months, have you felt the desire to stop or restrict the activity because you noticed you were using it too much?
IC3	In the past 12 months, have you tried to stop or restrict the activity and failed with it?
IP1	In the past 12 months, have you given the activity an increasingly higher priority than other activities or interests in your daily life?
IP2	In the past 12 months, have you lost interest in other activities you used to enjoy because of the activity?
IP3	In the past 12 months, have you neglected or given up other activities or interests that you used to enjoy because of the activity?
CE1	In the past 12 months, have you continued or increased the activity even though it has threatened or caused you to lose a relationship with someone important to you?
CE2	In the past 12 months, have you continued or increased the activity even though it has caused you problems in school/training/work?
CE3	In the past 12 months, have you continued or increased the activity even though it has caused you physical or mental complaints/diseases?
FI1	Thinking about all areas of your life, has your life been noticeably affected by the activity in the past 12 months?
MD1	Thinking about all areas of your life, did the activity cause you suffering in the past 12 months?

Notes. IC = impaired control; IP = increased priority; CE = continuation/escalation; FI = functional impairment; MD = marked distress; The original German items can be found in Appendix A.

Ten-Item Internet Gaming Disorder Test: IGDT-10 – ASSIST version

As a measure of convergent validity, we used the ten-item IGDT-10 (Király et al., 2017) in an extended version. The IGDT-10 operationalizes the nine DSM-5 criteria for Internet gaming disorder (American Psychiatric Association, 2013). In this study, we extended the original gaming specific version so that all forms of specific Internet-use disorders were assessed. To implement this, and to keep the methodology comparable, we also used the multibehavioral response format on the example of ASSIST here. For this, the items were modified so that ’gaming’ was replaced by ’the activity’. Every item was then answered for all online activities that the participants had previously indicated to use (from a selection of ‘gaming’, ‘online shopping’, ‘use of online pornography’, ‘use of social networks’, and ‘online gambling’). Per item, each activity was rated on a three-point Likert scale (0 = ‘never’, 1 = ‘sometimes’, 2 = ‘often’). The scoring was the same as the original version of the IGDT-10: Each criterion received a score of 0 if the response was ‘never’ or ‘sometimes’ and a score of 1 if the response was ‘often’. Items 9 and 10 represent the same criterion (i.e., ‘jeopardy or losing a significant relationship, job, or educational or career opportunity because of participation in Internet games’) and count together one point if one or both items are met. A final sum score was calculated for each activity. It could range from 0 to 9 with higher scores indicating higher symptom severity. Regarding gaming disorder, a score of five or more indicates clinical relevance (Király et al., 2017).

Patient Health Questionnaire-4: PHQ-4

The Patient Health Questionnaire-4 (PHQ-4; Kroenke, Spitzer, Williams, & Löwe, 2009) is a brief measure of symptoms of depression and anxiety. It consists of four items taken from the Generalized Anxiety Disorder–7 scale and the PHQ-8 module for depression. Participants should indicate the frequency of occurrence of certain symptoms on a four-point Likert scale ranging from 0 (‘not at all’) to 3 (‘nearly every day’). The total score can range between 0 and 12 indicating none/minimal, mild, moderate, and severe levels of psychological distress with scores from 0–2, 3–5, 6–8, 9–12, respectively (Kroenke et al., 2009).

General well-being

General life satisfaction was assessed using the Life Satisfaction Short Scale (L-1) in the German original version (Beierlein, Kovaleva, László, Kemper, & Rammstedt, 2015) answered on an 11-point Likert scale ranging from 0 (‘not at all satisfied’) to 10 (‘completely satisfied’). The single item scale is well-validated and correlates strongly with multiple-item-scales assessing satisfaction with life (Beierlein et al., 2015). We additionally asked for specific life satisfaction in the domain of health (H-1): ‘All things considered, how satisfied are you with your health these days?’ answered on the same 11-point scale (cf. Beierlein et al., 2015).

Procedure

The study was conducted online using the online survey tool Limesurvey®. The ACSID-11 and IGDT-10 were implemented in such a way that only the activities that were selected in the pre-query were displayed for the respective items. Participants received individualized links from the service panel provider that led to the online survey created by us. After completion, participants were redirected back to the provider’s website to receive their renumeration. Data was collected in the period from April 8 to April 14 in 2021.

Statistical analyses

We used confirmatory factor analysis (CFA) to test the dimensionality and construct validity of the ACSID-11. The analyses were run with Mplus version 8.4 (Muthén & Muthén, 2019) using weighted least squares means and variance adjusted (WLSMV) estimation. To evaluate model fit, we used multiple indices, namely the chi-square (χ ²) test for exact fit, the Comparative Fit Index (CFI), the Tucker-Lewis fit index (TLI), Standardized Root Mean Square Residual (SRMR), and the Root Mean Square Error of Approximation (RMSEA). According to Hu and Bentler (1999), cutoff values for CFI and TLI > 0.95, for SRMR < 0.08, and for RMSEA < 0.06 indicate good model fit. Furthermore, a chi-square value divided by degrees of freedom (χ²/df) < 3 is another indicator for acceptable model fit (Carmines & McIver, 1981). Cronbach’s alpha (α) and Guttman’s Lambda-2 (λ ₂) were used as measures of reliability with coefficients > 0.8 (> 0.7) indicating good (acceptable) internal consistency (Bortz & Döring, 2006). Correlation analyses (Pearson) were used to test convergent validity between different measures of the same or related constructs. These analyses were run with IBM SPSS statistics (version 26). According to Cohen (1988), a value of |r| = 0.10, 0.30, 0.50 indicates a small, medium, large effect, respectively.

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki. The study was approved by the ethics committee of the division of Computer Science and Applied Cognitive Sciences at the Faculty of Engineering of the University of Duisburg-Essen. All subjects were informed about the study and all provided informed consent.

Results

Within the current sample, the specific Internet-use behaviors were distributed as follows: Gaming was indicated by 440 (45.9%) individuals (age: M = 43.59, SD = 14.66; 259 male, 180 female, 1 divers), 944 (98.5%) of the individuals engaged in online shopping (age: M = 47.58, SD = 14.49; 491 male, 452 female, 1 divers), 340 (35.5%) of the individuals used online-pornography (age: M = 44.80, SD = 14.96; 263 male, 76 female, 1 divers), 854 (89.1%) of the individuals used social networks (age: M = 46.52, SD = 14.66; 425 male, 428 female, 1 divers), and 200 (20.9%) individuals engaged in online gambling (age: M = 46.91, SD = 13.67; 125 male, 75 female, 0 divers). The minority of participants (n = 61; 6.3%) indicated to use only one activity. Most participants (n = 841; 87.8%) used at least online shopping together with social-networks and 409 (42.7%) of them also indicated to play online games. Sixty-eight (7.1%) of the participants indicated to use all of the mentioned online activities.

Given that gaming and gambling disorders are the two types of disorders due to addictive behaviors that are officially recognized and given that the number of individuals in our sample who reported to do online gambling was rather limited, we will first concentrate on the results regarding the assessment of criteria for gaming disorder with the ACSID-11.

Descriptive statistics

Regarding gaming disorder, all ACSID-11 items have ratings between 0 and 3 which reflects the maximum range of possible values (see Table 2). All items show relatively low mean values and a right-skewed distribution as expected in a non-clinical sample. Difficulty is highest for Continuation/Escalation and Marked Distress items while Impaired Control (especially IC1) and Increased Priority items are of lowest difficulty. Kurtosis is especially high for the first item of Continuation/Escalation (CE1) and the Marked Distress item (MD1).

Table 2.

Descriptive statistics of the ACSID-11 items measuring gaming disorder.

No.	Item	Min	Max	M	(SD)	Skewness	Kurtosis	Difficulty
a)	Frequency scale
01a	IC1	0	3	0.827	(0.956)	0.808	−0.521	27.58
02a	IC2	0	3	0.602	(0.907)	1.237	0.249	20.08
03a	IC3	0	3	0.332	(0.723)	2.163	3.724	11.06
04a	IP1	0	3	0.623	(0.895)	1.180	0.189	20.76
05a	IP2	0	3	0.405	(0.784)	1.913	2.698	13.48
06a	IP3	0	3	0.400	(0.784)	1.903	2.597	13.33
07a	CE1	0	3	0.170	(0.549)	3.561	12.718	5.68
08a	CE2	0	3	0.223	(0.626)	3.038	8.797	7.42
09a	CE3	0	3	0.227	(0.632)	2.933	7.998	7.58
10a	FI1	0	3	0.352	(0.712)	1.997	3.108	11.74
11a	MD1	0	3	0.155	(0.526)	3.647	13.107	5.15
b)	Intensity scale
01b	IC1	0	3	0.593	(0.773)	1.173	0.732	19.77
02b	IC2	0	3	0.455	(0.780)	1.700	2.090	15.15
03b	IC3	0	3	0.248	(0.592)	2.642	6.981	8.26
04b	IP1	0	3	0.505	(0.827)	1.529	1.329	16.82
05b	IP2	0	3	0.330	(0.703)	2.199	4.123	10.98
06b	IP3	0	3	0.302	(0.673)	2.302	4.633	10.08
07b	CE1	0	3	0.150	(0.505)	3.867	15.672	5.00
08b	CE2	0	3	0.216	(0.623)	3.159	9.623	7.20
09b	CE3	0	3	0.207	(0.608)	3.225	10.122	6.89
10b	FI1	0	3	0.284	(0.654)	2.534	6.172	9.47
11b	MD1	0	3	0.139	(0.483)	3.997	16.858	4.62

Notes. N = 440. IC = impaired control; IP = increased priority; CE = continuation/escalation; FI = functional impairment; MD = marked distress.

Regarding mental health, the overall sample (N = 958) has a mean PHQ-4 score of 3.03 (SD = 2.82) and shows moderate levels of satisfaction with life (L-1: M = 6.31, SD = 2.39) and health (H-1: M = 6.05, SD = 2.68). In the gaming subgroup (n = 440), 13 individuals (3.0%) reach the IGDT-10 cutoff for clinically relevant cases of gaming disorder. The mean IGDT-10 score varies between 0.51 for buying-shopping disorder and 0.77 for social-networks-use disorder (see Table 5).

Table 5.

Descriptive statistics of the factor and overall scores of the ACSID-11 and IGDT-10 (ASSIST version) for specific Internet-use disorders.

	Gaming (n = 440)				Online buying-shopping (n = 944)				Online pornography use (n = 340)				Social-networks use (n = 854)				Online gambling (n = 200)
Variable	Min	Max	M	(SD)	Min	Max	M	(SD)	Min	Max	M	(SD)	Min	Max	M	(SD)	Min	Max	M	(SD)
Frequency
ACSID-11_IC	0	3	0.59	(0.71)	0	3	0.46	(0.67)	0	3	0.58	(0.71)	0	3	0.78	(0.88)	0	3	0.59	(0.82)
ACSID-11_IP	0	3	0.48	(0.69)	0	3	0.28	(0.56)	0	3	0.31	(0.59)	0	3	0.48	(0.71)	0	3	0.38	(0.74)
ACSID-11_CE	0	3	0.21	(0.51)	0	3	0.13	(0.43)	0	3	0.16	(0.45)	0	3	0.22	(0.50)	0	3	0.24	(0.60)
ACSID-11_FI	0	3	0.25	(0.53)	0	3	0.18	(0.48)	0	2.5	0.19	(0.47)	0	3	0.33	(0.61)	0	3	0.33	(0.68)
ACSID-11_total	0	3	0.39	(0.53)	0	3	0.27	(0.47)	0	2.6	0.32	(0.49)	0	3	0.46	(0.59)	0	2.7	0.39	(0.64)
Intensity
ACSID-11_IC	0	3	0.43	(0.58)	0	3	0.34	(0.56)	0	3	0.45	(0.63)	0	3	0.60	(0.76)	0	3	0.47	(0.73)
ACSID-11_IP	0	3	0.38	(0.62)	0	3	0.22	(0.51)	0	3	0.25	(0.51)	0	3	0.40	(0.67)	0	3	0.35	(0.69)
ACSID-11_CE	0	3	0.19	(0.48)	0	3	0.11	(0.39)	0	2.7	0.15	(0.41)	0	3	0.19	(0.45)	0	3	0.23	(0.58)
ACSID-11_FI	0	3	0.21	(0.50)	0	3	0.15	(0.45)	0	2.5	0.18	(0.43)	0	3	0.28	(0.57)	0	3	0.29	(0.61)
ACSID-11_total	0	3	0.31	(0.46)	0	3	0.21	(0.42)	0	2.6	0.26	(0.43)	0	3	0.37	(0.54)	0	3	0.34	(0.59)
IGDT-10_sum	0	9	0.69	(1.37)	0	9	0.51	(1.23)	0	7	0.61	(1.06)	0	9	0.77	(1.47)	0	9	0.61	(1.41)

Notes. ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items; IC = impaired control; IP = increased priority; CE = continuation/escalation; FI = functional impairment; IGDT-10 = Ten-Item Internet Gaming Disorder Test.

Confirmatory factor analysis

Assumed four-factor model

We tested the assumed four-factorial structure of ACSID-11 by means of multiple CFAs, one per specific Internet-use disorder and separately for frequency and intensity ratings. The factors (1) Impaired Control, (2) Increased Priority, and (3) Continuation/Escalation were formed by the respective three items. The two additional items measuring functional impairment in daily life and marked distress due to the online activity formed the additional factor (4) Functional Impairment. The four-factorial structure of the ACSID-11 is supported by the data. The fit indices indicate a good fit between the models and the data for all types of specific Internet-use disorders assessed by ACSID-11, namely gaming disorder, online buying-shopping disorder, and social-networks-use disorder, online pornography-use disorder, and online gambling disorder (see Table 3). Regarding online pornography-use disorder and online gambling disorder, TLI and RMSEA could be biased due to small sample sizes (Hu & Bentler, 1999). The factor loadings and residual covariances for the CFAs applying a four-factor model are shown in Fig. 2. To note, some of the models show singular anomalous values (i.e., negative residual variance for a latent variable or correlations of equal to or greater than 1).

Table 3.

Fit indices of the four-factor, unidimensional, and second-order CFA models for specific (potential) Internet-use disorders measured by ACSID-11.

		Gaming disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ 2/df	CFI	TLI	SRMR	RMSEA	χ 2/df
Four-factor model	38	0.991	0.987	0.031	0.051	2.13	0.993	0.990	0.029	0.043	1.81
Unidimensional model	27	0.969	0.961	0.048	0.087	4.32	0.970	0.963	0.047	0.082	3.99
Second-order factor model	40	0.992	0.988	0.031	0.047	1.99	0.992	0.989	0.032	0.045	1.89

		Online buying-shopping disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ 2/df	CFI	TLI	SRMR	RMSEA	χ 2/df
Four-factor model	38	0.996	0.994	0.019	0.034	2.07	0.995	0.992	0.020	0.037	2.30
Unidimensional model	27	0.981	0.976	0.037	0.070	5.58	0.986	0.982	0.031	0.056	3.98
Second-order factor model	40	0.996	0.994	0.021	0.036	2.19	0.994	0.992	0.023	0.038	2.40

		Online pornography-use disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ 2/df	CFI	TLI	SRMR	RMSEA	χ 2/df
Four-factor model	38	0.993	0.989	0.034	0.054	1.99	0.987	0.981	0.038	0.065	2.43
Unidimensional model	27	0.984	0.979	0.044	0.075	2.91	0.976	0.970	0.046	0.082	3.27
Second-order factor model	40	0.993	0.991	0.033	0.049	1.83	0.984	0.979	0.039	0.068	2.59

		Social-networks-use disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ 2/df	CFI	TLI	SRMR	RMSEA	χ 2/df
Four-factor model	38	0.993	0.990	0.023	0.049	3.03	0.993	0.989	0.023	0.052	3.31
Unidimensional model	27	0.970	0.963	0.048	0.096	8.89	0.977	0.972	0.039	0.085	7.13
Second-order factor model	40	0.992	0.989	0.027	0.053	3.39	0.991	0.988	0.025	0.056	3.64

		Online gambling disorder
		Frequency					Intensity
Model	df	CFI	TLI	SRMR	RMSEA	χ 2/df	CFI	TLI	SRMR	RMSEA	χ 2/df
Four-factor model	38	0.997	0.996	0.027	0.059	1.70	0.997	0.996	0.026	0.049	1.47
Unidimensional model	27	0.994	0.992	0.040	0.078	2.20	0.991	0.989	0.039	0.080	2.28
Second-order factor model	40	0.997	0.996	0.029	0.054	1.58	0.997	0.995	0.029	0.053	1.55

Notes. Sample sizes varied for gaming (n = 440), online shopping (n = 944), online-pornography use (n = 340), social-networks use (n = 854), and online gambling (n = 200); ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items.

Fig. 2.

Factor loadings and residual covariances of the four-factor models of the ACSID-11 (frequency) for (A) gaming disorder, (B) online gambling disorder, (C) online buying-shopping disorder, (D) online pornography-use disorder, and (E) social-networks-use disorder. Notes. Sample sizes varied for gaming (n = 440), online shopping (n = 944), online-pornography use (n = 340), social-networks use (n = 854), and online gambling (n = 200); The intensity scale of the ACSID-11 showed similar results. ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items; Values represent standardized factor loadings, factor covariances, and residual covariances. All estimates were significant at p < 0.001.

Unidimensional model

Due to high intercorrelations between the different factors, we additionally tested unidimensional solutions with all items loading on one factor, as implemented, e.g., in the IGDT-10. The unidimensional models of the ACSID-11 showed acceptable fit, but with RMSEA and/or χ²/df being above the suggested cutoffs. For all behaviors, the model fits for the four-factor models are better compared to the respective unidimensional models (see Table 3). Consequently, the four-factor solution appears to be superior to the unidimensional solution.

Second-order factor model and bifactor model

An alternative to account for the high intercorrelations is to include a general factor representing the general construct, which is comprised of related subdomains. This can be implemented via second-order factor model and bifactor model. In the second-order factor model, a general (second-order) factor is modeled in an attempt to explain the correlations among the first-order factors. In the bifactor model, it is assumed that the general factor accounts for the commonality between the related domains and that, additionally, there are multiple specific factors, each of which has unique effects on and beyond the general factor. This is modeled so that each item is allowed to load on the general factor as well as on its specific factor where all factors (including correlations between general factor and specific factors) are specified to be orthogonal. The second-order factor model is more constrained than the bifactor model and is nested within the bifactor model (Yung, Thissen, & McLeod, 1999). In our samples, the second-order factor models show similar good fit as the four-factor models (see Table 3). For all behaviors, the four (first-order) factors load high on the (second-order) general factor (see Appendix B), which justifies the use of an overall score. As with the four-factor models, some of the second-order factor models show occasional anomalous values (i.e., negative residual variance for a latent variable or correlations of equal to or greater than 1). We also tested complementary bifactor models which showed comparably superior fit, however, not for all behaviors a model could be identified (see Appendix C).

Reliability

Based on the identified four-factorial structure, we calculated factor scores for the ACSID-11 from the means of the respective items as well as overall mean scores for each specific (potential) Internet-use disorder. We had a look at the reliability of the IGDT-10 as we used the multibehavioral variant following the example of the ASSIST (assessing multiple specific Internet-use disorders) for the first time. The results indicate high internal consistency of the ACSID-11 and lower but also acceptable reliability of the IGDT-10 (see Table 4).

Table 4.

Reliability measures of the ACSID-11 and IGDT-10 measuring specific Internet-use disorders.

	ACSID-11				IGDT-10
	Frequency		Intensity		(ASSIST version)
Type of disorder	α	λ2	α	λ2	α	λ2
Gaming	0.900	0.903	0.894	0.897	0.841	0.845
Online buying-shopping	0.910	0.913	0.915	0.917	0.858	0.864
Online pornography use	0.907	0.911	0.896	0.901	0.793	0.802
Social-networks use	0.906	0.912	0.915	0.921	0.855	0.861
Online gambling	0.947	0.950	0.944	0.946	0.910	0.912

Notes. α = Cronbach's alpha; λ ₂ = Guttman's lambda-2; ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11 items; IGDT-10 = Ten-Item Internet Gaming Disorder Test; Sample sizes varied for gaming (n = 440), online buying-shopping (n = 944), online-pornography use (n = 340), social-networks use (n = 854), and online gambling (n = 200).

Table 5 shows the descriptive statistics of the ACSID-11 and IGDT-10 scores. For all behaviors, the means of the ACSID-11 factors Continuation/Escalation and Functional Impairment are the lowest compared with those of the other factors. The factor Impaired Control shows highest mean values for both frequency and intensity. The ACSID-11 total scores are highest for social-networks-use disorder, followed by online gambling disorder and gaming disorder, online pornography-use disorder, and online buying-shopping disorder. IGDT-10 sum scores show a similar picture (see Table 5).

Correlation analysis

As a measure of construct validity, we analyzed correlations between ACSID-11, IGDT-10, and measures of general well-being. The correlations are shown in Table 6. The ACSID-11 total scores correlate positively with the IGDT-10 scores with medium to large effect sizes, where the correlations between the scores for the same behaviors are highest. Furthermore, ACSID-11 scores correlate positively with PHQ-4, with a similar effect as IGDT-10 and PHQ-4 do. Correlation patterns with measures of life satisfaction (L-1) and health satisfaction (H-1) are very similar between symptom severity assessed with ACSID-11 and that with IGDT-10. Intercorrelations between ACSID-11 total scores for the different behaviors are of large effects. Correlations between the factor scores and IGDT-10 can be found in the supplemental material.

Table 6.

Correlations between ACSID-11 (frequency), IGDT-10, and measures of psychological well-being

			1)	2)	3)	4)	5)	6)	7)	8)	9)	10)	11)	12)
	ACSID-11_total
1)	Gaming		1
2)	Online buying-shopping	r	0.703**	1
		(n)	(434)	(944)
3)	Online pornography use	r	0.659**	0.655**	1
		(n)	(202)	(337)	(340)
4)	Social-networks use	r	0.579**	0.720**	0.665**	1
		(n)	(415)	(841)	(306)	854
5)	Online gambling	r	0.718**	0.716**	0.661**	0.708**	1
		(n)	(123)	(197)	(97)	(192)	(200)
	IGDT-10_sum
6)	Gaming	r	0.596**	0.398**	0.434**	0.373**	0.359**	1
		(n)	(440)	(434)	(202)	(415)	(123)	(440)
7)	Online buying-shopping	r	0.407**	0.632**	0.408**	0.449**	0.404**	0.498**	1
		(n)	(434)	(944)	(337)	(841)	(197)	(434)	(944)
8)	Online pornography use	r	0.285**	0.238**	0.484**	0.271**	0.392**	0.423**	0.418**	1
		(n)	(202)	(337)	(340)	(306)	(97)	(202)	(337)	(340)
9)	Social-networks use	r	0.255**	0.459**	0.404**	0.591**	0.417**	0.364**	0.661**	0.459**	1
		(n)	(415)	(841)	(306)	(854)	(192)	(415)	(841)	(306)	(854)
10)	Online gambling	r	0.322**	0.323**	0.346**	0.423**	0.625**	0.299**	0.480**	0.481**	0.525**	1
		(n)	(123)	(197)	(97)	(192)	(200)	(123)	(197)	(97)	(192)	(200)
11)	PHQ-4	r	0.292**	0.273**	0.255**	0.350**	0.326**	0.208**	0.204**	0.146**	0.245**	0.236**	1
		(n)	(440)	(944)	(340)	(854)	(200)	(440)	(944)	(340)	(854)	(200)	(958)
12)	L-1	r	−0.069	−0.080*	−0.006	−0.147**	−0.179*	−0.130**	−0.077*	−0.018	−0.140**	−0.170*	−0.542**	1
		(n)	(440)	(944)	(340)	(854)	(200)	(440)	(944)	(340)	(854)	(200)	(958)	(958)
13)	H-1	r	−0.083	−0.051	0.062	−0.014	0.002	−0.078	−0.021	0.069	0.027	−0.034	−0.409**	0.530**
		(n)	(440)	(944)	(340)	(854)	(200)	(440)	(944)	(340)	(854)	(200)	(958)	(958)

Notes. ** p < 0.01; * p < 0.05. ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items; IGDT-10 = Ten-Item Internet Gaming Disorder Test; PHQ-4 = Patient Health Questionnaire-4; Correlations with the ACSID-11 intensity scale were in a similar range.

Discussion and Conclusions

This report introduced the ACSID-11 as a new tool for the easy and comprehensive screening of major types of specific Internet-use disorders. The results of the study indicate that ACSID-11 is suitable to capture the ICD-11 criteria for gaming disorder in a multifaceted structure. Positive correlations with a DSM-5 based assessment tool (IGDT-10) further indicated construct validity.

The assumed multifactorial structure of the ACSID-11 was confirmed by the results of the CFA. The items fit well with a four-factor model representing the ICD-11 criteria (1) impaired control, (2) increased priority, (3) continuation/escalation despite negative consequences, as well as the additional components (4) functional impairment and marked distress to be considered as relevant for addictive behaviors. The four-factor solution showed superior fit compared to the unidimensional solution. The multidimensionality of the scale is a unique feature compared to other scales covering ICD-11 criteria for gaming disorder (cf. King et al., 2020; Pontes et al., 2021). Furthermore, the equally superior fit of the second-order factor model (and partly bifactor model) indicate that the items assessing the four related criteria comprise a general “disorder” construct and justifies the use of an overall score. The results were similar for online gambling disorder and the other potential specific Internet-use disorders measured by ACSID-11 in the multibehavioral format on the example of the ASSIST, namely online buying-shopping disorder, online pornography-use disorder, social-networks-use disorder. For the latter, there are hardly any instruments based on WHO criteria for disorders due to addictive behaviors, although researchers recommend this classification for each of them (Brand et al., 2022; Müller et al., 2019; Stark et al., 2018). New comprehensive measures, such as the ACSID-11, can help to overcome the methodological difficulties and enable systematic analyses of commonalities and differences between these different types of (potential) addictive behaviors.

The reliability of the ACSID-11 is high. For gaming disorder, the internal consistency is comparable or higher than that of most other instruments (cf. King et al., 2020). Reliability in terms of internal consistency is also good for the other specific Internet-use disorders measured by both ACSID-11 and IGDT-10. From this we can conclude that an integrated response format, such as that of the ASSIST (WHO ASSIST Working Group, 2002) is suitable for a joint assessment of different types of behavioral addictions. In the current sample, the ACSID-11 total score was highest for social-networks-use disorder. This fits with the relatively high prevalence of this phenomenon which is currently estimated at 14% for individualist countries and 31% for collectivist countries (Cheng, Lau, Chan, & Luk, 2021).

Convergent validity is indicated by medium to large positive correlations between ACSID-11 and IGDT-10 scores despite different scoring formats. Furthermore, the moderate positive correlations between ACSID-11 scores and the PHQ-4 measuring symptoms of depression and anxiety supports the criterion validity of the new assessment tool. The results are consistent with previous findings on associations between (comorbid) mental problems and specific Internet-use disorders including gaming disorder (Mihara & Higuchi, 2017; but see; Colder Carras, Shi, Hard, & Saldanha, 2020), pornography-use disorder (Duffy, Dawson, & Das Nair, 2016), buying-shopping disorder (Kyrios et al., 2018), social-networks-use disorder (Andreassen, 2015), and gambling disorder (Dowling et al., 2015). Also, the ACSID-11 (especially online gambling disorder and social-networks-use disorder) was inversely correlated with the measure of life satisfaction. This result is consistent with previous findings on associations between impaired well-being and symptom severity of specific Internet-use disorders (Cheng, Cheung, & Wang, 2018; Duffy et al., 2016; Duradoni, Innocenti, & Guazzini, 2020). Studies suggest well-being to be particularly impaired when multiple specific Internet-use disorders co-occur (Charzyńska et al., 2021). The joint occurrence of specific Internet-use disorders is not infrequent (e.g., Burleigh et al., 2019; Müller et al., 2021) which may partly explain the relatively high intercorrelations between the disorders measured by ACSID-11 and IGDT-10 respectively. This underscores the importance of a uniform screening tool to determine commonalities and differences more validly across different types of disorders due to addictive behaviors.

A main limitation of the current study is the non-clinical, relatively small and non-representative sample. Thus, with this study, we cannot show whether ACSID-11 is suitable as a diagnostic tool, as we cannot provide clear cutoff scores, yet. Furthermore, the cross-sectional design did not allow to make inferences about test-retest reliability or causal relationships between ACSID-11 and the validating variables. The instrument needs further validation to verify its reliability and suitability. However, the results from this initial study suggest that it is a promising tool that may be worth testing further. To note, a larger data base is needed not only for this instrument, but for the entire field of research to determine which of these behaviors can be considered diagnostic entities (cf. Grant & Chamberlain, 2016). The structure of the ACSID-11 appears to work well as confirmed by the results of the current study. The four specific factors and the general domain were adequately represented across the different behaviors, although each item was answered for all indicated online activities done at least occasionally in the last twelve months. We already discussed that specific Internet-use disorders are likely to co-occur, nevertheless, this must be confirmed in follow-up studies as the reason for the moderate to high correlations of ACSID-11 scores across behaviors. Furthermore, occasional anomalous values might indicate that for some behaviors the model specification needs to be optimized. The criteria used are not necessarily equally relevant to all of the included types of potential disorders. It may be possible that ACSID-11 cannot adequately cover disorder-specific features in symptom manifestations. Measurement invariance across the different versions should be tested with new independent samples including patients with diagnosed specific Internet-use disorders. Furthermore, the results are not representative of the general population. The data approximately represent Internet users in Germany and there was no lockdown at the time of the data collection; nevertheless, the COVID-19 pandemic has a potential influence on stress levels and (problematic) Internet use (Király et al., 2020). Although the single-item L-1 scale is well-validated (Beierlein et al., 2015), (domain-specific) life satisfaction could be captured more comprehensively in future studies using the ACSID-11.

In conclusion, the ACSID-11 proved to be suitable for the comprehensive, consistent, and economic assessment of symptoms of (potential) specific Internet-use disorders including gaming disorder, online buying-shopping disorder, online pornography-use disorder, social-networks-use disorder, and online gambling disorder based on ICD-11 diagnostic criteria for gaming disorder. Further evaluation of the assessment tool should be conducted. We hope that the ACSID-11 may contribute to a more consistent assessment of addictive behaviors in research and that it may become helpful also in clinical practice in the future.

Funding sources

Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 411232260.

Authors’ contribution

SMM: Methodology, Formal analysis, Writing - Original Draft; EW: Conceptualization, Methodology, Writing - Review & Editing; AO: Methodology, Formal analysis; RS: Conceptualization, Methodology; AM: Conceptualization, Methodology; CM: Conceptualization, Methodology; KW: Conceptualization, Methodology; HJR: Conceptualization, Methodology; MB: Conceptualization, Methodology, Writing - Review & Editing, Supervision.

Conflict of interest

The authors report no financial or other conflict of interest relevant to the subject of this article.

Supplementary Materials

jba-11-427-s001.xlsx ^{(33.6KB, xlsx)}

Appendix

Appendix A Original version of the Assessment of Criteria for Specific Internet-use Disorders (ACSID-11) including pre-query and instructions

Pre-Query (German original [proposed English translation]):

Auswahl der Aktivitäten im Internet

[Selection of activities on the Internet]

Im Folgenden geht es um verschiedene Aktivitäten im Internet. Bitte geben Sie dabei an, welche Aktivitäten im Internet Sie in den letzten 12 Monaten mindestens gelegentlich genutzt haben.

[The following is about various activities on the Internet. Please indicate which activities on the Internet you have used at least occasionally in the last 12 months.]

	Ja [Yes]	Nein [No]
Gaming [Gaming] Dies umfasst das Spielen von Computer-, Video- und Online-Spielen. Diese lassen sich aufteilen in verschiedene Genres wie MMORPGs (z.B. World of Warcraft, Guild Wars), First Person Shooters (z.B. Call of Duty, Overwatch, Battlefield), Sport-Simulationen (z.B. Fifa), MOBAs (z.B. League of Legends, Dota) oder Strategiespiele (z.B. Desperados, Age of Empires). Sie können auf unterschiedlichen Konsolen wie NintendoSwitch, PlayStation, Xbox oder auf dem PC gespielt werden. [This includes the playing of computer, video and online games. These can be divided into different genres such as MMORPGs (e.g., World of Warcraft, Guild Wars), first person shooters (e.g., Call of Duty, Overwatch, Battlefield), sports simulations (e.g., Fifa), MOBAs (e.g., League of Legends, Dota) or strategy games (e.g., Desperados, Age of Empires). They can be played on different consoles like NintendoSwitch, PlayStation, Xbox or on the PC.]	☐	☐
Online-Shopping [Online shopping] Dies umfasst Online-Shoppingaktivitäten im Einzelhandel (z.B. Amazon, Zalando) oder Auktionshandel (z.B. eBay). Die Nutzung beinhaltet den tatsächlichen Kauf von Waren als auch andere Aktivitäten (z.B. Preisvergleiche, Produktsuche). [This includes online shopping activities in retail (e.g., Amazon, Zalando) or auction trading (e.g., eBay). Usage includes the actual purchase of goods as well as other activities (e.g., price comparisons, product searches).]	☐	☐
Nutzung von Online-Pornographie [Online pornography use] Dies umfasst die Nutzung von Angeboten im Internet mit pornographischen Inhalten (z.B. jede Art von explizit sexuellen Filmen, Videos oder Bildern, in denen der Genitalbereich dargestellt wird, mit der Absicht den/die Betrachter*in sexuell zu erregen), die Sie auf internetfähigen Endgeräten anschauen (z.B. Computer, Tablet, Smartphone, SmartTV). [This includes the use of offers on the Internet with pornographic content (e.g., any kind of explicitly sexual films, videos or images in which the genital area is depicted with the intention of sexually arousing the viewer) that you watch on Internet-enabled devices (e.g., computer, tablet, smartphone, smart TV).]	☐	☐
Nutzung von sozialen Netzwerken [social networks use] Dies umfasst die Nutzung verschiedener sozialer Netzwerke und Blogs (z.B. Facebook, Twitter, Instagram) als auch Messenger (z.B. WhatsApp). Die Nutzung kann sowohl aktive Tätigkeiten (z.B. Posten von Einträgen) als auch passive Tätigkeiten (z.B. Lesen von Einträgen) umfassen. [This includes the use of various social networks and blogs (e.g., Facebook, Twitter, Instagram) as well as messengers (e.g., WhatsApp). Use can include both active activities (e.g., posting entries) and passive activities (e.g., reading entries).]	☐	☐
Spielen von Online-Glücksspielen [Online gambling] Dies umfasst Online-Glücksspielaktivitäten wie Casino-Spiele (z.B. Poker, Black Jack, Roulette, Slot Machines), Online-Wetten (z.B. Sportwetten, Pferderennen) als auch Online-Lotterien und Bingo. [This includes online gambling activities such as casino games (e.g., poker, blackjack, roulette, slot machines), online betting (e.g., sports betting, horse racing) as well as online lotteries and bingo.]	☐	☐
Sonstiges [Other] Wenn Sie weitere Aktivitäten im Internet regelmäßig nutzen, die nicht von den bisher genannten Aktivitäten eingeschlossen werden, nennen Sie diese bitte: [If you regularly use other activities on the Internet that are not included by the activities mentioned so far, please name them:]	☐	☐

Instructions (German original [proposed English translation]):

Bitte lesen Sie die folgenden Fragen im Zusammenhang mit Aktivitäten im Internet (Gaming, Online-Shopping, Nutzung von Online-Pornographie, Nutzung von sozialen Netzwerken und Online-Glücksspielen) aufmerksam durch. Wählen Sie aus den ersten vier Antwortmöglichkeiten (linke Spalte) diejenige aus, die am besten beschreibt, wie häufig Sie die jeweiligen Erfahrungen in den letzten 12 Monaten gemacht haben. Bitte geben Sie zusätzlich an, wie stark die Erfahrungen in den letzten 12 Monaten waren (mit einer der vier Antwortmöglichkeiten in der rechten Spalte). Hierbei gibt es weder richtige noch falsche Antworten. Wenn Sie eine Erfahrung in den letzten 12 Monaten nie gemacht haben, geben Sie bei der Frage nach der empfundenen Stärke bitte “gar nicht stark” an. Bitte beurteilen Sie jede Erfahrung für alle aufgelisteten Aktivitäten.

[Please read carefully the following questions related to activities on the Internet (gaming, online shopping, use of online pornography, use of social networks and online gambling). From the first four answer choices (left column), select the one that best describes how often you have had each experience in the past 12 months. In addition, please indicate how strong the experience was in the past 12 months (using one of the four answer choices in the right column). There are neither correct nor incorrect answers. If you have never had an experience in the past 12 months, please indicate "not at all strong" when asked about perceived strength. Please rate each experience for all activities listed.]

Items (German original, for proposed English translation see Table 1 and Fig. 1):

Appendix B

Fig. B-1.

Factor loadings and residual covariances of the second-order factor models of the ACSID-11 for (A) gaming disorder, (B) online gambling disorder, (C) online buying-shopping disorder, (D) online pornography-use disorder, and (E) social-networks-use disorder. Notes. Model df = 40; gf = general factor; ic = impaired control; ip = increased priority; ce = continuation/ escalation; fi = functional impairment; Values represent standardized factor loadings, factor covariances, and residual covariances. Grey arrows represent non-significant estimates at p > 0.05.

Appendix C Results of the complementary bifactor models of (potential) specific Internet-use disorders measured by ACSID-11

Table C-1.

Fit indices for valid bifactor models of (potential) Internet-use disorders measured by ACSID-11

ACSID-11 model	CFI	TLI	SRMR	RMSEA	χ2/df
Frequency
Gaming	0.996	0.993	0.028	0.036	1.58
Online buying-shopping	0.997	0.995	0.019	0.031	1.90
Social-networks use	0.994	0.991	0.025	0.047	2.91
Online gambling	0.997	0.996	0.029	0.058	1.68
Intensity
Gaming	0.996	0.993	0.028	0.035	1.55
Online gambling	0.997	0.995	0.029	0.056	1.62

Notes. Model df = 35; Sample sizes varied for gaming (n = 440), online shopping (n = 944), social-networks use (n = 854), and online gambling (n = 200); ACSID-11 = Assessment of Criteria for Specific Internet-use Disorders, 11-items. Bifactor models for online buying-shopping (intensity), pornography use (frequency & intensity), and social-networks use (intensity) could not be identified.

Fig. C-1.

Factor loadings and residual covariances of the identified bifactor models of the ACSID-11 (frequency subscale) for (A) gaming disorder, (B) online gambling disorder, (C) online buying-shopping disorder, and (D) social-networks-use disorder. Notes. Model df = 35; Values represent standardized factor loadings, factor covariances, and residual covariances. Grey arrows represent non-significant estimates at p > 0.05.

Table C-2.

Ancillary bifactor measures on dimensionality and model-based reliability of ACSID-11.

	Gaming		Online gambling		Online buying/shopping	Social-networks use
Parameter	freq.	int.	freq.	int.	(frequency)	(frequency)
ω	0.967	0.976	0.985	0.987	0.979	0.972
ωH	0.935	0.930	0.967	0.968	0.951	0.936
ωHSIC	0.198	0.208	0.115	0.128	0.175	0.227
ωHSIP	0.118	0.221	0.002	0.003	0.106	0.082
ωHSCE	0.059	0.091	0.103	0.104	0.064	0.112
ECVGeneral	0.870	0.779	0.932	0.907	0.880	0.843
IECVIC1	0.923	0.922	0.915	0.976	0.909	0.835
IECVIC2	0.713	0.654	0.823	0.625	0.636	0.689
IECVIC3	0.693	0.717	0.888	0.955	0.863	0.728
IECVIP1	0.995	0.996	0.999	0.972	0.981	0.991
IECVIP2	0.711	0.971	0.964	0.995	0.669	0.893
IECVIP3	0.850	0.380	0.998	0.937	0.953	0.802
IECVCE1	0.937	0.875	0.935	0.930	0.942	0.958
IECVCE2	0.932	0.891	0.886	0.889	0.891	0.582
IECVCE3	0.936	0.932	0.853	0.855	0.959	0.996
IECVFI1	1.00	1.00	1.00	1.00	1.00	1.00
IECVMD1	1.00	1.00	1.00	1.00	1.00	1.00

Notes. Ancillary bifactor measures based on standardized model results (Rodriguez, Reise, & Haviland, 2016): ω = Omega, i.e., proportion of total score variance that can be attributed to all factors; ωH = Omega hierarchical, i.e., proportion of total score variance that can be attributed to the general factor, ωH > 0.80 indicates essential unidimensionality of the total score; ωHS = Omega hierarchical subscale, i.e., reliability of a subscale score after controlling for the variance explained by the general factor; ECV = Explained common variance, i.e., degree of essential unidimensionality with higher values indicating a strong general factor; IECV = Item explained common variance, i.e., proportion of item variance attributable to the general factor; IC = Impaired control; IP = Increased priority; CE = Continuation/escalation; FI = Functional impairment; MD = Marked distress.

Acknowledgements

The work on this article was carried out in the context of the Research Unit ACSID, FOR2974, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 411232260.