Internet Addiction and the Psychometric Properties of the Nine-item Internet Disorder Scale-Short Form: An Application of Rasch Analysis

Abstract

Background

The aim of the present study was to determine the prevalence of disordered internet use among adolescent university students and its association with various health complaints and behaviours, and most importantly to examine the psychometric properties of 9-item Internet Disorder Scale-Short Form (IDS9-SF) using factor analyses and Rasch analysis.

Methods

A total of 1988 university students aged 18 to 25 years were selected via a multi-stage stratified random sampling technique among university students in Istanbul, Turkey (September 2017 to February 2018). Data collected included socio-demographics, lifestyle and dietary habits, and the 9-item IDS9-SF. Statistical analysis included descriptive statistics, multivariate analyses, factor analyses, path analysis, and Rasch analysis.

Findings

Using confirmatory factor analysis (CFA), the study investigated the latent structure of the IDS9-SF instrument and results supported its reliability and validity. The prevalence of disordered internet use was 18.3% in the sample. There were significant differences between those who had disordered internet use and those who did not in gender, family income, school performance, number of bedrooms at home, and number of people living at home, as well as internet use duration. Using multivariate regression analysis, key predictors of disordered internet use included (among others): gender, body mass index (BMI), household income, number of people living at home, having a computer at home, internet facilities, duration of internet use, sleeping hours, frequency of eating fast food, watching television, headache, hurting eyes, tired eyes, and hearing problems. Rash analysis demonstrated that four of the nine items (2, 3, 6, and 7) were more difficult for individuals to endorse compared to other items.

Conclusion

Problems arising from excessive internet use were apparent among the study sample and the IDS9-SF is a valid and reliable measure for assessing disordered internet use among Turkish adolescent population.

Introduction

Despite the many potential benefits associated with using the internet, numerous problems such as exposure to inappropriate images and content, absence of privacy, and internet addiction among a small minority have been reported as a result of this increasing usage.1 Research into internet addiction began in the 1990s by Griffiths in the UK and Young in the US.2,3 Although internet addiction has been identified as a prevalent problem among a small minority of the population, it has never been classed as a formal disorder most likely because internet addiction is an umbrella term covering more specific online problematic behaviors such as online problematic gaming and problematic social media use.4 Many psychometric studies have been carried out and there are over 20 different assessment scales that have been developed to assess problematic internet use behavior.5 The theoretical frameworks underpinning many psychometric instruments are still controversial, and a study has evolved in a divergent manner with little or no consensus.6

In 2013, new criteria and definitions of various internet addictions began to develop as a consequence of Internet Gaming Disorder (IGD) being introduced to section 3 of the latest (fifth) edition of the Diagnostic and Statistical Manual of Mental Disorders-5^th Edition (DSM-5), leading to the development of more new instruments.7 Many of these new instruments also included core criteria of addiction such as those outlined by Griffiths (2005) and also used in the context of problematic internet use.8,9 For instance, Pontes and Griffiths10 developed the nine-item IGD Scale-Short Form (IGDS-SF) which was letter adapted to develop the 15-item Internet Disorder Scale (IDS-15)11 and the 9-item IDS-Short Form (IDS9-SF).12

Irrespective of whether disordered internet use is an addiction, research has consistently shown that excessive internet use can lead to psychological and behavioral problems among a small minority of individuals,13 including physical health issues (e.g., dry eyes, neck, back, and shoulder pain, regular headaches, numbness pain in the thumb, index, and middle fingers).14,15 Again, irrespective of whether it is a dependency, addiction, and/or disorder, there have been many studies reporting negative impacts of excessive internet use among adolescents including poor dietary habits,16 physical inactivity, lack of adequate sleep,15,17 increased depression and loneliness, and social anxiety.15,18

The present study aimed to determine the prevalence of disordered internet use among adolescent university students, its association with various health complaints and behaviors, and most importantly to examine the psychometric properties of IDS9-SF using Rasch analysis.

Methods

Turkish adolescent university students aged 18 to 25 years who studied in five Istanbul government and private trust universities completed the current cross-sectional survey. The Institutional Review Board (IRB) (Istanbul Medipol University) gave ethical clearance for the study. A multi-stage stratified random sampling method was performed from September 2017 to February 2018. A total of 2500 students were approached and 1988 (79.5%) students completed the measures. Content validity, face validity, and reliability of the whole questionnaire obtained high kappa = 0.86.

In addition to sociodemographic information (age, gender, income, academic performance), the survey included the IDS9-SF as well.12 The IDS9-SF is a unidimensional standardized psychometric scale that assesses internet use disorder (IUD). The IDS9-SF uses 5-point Likert scales and total scores can range from 9 to 45, with higher scores being indicative of a higher degree of IUD.

Statistical analysis and Rasch measurement: The Rasch method is used to examine a participant’s response to an item that is a function of the difference between an individual’s ability and the characteristics of the item. Rasch measurement determines the relationship between the difficulty of an item and the ability of an individual. It is expected that there will be a higher probability in answering easier items correctly and a lower probability in answering more difficult items incorrectly.19,20 According to the model, the probability of an individual (n) responding in category x to item i, is given by:

$$P_{xni}=\frac{exp{\sum }_{j=0}^x\left[{\beta }_n-({\sigma }_i+{\tau }_j)\right]}{{\sum }_{k=0}^{m}exp{\sum }_{j=0}^x\left[{\beta }_n-({\sigma }_i+{\tau }_j)\right]}$$

Where τ₀ = 0, so that

$$exp{\sum }_{j=0}^x\left[{\beta }_n-({\sigma }_i+{\tau }_j)\right]=1$$

βn is the individual’s position on the variable, δi is the scale value (difficulty to endorse) estimated for each item i, and τ₁ τ₂ … τ₃ are the m response thresholds estimated for the m + 1 rating categories.19,20

The statistical analysis included student’s t-test, chi-square test, and Fisher’s exact test (two-tailed). The Rasch Rating Scale Model (RSM) was used for the analyses of the data collected. Multiple regression analysis using the forward inclusion and backward deletion method was performed to determine the importance of risk factors for internet addiction. Statistical significance was assessed at the P < 0.01 level.

Results

Table 1 shows the socio-demographic characteristics of internet disordered participants compared to those not disordered. The prevalence of IUD in the present Turkish sample was 18.3%. There were significant differences between age, gender, family income, and rank in school exams (P < 0.001). Predictably, significantly more daily hours were spent on the internet among internet disordered participants compared to non-internet disordered participants (P = 0.001). Table 2 shows the differences between internet disordered and non-internet disordered participants with respect to diet and co-morbid factors. Those with internet disorder were significantly more likely to have headache, blurred vision, double vision, hurting eyes, and hearing problems, and to eat fast food frequently (P < 0.001). Significantly, fewer participants with internet disorder reported engaging in vigorous and moderate physical activity compared to non-internet disordered participants (P < 0.001).

Table 1.

Socio-demographic characteristics of participants (n = 1988) with and without internet disorder

Variables	Internet disorder (n = 364)	Non-internet disorder (n = 1624)	P
	n (%)	n (%)
Gender
Male	201 (55.2)	686 (42.2)	< 0.001
Female	163 (44.8)	938 (57.8)
Age group (year)
≤ 20	122 (33.5)	623 (38.4)	< 0.001
> 20	242 (66.5)	1001 (61.6)
Family income ($)
< 1000	139 (38.2)	875 (53.9)	< 0.001
1000-1500	106 (29.1)	478 (29.4)
> 1500	119 (32.7)	271 (16.7)
Academic performance
Very good	98 (27.0)	375 (23.1)	< 0.001
Good	148 (40.6)	763 (47.0)
Average	78 (21.4)	420 (25.9)
Poor	40 (11.0)	66 (4.0)
	Mean ± SD	Mean ± SD
Number of bedrooms at home	3.42 ± 0.90	3.57 ± 0.99	< 0.001
Number of people living at home	5.52 ± 2.01	4.80 ± 1.85	< 0.001
Number of sleeping hours	6.05 ± 1.09	6.80 ± 1.31	< 0.001
Hours of internet use/day	5.78 ± 2.67	5.58 ± 1.98	< 0.001

SD: Standard deviation

Table 2.

The characteristics of dietary and co-morbid factors (n = 1988)

Variables	Internet disorder (n = 364)	Non-internet disorder (n = 1624)	P*
Medical co-morbid factors**	n (%)	n (%)
Headache	208 (10.5)	1020 (51.3)	0.044
Blurred vision	95 (4.8)	578 (29.1)	0.001
Double vision	70 (3.5)	162 (8.1)	< 0.001
Hurting eyes	96 (4.8)	624 (31.4)	< 0.001
Tired eyes	101 (5.1)	379 (19.1)	0.076
Dizziness	138 (6.9)	582 (29.3)	0.457
Any hearing problem	107 (5.4)	272 (13.7)	< 0.001
Physical activity
Vigorous	158 (7.9)	840 (42.3)	0.004
Moderate	181 (9.1)	1035 (52.1)	< 0.001
Frequency of eating fast food*
Daily	97 (26.6)	317 (19.5)
Weekly	120 (33.0)	627 (38.6)	< 0.001
Monthly	88 (24.2)	274 (16.9)
Occasionally	59 (16.2)	406 (25.0)

^*Two-sided P-values based on student’s t-test;

^**Not adding to 100%

Table 3 shows the multiple linear regression analysis to determine the potential predictors as risk factors for internet disorder. This analysis demonstrated that gender, body mass index (BMI), household income, number of people living at home, having a computer at home, having internet facilities, duration of internet use, sleeping hours, frequency of eating fast food, watching television, headache, hurting eyes, tired eyes, and hearing problems were significantly associated with (and key predictors of) internet disorder.

Table 3.

Multivariable stepwise regression analysis predictors for determinants of internet disorder affect (n = 1988)

Independent variables	B	SE	β	t	P
Constant	41.553	3.814	-	10.895	< 0.001
Gender	-1.615	0.596	-0.050	-2.709	0.007
BMI	0.438	0.054	0.163	8.106	< 0.001
Household income	2.199	0.333	0.140	6.604	< 0.001
Number of people living at home	1.057	0.158	0.128	6.674	< 0.001
Computer at home	3.208	0.945	0.074	3.395	0.001
Internet facilities	-4.346	1.071	-0.087	-4.057	< 0.001
Internet use in hours	0.255	0.118	0.041	2.165	0.031
Sleeping hours	-2.307	0.245	-0.189	-9.413	< 0.001
Frequency of eating fast food	-1.053	0.283	-0.068	-3.723	< 0.001
Frequency of watching television	3.641	0.333	0.216	10.945	< 0.001
Headache	1.579	0.636	0.048	2.485	0.013
Hurting eyes	2.813	0.674	0.085	4.172	< 0.001
Tired eyes	-3.012	0.716	-0.082	-4.208	< 0.001
Hearing problems	-3.542	0.742	-0.088	-4.770	< 0.001

BMI: Body mass index; SE: Standard error

In table 4, the individual and item reliability indexes were calculated as 0.28 and 0.95, respectively, by Rasch analysis. Reliability ranged from 0 to 1.0 (where a coefficient of 0 means no reliability while 1.0 means perfect reliability).

Table 4.

Fit statistics for Internet Disorder Scale-Short Form (IDS9-SF) (n = 1988, # item of scale = 9)

Person	1988 input		1988 measured		Infit		Outfit
	Total	Count	Measure	RealSe	IMNSQ	ZStd	OMNSQ	ZStd
Mean	24.1	9.0	-0.32	0.32	1.00	0.0	1.00	0.00
SD	4.3	0.0	0.38	0.04	0.43	1.2	0.43	1.20
Real RMSE 0.32 True SD			0.20	Separation	0.62	Person reliability		0.28
Item	9 input		9 measured		Infit		Outfit
	Total	Count	Measure	RealSe	IMNSQ	ZStd	OMNSQ	ZStd
Mean	5313.4	1988	0.00	0.02	1.00	-.1	1.00	0.00
P.Sd	229.5	0.0	0.09	0.00	0.13	5.1	0.13	4.90
Real RMSE 0.02 True SD			0.09	Separation	4.32	Item reliability		0.95

RMSE: Root-mean-square error; IMNSQ: Infit mean square; ZSTD: Standardized; OMNSQ: Outfit mean square

Moreover, the individual and item separation indexes were 0.62 and 4.32, respectively. Table 4 demonstrates that the IDS9-SF has acceptable psychometric characteristics because the model fit mean-square (MNSQ) values range from 0.79 to 1.28, outfit MNSQ is 1.00, and infit MNSQ is 1.00. The values of infit and outfit MNSQs are in the acceptable range of 0.5-1.5 for these statistics.19

Winsteps 4.0.1 was used to conduct the Rasch analysis for the present study. As seen in figure 1, in the left-hand column, each “#” symbol represents 15 people and each “.” represents 1 to 14 people. In the right-hand column, each entry represents a scale item. The person-item map compared the range and position of the item measure distribution (left-hand side of figure 1) to the range and position of the person measure distribution (right-hand side of figure 1). The individuals at the top of figure 1 had the highest scores, while the items at the top of figure 1 were the most difficult. Individuals at the bottom of figure 1 had the lowest scores, and the items at the bottom of figure 1 were the easiest. Several items are situated high above the mean value (0.0 logit) with high logit measures. This means that these items such as item 2 (‘feel anxiety when trying to reduce and/or stop using internet’), item 3 (‘need to spend more time to achieve satisfaction’), item 6 (‘continue to be online when it leads to a problem’), and item 7 (‘deceive people about the time of being online’) are difficult for individuals to endorse.

Figure 1.

Person-item map for Internet Disorder Scale-Short Form (IDS9-SF) (n = 1988)

The vertical line between the two columns indicates the scale for parameter estimates measured in logits (i.e., log-odds units). Along the vertical line, M indicates the mean, S indicates one standard deviation (SD) above or below the mean, and T indicates two SDs above or below the mean.

Figure 2 indicates the path analysis of IDS9-SF in determining the significance of the hypothesized causal connections between sets of variables. In the present study, each rectangle represents a variable. Internet addiction and gender are endogenous variables and their variances are explained by other variables in the model. The other variables are extraneous and indicated by the arrow from ε. The path coefficients are the β weights from the multiple regression analyses. Analysis indicated that internet addiction was directly affected by gender (β = -0.07), having tired eyes (β = -0.11), sleep duration (β = 0.35), and internet use duration (β = -0.05). Moreover, sleep duration and having tired eyes had indirect effects via gender upon internet addiction.

Figure 2.

Summary of the path analysis of the Internet Disorder Scale–Short Form (IDS9-SF)

Discussion

The prevalence of IUD in the present study’s Turkish sample (18.3%) was comparable to 26.3% of United States (US) university students.21 However, it must be noted that comparing these (or any other) studies is difficult because of differences in study populations, assessment tools applied, and differences in social and cultural contexts. For example, a meta-analysis by Cheng and Li22 comprising 80 studies (n = 89281 participants) reported an estimated global prevalence rate of internet addiction of 6.0%. The highest prevalence was in the Middle East (including Turkey, Iran, Israel, Lebanon) with 10.9%, followed by 8.0% for North America (US), 7.1% for Asia (China, Hong Kong, India, South Korea, Taiwan), 6.1% for South and East Europe (Bulgaria, Cyprus, Czech Republic, Greece, Hungary, Italy, Poland, Romania, Serbia, Slovenia, Spain), 4.3% for Oceania (Australia), and the lowest was in Northern and Western Europe with 2.6% (Austria, Estonia, France, Germany, Ireland, Norway, Sweden, United Kingdom).22 In this meta-analytic study, internet addiction prevalence was higher for nations with greater traffic time consumption, pollution, and dissatisfaction with life in general and was found to be inversely associated with the quality of life. In a recent study, problematic internet use was studied in seven European countries and prevalence estimates ranged between 14.3% and 54.9% and many cross-cultural and gender differences have been observed in terms of relationship with psychopathology and online activities.23 The students use of the internet is more than the general population; surveying students also led to an inflated prevalence rate relative to the general population in both general and problematic use.24 The IDS9-SF was also developed using online users12 and the present study used the scale in a student sample. For that reason, the prevalence of IUD was likely to be much higher compared to European countries more generally.23

The IDS9-SF was developed by adapting from IGDS-SF,10 that uses common criteria in which both reliability and validity can be better ascertained across studies.25,26 The psychometric properties of the IDS9-SF in the present study were comparable to original scale development study of Pontes and Griffiths.12 The Cronbach’s alpha reliability coefficient was high (α = 0.85) and the factor determinacy was 0.89, which is well above the desired threshold of 0.80, supporting reliability and validity of the instrument. Additionally, Rash analysis was carried out in order to detect the level of difficulty of each question and demonstrated that item 2 (‘feel anxiety when trying to reduce and or stop using internet’), item 3 (‘need to spend more time to achieve satisfaction’), item 6 (‘continue to be online when it leads to a problem’), and item 7 (‘deceive people about the time of being online’) were more difficult for individuals to endorse compared to other items.

Our results confirm findings from previous studies showing that disordered internet use can have negative impacts on individuals’ abilities,27,28 physical inactivity,16 adequate sleep,15,17 and irregular dietary habits.14,16 The relationship between excessive internet use and sleep problems has become well established in the literature. Furthermore, disordered internet use can also play a contributory role in daytime sleepiness, sleeping disorders, sleep deprivation, and fatigue.15,29,30

The present study has several limitations. First, although it examined the associations between disordered internet use and many other behaviors and health issues, the data were cross-sectional and therefore, no conclusions can be made concerning issues of causality. Second, there may be social desirability and memory recall biases concerning the duration of internet use in self-report data. Thirdly, factors relating to family members, the social environment, and disordered internet use were not assessed in the present study. However, despite these limitations, the present study confirmed that disordered internet use was associated with many detrimental health issues using a relatively large-scale sample.

Conclusion

The results of the present study supported the internal consistency of the IDS9-SF which is based on adapted criteria for IGD in the DSM-5. Analysis demonstrated good Cronbach’s alpha and composite reliability. Therefore, the IDS9-SF appears to be a valid and reliable measure for assessing IUD among Turkish population. However, Rasch analysis demonstrated that some items in the IDS9-SF were more difficult for individuals to endorse than others.