Assessing the psychometric properties of the Internet Addiction Test (IAT) in US college students

Abstract

Internet addiction is a growing concern; however, both a clear understanding of the mechanisms driving problematic behaviors and a gold standard instrument for assessing symptoms are lacking. The purpose of this study was to perform a psychometric analysis of the most widely used screening instrument, the Young Internet Addiction Test (IAT), using a sample of US college students. 215 (70% response rate) undergraduate college students were recruited from two US universities to complete an online version of the IAT. Using exploratory factor analysis, two factors were extracted, “dependent use” and “excessive use” which together explained 91% of the total variance. Participants who scored into the problematic user category of the IAT scored items on the dependent use scale an average of 0.8±1.5 points higher and items on the excessive use scale an average of 1.4±1.5 points higher than participants who scored in the average user category. Results suggest that Internet addiction symptoms may cluster into two separate components, dependent and excessive use. Further, the IAT appears to be a valid instrument for assessing Internet addiction in US college students.

1. INTRODUCTION

The pervasive and influential presence of the Internet in today’s society has raised concern over the existence of an Internet addiction disorder. While this view may be controversial, the upcoming inclusion of Internet addiction in the DSM-V as a disorder in need of further study compels further investigation (Holden, 2010). At its core, Internet addiction is characterized by frequent, uncontrolled and harmful use of the Internet. Symptoms of the disorder most frequently observed in clinical settings include preoccupation, withdrawal, loss of control and functional impairment (Tao et al., 2010). Enhancing our understanding of how these symptoms define and drive Internet addiction will help to guide this emerging field of study.

Developing a standardized instrument for assessing Internet addiction symptoms in clinical and research settings is an important step. Of the available instruments, thus far the most widely used has been the Young Internet Addiction Test (IAT). The IAT was adapted from the DSM-IV pathological gambling criteria, and was first published in 1998 (Young, 1998). The instrument covers a variety of Internet use behaviors and common addiction symptoms, with the notable exception of tolerance. Initial investigation into the validity of the IAT has shown strong internal consistency (α = .90–.93) and good test-retest reliability (r = 0.85) (Widyanto and McMurran, 2004; Yang et al., 2005; Chang and Law, 2008; Khazaal et al., 2008; Milani et al., 2009; Korkeila et al., 2010).

Further assessment of the IAT is warranted for both theoretical and psychometric reasons. First, the IAT was designed as a uni-dimensional instrument with each item contributing equally to the overall score. However, prior analyses have suggested an ambiguous factor structure; table 1 provides a summary of these findings. Second, to date, the psychometric properties of the IAT have not been assessed in a US population. As cultural differences may affect the presentation of problematic Internet use behaviors, results obtained in international settings may not be relevant to US populations.

Table 1.

Previous factor analysis studies of the Internet Addiction Test (IAT)

	Widyanto and McMurran, 2004	Khazaal, 2008	Chang and Law, 2008	Korkeila 2010	Widyanto et al., 2010
IAT version	English	French	Bilingual- English and Chinese	Finnish	English
Sample	86 adults recruited through the internet in England	246 medical school students or community volunteers in France	410 university undergraduates in China	1,825 medical school or vocation college students in Finland	255 internet users (age unspecified) in England
Factors proposed	Six factors Salience Excessive use Neglect work Anticipation Lack of control Neglect social life	Single factor	Three factors Withdrawal and social problems Time management and performance Reality substitute	Support for both a single and two factor model Salient Use Loss of control	Three factors Emotional/psychological conflict Time management issues Mood modification
Cronbach’s alphas	α = 0.82, 0.77, 0.75, 0.61, 0.76, 0.54	α = 0.93	--	Single factor: α = 0.92 Two factors: α = 0.91, 0.81	--
Eigenvalues	7.17, 1.8, 1.3, 1.2, 1.11, 1.04	--	7.8, 1.3, 1.1	Single factor: 14.0 Two factors: 15.0, 1.5	8.53, 1.59, 1.12
Total variance explained	68.11%	45%	57.1%	--	56.3%

-- Data not reported

This work seeks to build on the current literature by assessing the construct validity of the IAT in an at-risk, US population. Behavioral addictions often have their onset during adolescence or young adulthood (Grant et al., 2010). College students, the target population for this analysis, are particularly vulnerable to Internet addiction (Kandell, 1998). College students report near universal rates of computer ownership, Internet accessibility and daily use; most spend at least two hours online daily (Cotten and Jelenewicz, 2006; Fortson et al., 2007; Jones et al., 2009). The prevalence of Internet addiction is often reported to be higher among college students than other adolescent populations (Huang et al., 2009; Ko et al., 2009; Tsai et al., 2009; Fu et al., 2010). Among US college students, the prevalence of Internet addiction has been estimated to be between 8–25% (Scherer, 1997; Lavin et al., 1999; Morahan-Martin and Schumacher, 2000; Anderson, 2001; Lavin et al., 2004; Iacovelli and Valenti, 2009), placing it on a comparable scale with substance use disorders and pathological gambling (Shaffer and Hall, 1996; Slutske, 2005). Given the high risk for problematic behavior within this group, the development of a standardized instrument for assessment is greatly needed.

By conducting a factor analysis of the IAT using a college student sample, we aim to (1) explore the behavioral components underlying addictive use of the Internet, and (2) assess the validity of the IAT as a screening instrument within this at-risk population.

2. METHODS

This study was conducted between September 1, 2009 and September 15, 2010 and received IRB approval from both the University of Wisconsin and the University of Washington.

2.1. Setting and subjects

Participants for this study were undergraduate students aged 18–20 enrolled at two large public universities in the US. In order to achieve a target population of regular Internet users, we used the social networking site Facebook to identify potential participants. Facebook (www.Facebook.com) was chosen because over 90% of students report maintaining a profile through the site, making it is the most popular social networking site among our target college populations (Pempek et al., 2009; Ross et al., 2009).

Eligible participants were identified by using the Facebook search engine to search for profiles belonging to freshmen, sophomore and junior undergraduate students at our two selected universities. Outside of these identified search criteria, Facebook search results are returned in a random order with respect to other demographic and identifying characteristics. All resulting profiles were assessed sequentially for eligibility until the target sample size was reached. From an initial search result of 3000 profiles, the majority of profiles were excluded because their profile owners were incorrectly listed and were not undergraduates (14.9%), did not meet the age criteria (10.4%) or did not display their age (1.6%), would not be reachable for follow-up as no contact information (phone number or email) was listed on the profile or in the university directory (10.1%), or due to privacy settings (54.3%). The first 307 eligible profiles owners were invited to participate in the study, with the goal of recruiting a minimum of 200 participants.

2.2 Data Collection

Eligible participants were contacted over the phone. The study was explained and permission was requested to send an email that contained further information about the study. If the participant consented to receive the email, an email was sent to the participant’s university email account that provided detailed information about the study as well as a link to the online survey. The survey was administered via a Catalyst WebQ online survey engine; respondents were provided a $15 iTunes gift card as compensation.

Survey components included an informed consent statement, demographic questions and the IAT. The IAT contains twenty categories assessing a respondent’s productivity at work, school, or home (3 questions), social behaviors (3 questions), emotional connection to and response from using the internet (7 questions), and general patterns of Internet use (7 questions). Participants responded to the 20 IAT items on a 6-point Likert measure (“does not apply” to “always”), which produced an overall score between 0 and 100. IAT Author guidelines recommend that participants scoring less than 40 be categorized as average users, those scoring 40 or above be categorized as problematic internet users, and those scoring 70 or above be categorized as severely problematic users (Young, 1998).

2.3. Statistical Analysis

All statistical analyses were conducted using STATA version 11.0 (Statacorp, College Station, TX). Demographic characteristics and individual IAT item scores were summarized using descriptive statistics. Overall IAT scores were dichotomized into a binary variable coded as “average user” (0 to 39) and “problematic user” (40 to 100).

Exploratory factor analysis was conducted to assess the construct validity of the IAT. The number of factors to be extracted was determined through visual examination of a scree plot in combination with the conventional cut-off of eigenvalues greater than one. In order to distinguish independent underlying constructs, varimax rotation was employed to determine factor loadings. Items were assigned to the factor that produced the highest factor loading. The internal consistency of each factor was confirmed by calculating Cronbach’s alphas. We then assessed the linear correlation coefficient between the two factors to confirm the degree of independence between the factors identified.

Factor subscale scores were calculated as the mean item scores for all items loading to each factor. Because the number of items loading onto each factor differed, this approach was taken to allow a more direct comparison between subscale scores. The relationships between the subscale scores for each factor and the binary “problem user” variable were evaluated using t-tests. Finally, we assessed whether the effect of problem use differed between the sub-scales using mixed effects linear regression in which subscale mean item score was the independent variable and subscale type, problem user, and their interaction were the independent covariates. Each student contributed two observations: one for each factor subscale; thus, the model included a random effect for student to account for this correlation between observations on the same student.

3. RESULTS

3.1. Participant Demographics

A total of 224 participants responded to the survey (73% response rate). 215 surveys were complete and were included in the analysis. Participants were on average 18.8 years old, 54% were female and 68% were Caucasian. Approximately half of participants were from each university. Table 2 further describes demographic characteristics of the sample.

Table 2.

Demographics for 215 study participants

	Study Sample n (%)
Gender
Males	101 (48.0)
Females	114 (52.0)
Ethnicity
Caucasian/White	150 (69.8)
Asian/Asian American	38 (17.7)
Native American/Alaskan Native	1 (0.5)
Hispanic/Latino	5 (2.3)
Native Hawaiian/Pacific Islander	2 (0.9)
African American	0 (0.0)
Other/unspecified	19 (8.8)
Age
18	76 (35.3)
19	110 (51.2)
20	29 (13.5)

3.2. IAT Scoring

Overall, the mean IAT score was 28.4±10.3; scores did not differ significantly by age, gender, race or university. The majority of participants (n=190, 88%) were categorized as “average users” (IAT overall score < 40). The remaining participants (n=25, 12%) were categorized as “problematic users” (IAT overall score ≥ 40). Table 3 provides descriptive statistics for individual item scores for the IAT. Mean item scores ranged from 0.7±0.5 to 3.0±1.0 among average users, and 1.1±0.9 to 4.2±1.0 among problematic users.

Table 3.

Descriptive statistics for Internet Addiction Test (IAT) item scores

Item	How often …	Average Users (n=190)		Problematic Users (n=25)
		Mean (SD)	Range	Mean (SD)	Range
Q1	do you find that you stay online longer than you intended?	3.0 (1.0)	1–5	4.1 (1.1)	1–5
Q2	do you neglect household chores to spend more time online?	1.9 (0.9)	0–5	3.4 (0.9)	2–5
Q3	do you prefer the excitement of the Internet to intimacy with your partner?	0.7 (0.5)	0–3	1.1 (0.9)	0–3
Q4	do you form new relationships with fellow online users?	1.1 (0.7)	0–4	1.8 (1.0)	1–5
Q5	do others in your life complain to you about the amount of time you spend online?	0.9 (0.5)	0–4	2.0 (0.9)	1–4
Q6	do your grades or schoolwork suffer because of the amount of time you spend online?	1.3 (0.7)	0–4	2.7 (1.0)	1–5
Q7	do you check your email before something else that you need to do?	2.7 (1.4)	0–5	4.2 (1.0)	2–5
Q8	does your job performance or productivity suffer because of the Internet?	1.1 (0.7)	0–4	2.2 (1.3)	0–5
Q9	do you become defensive or secretive when anyone asks you what you do online?	0.9 (0.5)	0–3	1.3 (0.6)	0–3
Q10	do you block out disturbing thoughts about your life with soothing thoughts of the Internet?	0.8 (0.5)	0–2	1.7 (1.1)	0–5
Q11	do you find that you find yourself anticipating when you will go online again?	1.2 (0.7)	0–3	2.3 (1.0)	1–4
Q12	do you fear that life without the Internet would be boring, empty or joyless?	1.1 (0.8)	0–4	2.2 (1.0)	0–5
Q13	do you snap, yell or act annoyed if someone bothers you while you are online?	0.8 (0.5)	0–2	1.3 (0.8)	0–3
Q14	do you lose sleep due to late night log-ins?	1.3 (0.8)	0–4	2.8 (0.9)	1–4
Q15	do you feel preoccupied with the Internet when offline, or fantasize about being online?	0.8 (0.5)	0–4	1.9 (1.0)	0–4
Q16	do you find yourself saying “just a few more minutes” when online?	1.8 (1.0)	0–5	3.7 (1.0)	2–5
Q17	do you try to cut down the amount of time you spend online?	1.8 (1.1)	0–5	3.3 (1.1)	1–5
Q18	do you try to hide how long you’ve been online?	0.9 (0.5)	0–2	2.1 (1.2)	1–5
Q19	do you choose to spend more time online over going out with others?	0.9 (0.5)	0–2	1.8 (1.0)	0–5
Q20	do you feel depressed, moody or nervous when you are offline, which goes away when you are back online?	0.8 (0.5)	0–2	1.4 (1.0)	0–4

Items scored on a 6-point frequency scale (0=does not apply, 1=rarely, 2=occasionally, 3=often, 4=frequently, 5=always)

3.3. Exploratory Factor Analysis

Two interpretable factors were generated for the IAT; together they explained 91% of the total variance (Table 4). All items loaded at .40 or above; there were no cross-loadings. Cronbach’s alphas for the factors were found to be 0.91 and 0.83, respectively. There was a moderate linear correlation between the two factors (r=0.57).

Table 4.

Factor loadings from varimax rotation

	Item Abbreviation	Factor 1: Dependent Use	Factor 2: Excessive Use
Q20	Feels depressed offline, which goes away once back online	0.85
Q13	Snaps, yells or acts annoyed if when bothered while online	0.83
Q15	Feels preoccupied with the Internet when offline	0.83
Q19	Chooses online time over time with others	0.75
Q10	Blocks out disturbing thoughts by thinking of the Internet	0.70
Q9	Defensive or secretive when asked about online activities	0.63
Q11	Anticipates going online again	0.59
Q5	Others in life complain about amount of time spent online	0.58
Q18	Tries to hide time online	0.54
Q12	Fears life without the Internet would be empty or joyless	0.53
Q3	Prefers excitement of the Internet to intimacy with partner	0.44
Q4	Forms new relationships with fellow online users	0.40
Q16	Says “just a few more minutes” when online		0.75
Q1	Stays online longer than intended		0.68
Q17	Tries to cut down time spent online		0.62
Q6	Grades or schoolwork suffer because of time online		0.61
Q14	Loses sleep due to late night log-ins		0.59
Q2	Neglect household chores to spend more time online		0.56
Q7	Checks email before something else that needs to be done		0.49
Q8	Job performance suffers because of the Internet		0.44
Cronbach’s alpha		0.91	0.83
Eigenvalue		7.6	1.8
Variance explained		73.4%	17.3%

Factor 1 accounted for 73% of the variance, and grouped twelve questions. Questions loading most strongly on this factor included “feeling depressed, moody or nervous when offline, which goes away once back online,” “snapping, yelling, or acting annoyed if bothered when online,” “feeling preoccupied with the internet when offline,” and “choosing to spend time online over going out with others.” We termed this factor “dependent use.”

Factor 2 included the remaining eight questions and accounted for 17% of the variance. Questions loading most strongly on this factor included “staying online longer than intended,” “saying ‘just a few more minutes’ when online,” “trying to cut down on online time” and “grades or schoolwork suffering from time spent online.” Thus, we described factor two as “excessive use.”

Additional analysis using a three-factor model found that the third factor drew items evenly from both subscales, had an eigenvalue of 0.6 and explained an additional 6% of the variance. However, the two-factor model contained more robust Cronbach’s alphas and factor loadings, as well as greater theoretical consistency. Further, the loading of questions within the three-factor model was not consistent with any previously described three-factor solutions to the IAT; thus, the two-factor model was determined to be the best fit for this data.

3.4. Factor subscale scores

As a group, problematic users scored higher on both the “dependent use” (1.7±1.3 versus 0.9±0.4, P<0.0001) and “excessive use” (3.3±1.5 versus 1.9±0.6, P<0.0001) factors as compared to average users. The additional change in the “excessive use” subscale between average and problematic users was also significantly higher (Diff=0.6±1.5, P<0.001). Thus, while those who were considered problem users were on average more likely to report more frequent dependent use behaviors, they were even more likely to report more frequent excessive use behaviors.

4. DISCUSSION

Using exploratory factor analysis, we identified a two-factor model for the IAT; the factors, “dependent use” and “excessive use,” together accounted for 91% of the total variance. The high degree of variance explained, strong inter-item correlation and theoretical consistency of our model suggests that the IAT is a valid instrument for assessing Internet addiction in adolescent to young adult college students.

Previous analyses have found between one- and six-factor solutions for the IAT. In spite of these differences, factor arrangements from prior analyses show striking resemblances to our results (Table 5). The items clustered within the first factor of the two-factor solution by Korkeila et al. (2010) and the first and third factors in the three-factor solutions by both Chang and Law (2010) and Widyanto et al. (2010), show broad overlap with our first factor. Similarly, our second factor, “excessive use,” is consistent with the second factors of all three previous analyses. Although there is variation in factor loadings for a small subset of questions between the models, we believe these discrepancies are most likely due to differences inherent to the samples used for analysis such as age or differences in cultural norms surrounding Internet use, rather than differing representations of the overall construct.

Table 5.

Comparison of current factor solution with previous analyses

	Item Abbreviation	Previous Factor Solutions
		20041	20082	20103	20104
Factor 1: Dependent use	Prefers excitement of the Internet to intimacy with partner	6	1	2	1
	Forms new relationships with fellow online users	6	1	1	3
	Others in life complain about amount of time spent online	5	1	1	1
	Defensive or secretive when asked about online activities	3	1	1	1
	Blocks out disturbing thoughts by thinking of the Internet	1	3	1	1
	Anticipates going online again	4	--	1	1
	Fears life without the Internet would be empty or joyless	1	3	1	3
	Snaps, yells or acts annoyed if when bothered while online	1	1	1	3
	Feels preoccupied with the Internet when offline	1	1	1	3
	Tries to hide time online	2	1	1	1
	Chooses online time over time with others	1	1	1	1
	Feels depressed offline, which goes away once back online	2	1	1	3
Factor 2: Excessive use	Stays online longer than intended	2	2	1	2
	Neglect household chores to spend more time online	2	2	2	2
	Grades or schoolwork suffer because of time online	3	2	1	2
	Checks email before something else that needs to be done	4	--	2	2
	Job performance suffers because of the Internet	3	2	1	1,2†
	Loses sleep due to late night log-ins	2	3	1	3
	Says “just a few more minutes” when online	5	2	2	2
	Tries to cut down time spent online	5	2	2	1

-- Item dropped

^†Item loaded equally on multiple factors

¹Widyanto and McMurran, 2004

²Chang and Law, 2008

³Korkeila, 2010

⁴Widayanto et al., 2010

Thus, our results suggest that Internet addiction symptoms, as measured by the IAT, cluster into two distinct constructs which we have named, “dependent use” and “excessive use.” The first component, “dependent use,” encompassed the majority of previously established addiction symptoms, such as preoccupation and withdrawal, along with social impairment. The second component, “excessive use” grouped other forms of functional impairment with symptoms of overuse and loss of control. While the intrinsic nature of the two factors suggests that some degree of connection is to be expected, their moderate linear correlation supports the representation of these constructs as unique, distinguishable components.

Our findings also suggest several areas for refinement of the IAT. First, several IAT items produced lower factor loadings. Cultural and technological changes in Internet use since the development of the IAT, and characteristics specific to this population, may account for why these items are less precise in assessing Internet addiction symptoms. In particular, while the item, “do you form new relationships with fellow online users,” (factor loading 0.40) may have represented a problematic behavior in the earlier context of this instrument’s development, the recent rise of social networking sites, blogs, chat rooms and other internet social utilities may have normalized this behavior. Similarly, given the substantial integration of e-mail as a communication utility and increases in Internet availability through wireless connections and mobile devices, “do you check your e-mail before something else that you need to do” (factor loading 0.49) may no longer represent an abnormal behavior. Additionally, the items, “do you prefer the internet to intimacy with your partner” (factor loading 0.44) and “does your job performance suffer because of the internet,” (factor loading 0.44) may refer to lifestyle characteristics uncommon among young, full-time students, and thus may not be appropriate for assessing a disorder in this population. Although these items did produce acceptable factor loadings, re-working or removing them may enhance the performance of the IAT in this population.

A second area of refinement may be the instrument’s scoring system. Currently, in calculating the overall IAT score, equal weight is given to each item. However, several items such as “do you snap, yell or act annoyed when bothered online” and “do you feel depressed, moody, or nervous when offline” showed a reduced range of responses and lower mean scores, despite strong factor loadings (0.83 and 0.85, respectively). Thus, while these items appear to identifying key symptoms of Internet addiction, their contribution to the overall score may be diminished. Applying a weighted scoring scheme to the IAT may be useful in improving the instruments precision. Further, although IAT scores could range between 0 and 100, the limited range of overall scores suggests that narrowing the response scale may enhance the feasibility of categorizing non-problematic, problematic and severely problematic behavior.

There are several potential limitations of our study. First, because we identified potential participants through searching a social networking site database, there is a possible sampling bias in our study. However, the near-universal use of this site within our target population, our strong response rate and the comparable demographic breakdown between our sample and reference populations suggest that our methods are reliable. Second, because we focused on college students, generalizing results to other adolescent or young adult populations may not be warranted. Given that college students are a key population in which intense Internet use is common and potentially consequential, this was our population of choice for this analysis. Our findings should be interpreted with some caution, as results may have been affected by characteristics specific to a campus environment. In particular, participants’ reporting of excessive use symptoms may have been augmented in the context of high standards of Internet use found among students. If this were the case, it would suggest that when developing approaches for screening, diagnosing and treating Internet addiction, careful consideration should be taken to integrate clinical guidelines with the technology use norms relevant to the sample of interest.

In spite of these limitations, there are several ways our findings can inform future research on both the IAT and Internet addiction. First, while these findings support the IAT as a valid assessment of Internet addiction in this population, confirmatory analysis of our two-factor solution is necessary to corroborate these results. The effect of removing or reworking low-loading items on overall scale performance should also be assessed. Second, the concurrent and predictive validity of the overall score cut-offs, item-weighting scheme and sub-scale scores should be evaluate using clinical assessments. These will be challenging tasks, given the paucity of gold-standard measures for Internet dependency and excessive use available at present. Third, additional work is needed to confirm whether Internet addiction symptoms cluster similarly in other relevant populations such as individuals with existing psychiatric comorbidities or those reporting more severe addiction symptoms.

Our results also shed light on the potential significance of the dimensionality of Internet addiction. While increased reporting of both symptom categories was present among those scoring higher on the IAT, the more substantial rise in excessive use symptoms suggests that changes in this subscale may be a better indicator of increasingly addictive behavior. Alternatively, given the modest prevalence of PIU noted in this sample (12%), we may not have fully captured the extent of dependent use symptoms effecting problematic users. Thus, while determining the absolute number of symptoms present has thus far been used to determine the severity of an individual’s problematic use, further investigation into the influence of symptom dimensions may help to refine this practice. Given the growing exposure to Internet use within our society, continuing to develop effective means for identifying both those at risk for and currently suffering from problematic behavior is of the upmost importance.