The Problematic and Risky Internet Use Screening Scale (PRIUSS) for Adolescents and Young Adults: Scale Development and Refinement

Abstract

Problematic Internet use (PIU) is a growing health concern among adolescents and young adults. The purpose of this mixed-methods study was to develop and refine a theoretically-grounded and psychometrically-validated assessment instrument for PIU specifically tailored to adolescents and young adults. An item pool was developed using concept mapping and a review of the literature, and administered to 714 students from two universities between 18 and 25 years of age. Exploratory and confirmatory factor analyses were used in a development subsample (n=500) to construct the scale. A cross-validation sample (n=214) was used to confirm the scale’s reliability. The Problematic and Risky Internet Use Screening Scale (PRIUSS) is an 18-item scale with three subscales: Social Impairment, Emotional Impairment, and Risky/Impulsive Internet Use. Based on its strong theoretical foundation and promising psychometric performance, the PRIUSS may be a valuable tool for screening and prevention efforts in this population.

1. Introduction

Problematic Internet Use (PIU) among adolescents and young adults is a growing public health concern. An array of empirical studies have linked Internet overuse to negative outcomes including sleep disorders, personal injury, depression, and poor social and academic adjustment (Covell, Dion, & Dion, 1994; Dick, Pagan, & Holliday, 2007; Eagly, 1978; Edens & McCormick, 2000; Herbert, Ma, & Clemow, 1995; Hughes & Dodder, 1983; Ingrid & Moos, 1979; C. H. Ko, Yen, Chen, Chen, & Yen, 2008). PIU may also be an early warning sign in adolescents for other problematic behaviors such as alcohol or drug use (Fisoun, Floros, Siomos, Geroukalis, & Navridis, 2012).

The prevalence of PIU has been estimated using a variety of assessment tools and diagnostic approaches. Older adolescents engaged in post-secondary education report the highest levels of Internet use (Jones, Johnson-Yale, Millermaier, & Pérez, 2009), and may be at increased risk for PIU. Early investigations found a range in prevalence among US college students between 8% and 15% (Anderson, 2001; M. J. Lavin, Marvin, McLarney, Nola, & Scott, 1999; M. J. Lavin, Yuen, Weinman, & Kozak, 2004; Morahan-Martin & Schumacher, 2000; Scherer, 1997). However, recent studies of both college students and younger adolescents suggest the prevalence may be closer to 4% (Christakis, Moreno, Jelenchick, Myaing, & Zhou, 2011; Liu, Desai, Krishnan-Sarin, Cavallo, & Potenza, 2011), which places the condition on a comparable scale with other problem behaviors such as pathological gambling (Shaffer & Hall, 1996). The lack of uniformity in measurement approach within the field may explain the inconsistencies in estimated prevalence (M. A. Moreno, Jelenchick, Young, Cox, & Christakis, 2011).

At present, a theoretically grounded and psychometrically validated screening instrument for PIU is lacking. The majority of available instruments have been adapted from criteria from the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), either for substance abuse and dependency or pathological gambling (Meerkerk, Van Den Eijnden, Vermulst, & Garretsen, 2009; M. A. Moreno, Egan, Bare, Young, & Cox, In press; Young, 1998). Little is known regarding the translation of these criteria to PIU screening or prevention efforts. The theoretical foundations and psychometric properties of other instruments have been incompletely described or lack support from primary data (Caplan, 2002; R. A. Davis, Flett, & Besser, 2002). Finally, there is a notable lack of instruments with content domains that have been specifically designed to reflect the experiences of adolescents and young adults.

1.1. Study Approach

This study aims to develop a new instrument for assessing PIU that is specifically targeted to the older adolescent and young adult population. Based on their high rates of Internet use and potentially elevated baseline risk for PIU, college students were chosen as the target population for scale development.

The developed instrument, called the Problematic and Risky Internet Use Screening Scale (PRIUSS) is grounded in the conceptual framework of PIU in older adolescents described by Moreno, Jelenchick, & Christakis (2012), which defines the condition as “Internet use that is risky, excessive or impulsive in nature leading to adverse life consequences, specifically physical, emotional, social or functional impairment.” This theoretical framework describes a multidimensional view of PIU, with seven discrete constructs (Figure 1). Development of the framework included collaboration with both college students and the health providers caring for them.

Figure 1.

Conceptual framework of PIU in older adolescents ^a

^aFigure adapted from Moreno, Jelenchick & Christakis (2012)

2. Methods

This study was conducted between August of 2011 and May of 2012 at two public universities located in the Midwest. Approval of the study was granted by the Institutional Review Board at the University of Wisconsin. There were four distinct phases to the study: item pool development, survey data collection, scale development, and scale refinement. All participants completed informed consent prior to participating.

2.1. Phase I: Item Pool Development

2.1.1. Conceptual Framework

Establishing a guiding theoretical model is the first step in identifying the constructs to be measured by a new instrument and developing the corresponding content (Trochim & Kane, 2005). The theoretical framework guiding the current study was developed in a previous study through the use of a concept mapping approach, the detailed methods and results of which are reported elsewhere (M. A. Moreno, Jelenchick, & Christakis, 2012). In brief, concept mapping integrates qualitative processes with quantitative techniques in order to generate a visual representation, or “concept map,” of how ideas related to a complex topic cluster into core constructs. Due to its foundation in multidimensional scaling, concept mapping is a particularly suitable choice for scale construct validation (J. E. Davis, 1989). The phases of concept mapping also overlay methods for determining content validity, or the degree to which an instruments’ content relevantly and representatively reflects the conceptual definitions of its target constructs (Trochim & Kane, 2005).

In the first phase of the concept mapping study, participants generated a list of items in response to the prompt: “A symptom or characteristic of problematic Internet use in adolescents or young adults is …”. In the second phase, participants reviewed the full list of symptoms or characteristics and sorted them into groups based on meaning or theme. Participants’ sort data was used to produce the concept map describing PIU in adolescents and young adults. Participants also rated each item’s clinical utility by responding to the following prompt: “How strongly does each item individually identify the presence of PIU in adolescents and young adults?” on a scale of 1 (does not identify) to 7 (strongly identifies). The rating data was used to determine a mean rating for each item individually, and each cluster within the concept map. The seven constructs described within the concept map, which include: psychosocial risk factors, physical impairment, emotional impairment, social/functional impairment, risky internet use, impulsive internet use, and internet use dependency, formed the theoretical basis of the PRIUSS’ content domains. The list of symptoms and characteristics provided the basis for the item pool used for scale development.

2.1.2. Item pool refinement

To narrow the pool of potential items down to those that best represented the targeted constructs, items were removed from the pool that either: (1) scored in the bottom quartile of mean clinical utility ratings, or (2) yielded a mean rating at or below the response scale midpoint (4: item somewhat identifies the presence of PIU). A total of 75 items, or 70% of the initial symptom list, remained in the item pool after this reduction.

To maximize the representativeness of the initial item pool, a literature review was conducted for existing instruments for assessing PIU, as well as scales assessing highly comparable concepts such as Compulsive Internet Use or Internet Addiction. Ten scales were identified, which together constituted a bank of 183 questions (Anderson, 2001; Caplan, 2002; R. A. Davis, et al., 2002; Demtrovics, Szeredi, & Rozsa, 2008; Huang, Wang, Qian, Zhong, & Tao, 2007; Meerkerk, et al., 2009; Mitchell, Sabina, Finkelhor, & Wells, 2009; Morahan-Martin & Schumacher, 2000; Scherer, 1997; Young, 1998). The question bank was systematically reviewed for ideas that were not conceptually represented in the initial item pool. A total of 5 items were added to the item pool. In some cases closely related questions were drawn from multiple scales and merged into a new, representative item.

The resulting pool of 80 potential scale questions was reviewed and refined in a focus group conducted with current undergraduate students between the ages of 18 and 25 years. Participants were recruited from a large public university within the scope of a larger ongoing study, the detailed methods of which are reported elsewhere (M. A. Moreno, Jelenchick, & Christakis, 2012). During the focus group, item wording was altered when participants reached a consensus on changes that would improve clarity or comprehension, but did not alter question content. The full item pool organized by content domain is presented in the Appendix.

2.2. Phase II: Data Collection

2.2.1. Setting and Subjects

The refined item pool was administered within an online survey to a sample of current college students. Participants were recruited from two campuses: a large, public university located in a mid-sized town, and a medium, public university located in a large urban center. Participant inclusion criteria included being an undergraduate student between 18 and 25 years old. Participants were identified from seven diverse set of lecture courses on Journalism and Mass Communication, Biology, Education, Nursing, Public Health, and Computer Science.

A total of 714 students completed the survey (60% response rate). The sample was predominantly female (70%) and White (83%); the mean age was 19.7 (SD=1.4) years. Detailed demographic data for the sample can be found in Table 1.

Table 1.

Sample characteristics

	N (%)/M ± SD
	Total Sample N=714	Development subsample N=500	Validation subsample N=214
Gender
Females	498 (69.7)	284 (54.8)	214 (100)
Males	219 (30.3)	216 (43.2)	–
Race
African American/Black	8 (1.1)	2 (0.4)	6 (2.8)
Asian/Asian American	54 (7.6)	41 (8.2)	13 (6.1)
Caucasian/White	590 (82.6)	412 (82.4)	178 (83.2)
Hispanic/Latino	10 (1.4)	7 (1.4)	3 (1.4)
Multi-racial/unspecified	52 (7.3)	38 (7.6)	14 (6.5)
School
University 1	585 (81.9)	417 (83.4)	168 (78.5)
University 2	129 (18.1)	83 (16.6)	46 (21.5)
Major Field of Study
Agricultural, Biological, Environmental Sciences	133 (18.6)	83 (16.6)	50 (23.4)
Business	76 (10.6)	58 (11.6)	18 (8.4)
Engineering and Physical Sciences	54 (7.6)	51 (10.2)	3 (1.4)
Health Sciences	144 (20.2)	92 (18.4)	52 (24.3)
Humanities	99 (13.9)	72 (14.4)	27 (12.6)
Social Sciences	73 (10.2)	45 (9.0)	28 (13.1)
Education	35 (4.9)	21 (4.2)	14 (6.5)
Undeclared/Unspecified	100 (14.0)	60 (15.6)	22 (10.3)
Age	19.9 ± 2.1	19.8 ± 2.0	20.2 ± 2.4

2.2.2. Procedure

Participants were initially invited to take part in the study through a brief in-class announcement. A link to the study’s online survey site was distributed on flyers, and by the course instructor in an email or posting on the course webpage. Participants who completed the online survey received a $5 gift card incentive.

The survey included (1) demographic questions, (2) a short form of the Marlowe-Crowne Social Desirability scale (Reynolds, 1982) and (3) the complete item pool. Participants responded to the item scale on a 5 point scale (never, rarely, sometimes, often, and very often). Participants were also asked to report their average number of daily Internet use hours over the past month, as well as the total number of times per day they used the internet. Participants were instructed to include time spent on any Internet application, and use of both computers and mobile devices. Time spent text messaging was not included, unless text messages were being used to interact with an online application such as Facebook or Twitter.

2.2.3. Data Analysis

Statistical analyses were performed using SAS software version 9.2 (SAS Institute, Cary, NC). All p values were 2-sided, and p <.05 was used to indicate statistical significance. Descriptive statistics, including demographics and Internet use variables, were summarized as frequencies and percentages or means and standard deviations.

For analysis, participant data from the online survey was split into a development subsample and a cross-validation subsample. In order to maximize the generalizability of the development subsample used for scale construction, surveys from 216 male participants were combined with 284 surveys randomly selected from the total sample of female participants. The remaining 214 participants constituted the cross-validation sample.

2.3. Phase III: Scale Development

2.3.1. Initial Item Reduction

The development subsample was used to assess the individual items and determine the preliminary scale. Each item was assessed individually based on variation in responses and item-scale correlation. The association between each item and social desirability scale scores was calculated using Jackson’s Differential Reliability Index (DRI) (Streiner, 2003). Items with a DRI approaching zero are highly associated with social desirability and should be removed (Streiner, 2003).

2.3.2. Exploratory and Confirmatory Factor Analyses

Within the development subsample, an iterative exploratory factor analysis with Biquartimin rotation was conducted using the remaining items to explore the scale’s factor structure and reduce the total number of items. The Kaiser-Guttman criterion was used as the primary tool for determining the number of factors retained, and Bartlett’s test for sphericity along with scree plots were employed to confirm the selected number of factors. In order to increase the factor analytic validity of the scale and to reduce the length of the scale, items with factor loadings below 0.40 and multiple cross-loadings (>0.25) were removed. Cronbach’s alpha coefficients were computed for the remaining items to determine the internal consistency of the instrument.

A weighted least square approach based on the asymptotic variances and covariances of the polychoric correlations was used to fit the confirmatory factor analysis model (Muthen & Kaplan, 1992). The Goodness of Fit Index (GFI) and Root Mean Square Error of Approximation (RMSEA) estimate were used to confirm the fit of each model. Models with GFI indices above .90 are considered a good fit, with values closer to 1 showing a better fit (Stevens, 2002). RMSEA values below .08 are considered to indicate moderate to good fit, and values above .10 indicate poor fit (Brown & Cudeck, 1993). Analyses were repeated in the cross-validation sample using a confirmatory factor analysis model. The GFI and RMSEA were calculated to confirm the model’s fit, and Cronbach’s alpha coefficients were estimated to confirm the instrument’s reliability.

2.3.3. Overall and Subscale Scores

Analysis of variance was used to assess differences in both overall score and subscale scores across demographic variables and Internet use categories. The nonparametric Spearman rank correlation coefficient was computed between the overall and subscale scores and self-reported daily quantity of Internet use. As PIU symptoms are hypothesized to occur at excessive levels of Internet use (C.-H. Ko et al., 2009; Tao et al., 2010), it was expected that higher scores would correspond with greater estimates of daily Internet use.

2.4. Phase IV: Scale Refinement

During scale refinement, the PRIUSS’ face validity and content validity were assessed. First, a pilot group of college students without prior exposure to the project was recruited to individually review the scale. Each participant read through the items and made written suggestions related to item interpretation. The primary investigators made subsequent changes to an items’ wording when multiple comments suggesting similar changes.

Second, three focus groups were held, two with college students and a third with college health providers, which included pediatricians, adolescent medicine specialists, nurses, social workers, psychologists, and psychiatrists. During focus group discussions, participants were asked to reflect on the content and presentation of the PRIUSS, as well as how it could be best used. Participants were recruited within the scope of a larger ongoing study, the detailed methods of which are reported elsewhere (M. A. Moreno, Jelenchick, & Christakis, 2012). Student participants included current undergraduates between the ages of 18 and 25 years recruited from a large, public university. In the first two focus groups a total of 10 students participated; overall, the sample was 40% female, 90% Caucasian, and the mean age was 18.8 (SD=0.6) years. For the third focus group, health providers were recruited from the primary care clinic of the university’s student health service. A total of eight providers participated, the majority were physicians (62%) and female (63%); the mean number of years in practice was 17.5 (SD=10.9). Focus groups were led by a trained facilitator and lasted between 30 and 50 minutes. Participants received a small gift card or a meal for participating. Focus groups were audio recorded and then transcribed verbatim. Finally, the investigators reviewed the transcripts for themes in an iterative manner.

3. Results

3.1. Descriptive Statistics

On average, participants completing the online survey reported using the Internet 17.4 times per day (SD=50.5), and a total of 4.7 hours per day (SD=8.4). White participants reported less time online than non-white participants (M=4.4 SD=7.0 versus M=6.1 SD=13.6, p=0.05). There were no significant differences by gender, major field of study, or school attended.

3.2. Scale Development

3.2.1. Initial Item Reduction

Each item’s DRI was >0.25; thus, no items were removed as a result of social desirability bias. Four items were removed due to substantial uniformity in response values. Item-scale correlations for the remaining items (n=76) ranged from 0.326 to 0.711; just under half of the items (n=35) had a correlation of 0.60 or greater. Item-scale correlations for the full item pool are contained in the Appendix.

3.2.2. Exploratory and Confirmatory Factor Analyses

Three subgroups of items meeting progressively stricter minimum item-scale correlation thresholds (0.50, 0.55, and 0.60) were used as the starting item pools for the exploratory factor analyses. Scree plots for all subgroups indicated that either two or three factors should be retained (Figure 2), so for each subgroup, a two-factor and a three-factor model were fitted. With a two and three factor models, the proportion of variance explained was 88% and 96%, respectively (Table 2).

Figure 2.

Scree plot of exploratory factor analysis

Table 2.

Eigenvalues and variance explained by top four factors extracted in the exploratory factor analysis^a

Number of factors	Eigenvalue	Proportion of variance explained	Cumulative proportion of variance explained
1	14.5	79.0%	79.0%
2	1.6	8.6%	87.6%
3	1.2	6.8%	96.2%
4	0.6	3.2%	97.6%

^aComprised of items with an item-scale correlation ≥ 0.60

A thematic analysis guided by the original concept mapping framework was used to assess the six potential models based on the items’ consistency and representativeness in describing the targeted constructs. The three factor model for items with a minimum item-scale correlation of 0.60 was found to have the strongest theoretical support, and was retained for further analysis.

The initial three factor model contained 21 items. The GFI and RMSEA estimates were 0.89 and 0.07, respectively, indicating an acceptable fit for the data. Two items from the first factor were found to represent concepts highly similar to alternate items loading on the factor and were removed. A third item from the second factor was similarly repetitive within its factor and selected for removal. Item loadings and the corresponding standard errors for each factor are described in Table 3. Cronbach’s alphas for the subscales were 0.89, 0.90, and 0.88, respectively. The final 18-item three-factor model provided a good fit in the cross-validation sample (GFI=0.92, RMSEA=0.06).

Table 3.

Findings from the Confirmatory Factor Analysis of the Three Factor Model in the Development Subsample

Item	Abbreviated text		Factor Loading (SE)
Factor 1 – Social Consequences
Problems with communicating face to face			0.79 (0.039)
Experience increased social anxiety			0.79 (0.039)
Fail to create real-life relationships			0.78 (0.039)
Offline relationships suffer			0.78 (0.039)
Choose to socialize online instead of in-person			0.74 (0.040)
Skip out on social events			0.73 (0.040)
		Cronbach’s alpha	0.89
Factor 2 – Emotional Consequences
Feel anxious when away from internet			0.88 (0.036)
Feel irritated when not using the internet			0.86 (0.036)
Experience feelings of withdrawal when not using the internet			0.82 (0.037)
Feel angry when away from the internet			0.75 (0.039)
Feel vulnerable when the internet isn’t available			0.73 (0.039)
		Cronbach’s alpha	0.90
Factor 3 – Risky and Impulsive Internet Use
Allow time on the internet to negatively affect your school performance			0.77 (0.039)
Lose motivation to do other things that need to get done			0.76 (0.039)
Neglect responsibilities			0.75 (0.040)
Avoid other activities in order to stay online			0.72 (0.040)
Put internet use in front of important, everyday activities			0.70 (0.041)
Lose sleep due to nighttime internet use			0.65 (0.042)
Use the internet excessively			0.64 (0.042)
		Cronbach’s alpha	0.88

The first factor, Social Impairment, described difficulty communicating and socializing offline, as well as difficulty forming and maintaining relationships as a result of Internet use. The factor contained six items: five from the social/functional impairment content domain, and a sixth from the psychosocial risk factors content domain. The second factor, Emotional Impairment, described a maladaptive psychological connection to Internet use. The factor contained four items from the emotional impairment content domain, along with a fifth from the internet use dependency domain. The third factor, Risky/Impulsive Internet Use, described an inability to constrain Internet use, and interference with normal daily life due to Internet use. The factor consisted of seven items: three from the risky internet use content domain and one each from the physical impairment, social/functional impairment, impulsive internet use, and internet use dependency domains.

3.2.3. Overall and subscale scores

Participants had an overall mean score of 15.4 (SD=10.9). Mean subscale scores for the social Impairment, emotional Impairment, and risky/impulsive Internet use subscales were 2.6 (SD=3.5), 3.1 (SD=3.6), and 9.7 (SD=5.5), respectively. Table 4 describes differences in overall PRIUSS scores by demographic variables. There was a significant positive correlation between daily Internet use quantity and overall PRIUSS scores (rho=0.28, p<0.0001), as well as the three subscales (social subscale=0.18, p<0.0001; emotional subscale=0.19, p<0.0001; risky/impulsive subscale=0.29, p<0.0001).

Table 4.

PRIUSS scores by demographic groups

	Overall Score M (SD)	p-value
Gender		0.37a
Females	15.0 (10.1)
Males	16.4 (12.4)
Race		0.008a
Non-white	18.0 (12.2)
White	14.8 (10.5)
School		0.03a
University of Wisconsin – Madison	15.7 (10.7)
University of Wisconsin – Milwaukee	14.1 (11.6)
Major Field of Study		0.0002b
Agricultural, Biological, Environmental Sciences	12.9 (8.7)
Business	17.5 (9.9)
Engineering and Physical Sciences	20.5 (16.0)
Health Sciences	13.3 (10.7)
Humanities	17.7 (11.0)
Social Sciences	15.0 (10.5)
Education	14.0 (9.5)
Undeclared/Unspecified	16.0 (10.1)

^aWilcoxon two sample test

^bKruskal wallis test

3.3. Scale Refinement

3.3.1. Face Validity

After the pilot group’s individual evaluations of the PRIUSS, minor wording changes were made to six of the items. The final, formatted version of the PRIUSS is shown in Figure 3.

Figure 3.

The Problematic and Risky Internet Use Scale (PRIUSS)

3.3.2. Content Validity

During focus groups, both student and provider participants responded positively to the PRIUSS. All three groups rated the PRIUSS’ items as consistent with the theoretical framework as described by Moreno, Jelenchick & Christakis (2012), as well as their own expectations for such an instrument. Participants also discussed ways in which the PRIUSS could be incorporated into available health care services.

Providers acknowledged the difficulty in fitting an assessment into already busy visit schedules, with one physician noting, “There’s pressure to get our work done. I think in a physical when someone is coming in for guidance on their health, I think [the PRIUSS] could slip in quite easily, if the clinician is aware of the issues we are talking about.” Students echoed the idea of incorporating an assessment into overall health promotion, with one student remarking, “I think this is definitely something you could ask at yearly check-up. It could be grouped with alcohol, tobacco, or even seatbelt use questions.”

Providers also suggested the idea of using computer-based platforms for screening, with one describing, “If they could score it and see what it meant—and just like we have with alcohol, if they’re at that level [the score would show] ‘this may be something you need to talk to a healthcare provider about.’” Students noted the influence of where the scale was introduced, as one student summarized, “If I found it on the Internet then I wouldn’t really think of it too much as an issue, but if in a doctor’s office setting, then maybe I’d think about it more.”

4. Discussion

This study described the rigorous development and refinement of the PRIUSS, a self-report instrument for assessing PIU in adolescents and young adults. The operational definition of PIU and description of its constructs by Moreno, Jelenchick & Christakis (2012) guided designation of the instrument’s content domains, and the development of a pool of items describing the symptoms and characteristics of PIU. The items were assessed in a sample of college students using exploratory and confirmatory factor analyses to yield the 18-item instrument. Finally, qualitative assessment of the items’ face and content validity was used to refine the instrument. The PRIUSS has three subscales: social Impairment, emotional Impairment, and risky/impulsive internet use, each of which demonstrated robust internal consistency. Overall, the three-factor model provided both a solid representation of the theoretical framework and a strong fit for the empirical data.

Along with excellent reliability, the PRIUSS also demonstrated promising validity. The association between quantity of Internet use and PRIUSS scores is consistent with related work (C.-H. Ko, et al., 2009; Tao, et al., 2010). The stronger associations noted with both the overall score and risky/impulsive use subscale, as compared to the social and emotional impairment subscales, supports the construct validity of the scale. As quantity of Internet use is only a single component of the multidimensional PIU construct, the modest degree of association between daily Internet use and the PRIUSS scores was anticipated. Further, recent findings have suggested that self-reported point estimates of daily Internet use may not reliably represent actual online behaviors (M. A. Moreno et al., 2012). Thus, some degree of measurement error may also have influenced the associations.

A key strength of this study is the use of a strong theoretical foundation reinforced by primary data to guide construction of the PRIUSS. The consistency in item clustering between the previous concept mapping study, which used multidimensional scaling and cluster analysis, and the exploratory and confirmatory factor analyses employed here provides strong support for the PRIUSS’ construct validity. Further, the PRIUSS’ subscales reflect the content domains of the PIU conceptual framework that were identified as those most suited to detecting PIU during development of the conceptual framework (M. A. Moreno, Jelenchick, & Christakis, 2012). The results of the content validity assessment further validate the PRIUSS as a reflection of the perspectives of both college students and their health providers. They also suggest potential avenues for integrating PIU assessments into preventative care for college students. Additional refinements to the PRIUSS, such as the development of scoring guidelines, may enhance its potential for use in such a manner.

4.1. Limitations and Implications

There are several limitations to the current study. The sample was derived from a college population that was largely white, and the majority was female: generalization to other adolescents may not be warranted. Results also suggest that adolescents from particular groups, such as minorities and those with particular educational or career interests, may be at greater risk for PIU. Studies of diverse populations of adolescents and young adults may help to illuminate these differences. Finally, the PRIUSS’ design was informed by college students’ experiences and views of PIU. Thus, additional study is needed to assess how the PRIUSS will perform in younger adolescents and non-college attending older adolescents and young adults, and adjustments to the content may be needed.

It is also important to note that the PRIUSS was developed specifically for use in identifying problematic behaviors related to Internet use. Although constructs such as problematic cell phone use or video gaming are conceptually similar, further work will be needed to understand the validity of the PRIUSS for use in these domains. Similarly, in the current study, use of any online application was included in the overall consideration of Internet use. Internet use as a platform for other addictive behaviors, such as viewing pornography, gambling, or shopping, was not assessed. The current study also did not distinguish recreational Internet use from activities undertaken for school or work. Future studies may wish to selectively exclude these forms of Internet use to further examine the PRIUSS’ specificity in identifying symptoms resulting from recreational Internet use that are not explained by a comorbid behavioral addiction.

4.2. Conclusion

In summary, the PRIUSS is an 18-item scale with three subscales: Social Impairment, Emotional Impairment, and Risky/Impulsive Internet Use. The PRIUSS has a well-defined theoretical basis and promising psychometric support as a risk-based screening instrument for PIU in adolescents and young adults. Additional validation studies are needed to advance its use as a tool in detecting and intervening in this emerging health concern.

Highlights.

• Factor analysis of the item pool yielded a 3-factor, 18-item model

• There are 3 subscales: Emotional Impairment, Social Impairment, Risky/Impulsive Internet Use

• Focus group review support for both face validity and content validity

• The PRIUSS is a theoretically-grounded screening instrument for adolescent and young adult PIU

Abbreviations

PIU

Problematic Internet Use

PRIUSS

Problematic and Risky Internet Use Screening Scale

DSM-IV

Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition

SD

standard deviation

DRI

Jackson’s Differential Reliability Index

GFI

Goodness of Fit Index

RMSEA

Root Mean Square Error of Approximation

M

Mean