Problematic Internet Use: A Longitudinal Study Evaluating Prevalence and Predictors

Abstract

Objective

To assess the prevalence over time and predictors of problematic internet use using the Problematic and Risky Internet Use Screening Scale (PRIUSS). We also identified an intermediate-risk PRIUSS score.

Study design

In this longitudinal cohort study, we recruited participants using random selection from 2 colleges. Participants completed a yearly PRIUSS. We used multivariate logistic regression analysis to evaluate predictors of problematic internet use. We pursued receiver operating curve analysis to identify an Intermediate risk PRIUSS score. Finally, we applied Markov modeling to test the dynamics of moving through problematic internet use risk states over time.

Results

Of 319 participants, 56% were female, 58% were from the Midwest, and 75% were white. Problematic internet use prevalence estimates varied between 9% and 11% over the 4 years. Problematic internet use risk status from the previous time period was identified as the main predictor for problematic internet use (OR 24.1, 95% CI 12.8-45.4, P < .0001). Receiver operating curve analysis identified the optimal threshold for defining Intermediate risk was a PRIUSS score of 15.

Conclusions

This longitudinal study of problematic internet use among college students found that risks were present across groups and over time. The most salient predictor of problematic internet use was being at risk at the previous time point. On the basis of these results, we propose a PRIUSS score of 15 as an intermediate-risk cut-off to better identify those at risk of developing problematic internet use.

Problematic internet use, an ongoing health concern among adolescents and young adults, is defined as “Internet use that is risky, excessive or impulsive in nature leading to adverse life consequences, specifically physical, emotional, social or functional impairment.”¹ Problematic internet use has been associated with poor academic performance, stress, and fewer positive health behaviors.² Studies also have suggested bidirectional relationships between problematic internet use and other mental health conditions, such as depression.3, 4, 5

To date, there are no effective or efficacious interventions specific to problematic internet use. Thus, prevention and early identification of those at risk is paramount. To facilitate early identification of those at risk, a validated instrument is necessary. The Problematic and Risky Internet Use Screening Scale (PRIUSS)⁶ was developed based on a previous study that developed a conceptual framework for problematic internet use.¹ The current PRIUSS cutoff categorizes participants as “at risk” for problematic internet use or “not at risk” based on their overall PRIUSS score.⁷ To date, there are no criteria within the PRIUSS that identify participants at intermediate risk. Previous studies using the PRIUSS suggest that approximately 7%-11% of college students meet criteria as at risk for problematic internet use.6, 8, 9

The vast majority of studies of problematic internet use to date have been cross-sectional, yielding point prevalence for study samples at single time points. Few studies have evaluated risk factors for problematic internet use, although previous studies have identified correlations between problematic internet use and male sex.10, 11, 12 However, other studies focused on problematic social media use have identified correlations with female sex.¹³ A critical gap in the literature is longitudinal cohort studies to understand patterns of risk over time for individuals; little is understood about risk factors preceding the development of problematic internet use. Thus, the purpose of this longitudinal cohort study was to evaluate the prevalence estimate and predictors of problematic internet use over time among college students and to identify an intermediate-risk PRIUSS score toward earlier identification of being at risk.

Methods

Data from this study were derived from a larger prospective longitudinal cohort study focused on health and technology. Data used in this study included yearly participant interviews throughout 4 years of college.

Two large state universities were included in this study, one in the Midwest and one in the Northwest. Data for this study were collected yearly between May and June; overall study dates were between May 15, 2012, and June 15, 2015. This study was approved by the institutional review boards from the 2 participating universities.

Subjects

We recruited high school graduates the summer before college who were enrolled at 1 of the 2 participating universities. We randomly selected potential participants from the college registrar's lists of enrolled freshmen students at the 2 universities to recruit. Participants were eligible if they were between the ages of 17 and 19 years and enrolled as a freshman at 1 of these 2 universities. Informed consent was obtained from parents of minors and assent from minor participants, informed consent was obtained from all participants aged 18 and 19 years.

Recruitment

Potentially eligible incoming students were recruited using phone calls, e-mails, or Facebook messages. Sample size and exclusion criteria were established by the larger study; we excluded students who were non-English speaking, had begun college early, or did not have a Facebook profile at the time of enrollment.

Interview Procedure

This study included yearly phone interviews. Baseline interviews were conducted upon study enrollment to obtain demographic data. Trained staff conducted interviews yearly at the conclusion of each year of college. Participants were contacted by e-mail and phone to schedule their yearly interview. Interview data were recorded using a customized secure FileMaker database.

Measures

The PRIUSS was developed in 2011 and is grounded in a published conceptual framework for problematic internet use.¹⁴ The PRIUSS is an 18-item risk-based screening scale for problematic internet use with questions organized into 3 subscales: social impairment, emotional impairment, and risky/impulsive internet use. This scale has been validated in English and in Dutch.¹² The PRIUSS response options use a Likert scale with scores of 0 through 4. Answers include never = 0, rarely = 1, sometimes = 2, often = 3, and very often = 4. In the PRIUSS, a cutoff of 26 is used to identify those at risk of problematic internet use.⁶ Demographic variables included sex, race/ethnicity, and university attended.

Statistical Analyses

Demographic variables and PRIUSS scores were summarized in terms of frequencies and percentages. All reported P values were 2-sided, and P < .05 was used to define statistical significance. Statistical analyses were performed using SAS software, version 9.4 (SAS Institute, Cary, NC) and R software, version 2.15.1 (www.cran.r-project.org). Due to the limited number of non-white participants, the categories for race were collapsed into a binary white or non-white variable during analysis.

Prevalence Estimates of Problematic Internet Use over Time

Prevalence estimates of problematic internet use for each of the 4 years of the study were analyzed and reported along with the corresponding 95% CI. We evaluated differences over time using a logistic regression model with subject-specific random effects to account for repeated measurements.

Predictors of Problematic Internet Use over Time

Multivariate logistic regression analysis with subject-specific random effects was conducted to evaluate meeting criteria for at-risk for problematic internet use. The outcome variable was at risk at a current time point, the at-risk stats from the previous time point was included as a predictor. Additional predictors in the model included study year, sex, race, and university.

Defining an Intermediate-Risk PRIUSS Score

We also pursued whether having an intermediate-risk PRIUSS score could predict progressing to an at-risk PRIUSS score at future time points. We examined the association between the PRIUSS score from the previous time point and risk of problematic internet use at the current time point to identify an Intermediate risk cutoff. The optimal cut-off was determined by maximizing the Youden index (sensitivity + specificity − 1) of the receiver operating curve.¹⁵ Sensitivity and specificity estimates of the intermediate-risk cutoff were reported along with the corresponding 95% CIs.

Transitions between Risk States

We used a multistate Markov modeling approach to describe and evaluate the dynamics of moving through the no-at-risk and at-risk problematic internet use states over time.¹⁶ This allowed us to test the directionality and progression through at-risk states. We also assessed the Intermediate risk cutoff in these transitions between risk states. The results of the multistage Markov modeling were displayed graphically using a path diagram.

Missing Data

The majority of the participants (75%) completed the PRIUSS questionnaire every year; the completion rates for the 4 years of the study ranged between 81% and 98%. In the primary analysis, observations with missing PRIUSS assessment were excluded from the analysis. To evaluate the effect of missing data on the results, we conducted a sensitivity analysis. We compared the results obtained from the primary analysis with the results obtained from imputation-based analyses. Multi-imputation using the Markov chain Monte Carlo method was used to conduct the imputation-based analysis.¹⁷ Given there were no differences identified, we present the results without imputation.

Results

In total, 319 participants were included in this study. At the time of enrollment, participant's mean age was 18 years (SD 0.8). Participants were 56% female, 58% from the Midwest campus, and mostly white (75%). The initial response rate to recruitment was 54.6%. A total of 92.3% of participants remained in the study in 2015, and interview response rate was between 80.2% and 85.6% yearly. Table I describes demographic information for our participants.

Table I.

Participant demographics for 319 college students from 2 universities

Variables	n (%)
Sex
Female	179 (56.1)
Male	140 (43.9)
University
Northwest	131 (41.1)
Midwest	188 (58.9)
Race/ethnicity
White	240 (75.2)
Asian	37 (11.6)
Hispanic	10 (3.1)
More than 1	21 (6.6)
African-American/black	5 (1.6)
East Indian	3 (0.9)
Other	2 (0.6)
Native American/Alaska	1 (0.3)

Prevalence Estimates of Problematic Internet Use over Time

Across the 4 years of the study, the problematic internet use prevalence estimates varied between approximately 9% and 11%. The greatest prevalence estimate was 11.1% at the end of year 4 (Table II). We found no statistically significant differences (P = .34) in prevalence estimates of problematic internet use over time. Overall, 29 (9%) of participants were at risk at multiple time points. Figure 1 illustrates the frequency of participants at risk across one or multiple time points.

Table II.

Prevalence of problematic internet use over 4 years among college students from 2 universities

Time points	n (%)	95% LCL	95% UCL
Year 1	26 (9.2)	6.3	13.1
Year 2	26 (9.2)	6.3	13.2
Year 3	28 (10.6)	7.4	14.9
Year 4	29 (11.1)	7.8	15.5

Figure 1.

Frequency of participants at risk for problematic internet use across 1 or more time points across 4 years of college.

Predictors of Problematic Internet Use over Time

Within our multivariate model, only problematic internet use status from the previous time period was identified as a significant predictor for problematic internet use risk at the current time period (OR 24.1, 95% CI 12.8-45.4, P < .0001). Sex, race, and university were not significant predictors of subsequent problematic internet use risk.

Defining an Intermediate-Risk PRIUSS Score

The receiver operating curve for the intermediate-risk score that was associated with subsequent development of problematic internet use risk is shown in Figure 2. The optimal threshold for defining intermediate risk was a PRIUSS score of 15. At this score, the sensitivity was 89.6 and the specificity was 72.9. The positive predictive value was 13.3 and the negative predictive value was 99.4. Youden criteria was 62.6. In contrast, a PRIUSS score of 14 had a Youden criteria of 58.8.

Figure 2.

Receiver operating curve for the intermediate PRIUSS risk score that was associated with subsequent development of problematic internet use risk.

Transitions between Risk States

On the basis of the results for an intermediate-risk PRIUSS score, multistate Markov modeling defined high risk as a PRIUSS score of 26 or greater, intermediate risk was defined as a score of 15-25, and low risk was defined as a score of 14 or lower. Markov modeling revealed that although many participants remained at low risk throughout the longitudinal study, moving to intermediate risk was highly predictive of progressing to high risk. Figure 3 illustrates the path of these transitions.

Figure 3.

Markov model: This model illustrates the direction and proportion of participants moving through 3 different problematic internet use at risk states (low risk, intermediate risk, and high risk) over 4 time points.

Discussion

This study illustrates the prevalence estimates and predictors of problematic internet use over a 4-year time period among college students from 2 universities. We found that the mean prevalence estimate of problematic internet use over time had little variation—it remained between 9% and 11%. This finding suggests that there is not a single unique time point in college at which problematic internet use risk is significantly greatest compared with other time points. However, the prevalence estimates of 9%-11% throughout college in our sample appears at the greater end in comparison with previously reported in US samples.¹⁸ The nearly ubiquitous presence of wireless internet access, smartphones, and interactive media use among current college students19, 20 may be contributing to a rise in problematic internet use rates, or there may be other factors that explain this greater prevalence estimate.

Our study results indicated that no single demographic predictor identified college students at risk for problematic internet use. This finding is in contrast with earlier studies of problematic internet use that frequently identified males as a salient population at risk.10, 11 Our data are intriguing in that female and male students were both at risk. This finding may suggest that we were underpowered to detect subtle differences but also may be explained by the nearly universal engagement in internet-based technology among today's college students.19, 20

Findings also illustrated that participants moved in and out of being at risk for problematic internet use over time, as seen in Figure 1. This finding suggests that for some participants, problematic internet use may occur at a single time point in which psychosocial risk factors such as loneliness or stress¹⁴ intersect with a particular difficult semester or time period. In that time period, problematic internet use may manifest and then resolves as the situation improves. However, we also found that many participants were at risk for problematic internet use at more than 1 time point, both consecutively and separately. This suggests that the psychosocial risk factors for problematic internet use may be triggered at different time points and may benefit from intervention approaches to stop the cycle of problematic internet use.

We found that the most salient predictor of problematic internet use was being at risk at the previous time point. Thus, an intermediate-risk score cut-off for the PRIUSS scale may assist in earlier identification of college students at risk for developing problematic internet use. On the basis of data from this study, we propose an intermediate-risk cut-off of 15. As PRIUSS scores can range from 0 to 72, evidence supports that intermediate risk for problematic internet use be defined as scores of 15-25, and high risk for problematic internet use be defined as scores of 26 and greater. By refining this scale, we hope that this validated tool can be used to identify those at intermediate risk for problematic internet use and consider early prevention approaches.

Our study is not without limitations. Although the sample is representative of the colleges from which the data were drawn, the sample is not representative of all colleges, especially those with more racially and ethnically diverse students. Our ascertainment of all interview variables was limited to self-report, creating the possibility for recall and social desirability bias. We collected limited demographic information, additional measures of interest such as socioeconomic status, academic achievements, specific internet activities, or other health behaviors would have enhanced our understanding of risks for problematic internet use. Future studies could consider these additional measures. Further, our study selected the PRIUSS assessment tool, given it has a theoretical background¹⁴ and psychometric properties.6, 7, 21 However, other instruments to measure problematic internet use exists in the literature.22, 23 As with other longitudinal studies, missing data were present in our study. Our sensitivity analysis did not indicate concerns with the validity of our findings based on missing data. It is also important to recognize that controversy around problematic internet use still exists in the literature; one report described a European research network on problematic internet use and highlighted research needs including a need for interventions, policy approaches.²⁴

Despite these limitations, our study has important implications. Understanding problematic internet use as a public health concern that reaches across sex and racial/ethnic groups can contribute to our understanding of problematic internet use as a critical clinical issue for today's adolescents and young adults. The finding of problematic internet use prevalence estimates between 9% and 11% in our study suggests that this is a public health issue with prevalence rates similar to other common illnesses in this population, such as depression.²⁵ Thus, our study findings support screening of all adolescents and young adults for problematic internet use on a regular basis regardless of demographic characteristics. Use of the brief PRIUSS-3 scale,²¹ followed by the full PRIUSS-18 for those who screen positive, may help integrate problematic internet use screening into busy clinical practices in a consistent and efficient manner. Application of the intermediate-risk category for the PRIUSS may assist in identifying those who are likely to develop problematic internet use, and prompt a prevention plan. Although there are no current validated interventions for problematic internet use, it is possible that working with college students at intermediate risk may allow modification of technology behaviors to prevent the onset of problematic internet use. One potential tool that clinicians can use in these patient care scenarios is the American Academic of Pediatrics Family Media Use Plan (https://www.healthychildren.org/English/media/Pages/default.aspx).²⁶ Recent evidence suggests this tool may reduce certain problematic internet use symptoms, including social anxiety when away from the internet and withdrawal from the internet.²⁷ This study did not evaluate potential problematic internet use prevention approaches, and this area of research is needed urgently to advance the field. Future studies may use the intermediate-risk cutoff for the PRIUSS to identify adolescents and young adults who may benefit from a preventive problematic internet use intervention.