Skip to main content
PMC home page
Journal of Behavioral Addictions logoLink to Journal of Behavioral Addictions
. 2017 Nov 10;6(4):620–629. doi: 10.1556/2006.6.2017.071

Psychometric validation of the Persian Bergen Social Media Addiction Scale using classic test theory and Rasch models

Chung-Ying Lin 1, Anders Broström 2, Per Nilsen 3, Mark D Griffiths 4, Amir H Pakpour 5,2,*
PMCID: PMC6034942  PMID: 29130330

Abstract

Background and aims

The Bergen Social Media Addiction Scale (BSMAS), a six-item self-report scale that is a brief and effective psychometric instrument for assessing at-risk social media addiction on the Internet. However, its psychometric properties in Persian have never been examined and no studies have applied Rasch analysis for the psychometric testing. This study aimed to verify the construct validity of the Persian BSMAS using confirmatory factor analysis (CFA) and Rasch models among 2,676 Iranian adolescents.

Methods

In addition to construct validity, measurement invariance in CFA and differential item functioning (DIF) in Rasch analysis across gender were tested for in the Persian BSMAS.

Results

Both CFA [comparative fit index (CFI) = 0.993; Tucker–Lewis index (TLI) = 0.989; root mean square error of approximation (RMSEA) = 0.057; standardized root mean square residual (SRMR) = 0.039] and Rasch (infit MnSq = 0.88–1.28; outfit MnSq = 0.86–1.22) confirmed the unidimensionality of the BSMAS. Moreover, measurement invariance was supported in multigroup CFA including metric invariance (ΔCFI = −0.001; ΔSRMR = 0.003; ΔRMSEA = −0.005) and scalar invariance (ΔCFI = −0.002; ΔSRMR = 0.005; ΔRMSEA = 0.001) across gender. No item displayed DIF (DIF contrast = −0.48 to 0.24) in Rasch across gender.

Conclusions

Given the Persian BSMAS was unidimensional, it is concluded that the instrument can be used to assess how an adolescent is addicted to social media on the Internet. Moreover, users of the instrument may comfortably compare the sum scores of the BSMAS across gender.

Keywords: adolescence, confirmatory factor analysis, differential item functioning, measurement invariance, social media addiction, Rasch analysis

Introduction

Internet accessibility worldwide has increased markedly as a result of the rapid improvement in technology. This development has led to many studies examining the use of various modern online activities, such as social media and Internet gaming (Cheng & Li, 2014; Kuss & Griffiths, 2012; Kuss, Griffiths, Karila, & Billieux, 2014; Mazzoni & Iannone, 2014; Ryan, Chester, Reece, & Xenos, 2014). Although individuals are rarely addicted to Internet itself, activities engaged through the medium of Internet appear to elicit addictive behaviors (e.g., gambling, gaming, and networking) in a small minority of individuals (Griffiths, 2017; Griffiths & Szabo, 2013). Addiction to such Internet-related activities may result in serious and negative psychosocial impact (Griffiths, 2000). Griffiths (2000) has argued that all addictions comprise six core components (i.e., salience, mood modification, tolerance, withdrawal, conflict, and relapse) and that individuals with Internet addiction experience preoccupation and cognitive distortions, deterioration of socialized behaviors, and unpleasant feelings or physical discomfort.

The latest version of the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (i.e., DSM-5), has acknowledged the potential negative impact of addiction to Internet-related activities on the well-being of an individual by recognizing Internet gaming disorder (IGD) as a tentative disorder (American Psychiatric Association, 2013). However, gaming is just one of many potentially addictive activities that can be engaged in online. Social media addiction appears to be a growing mental health problem according to recently reported evidence (e.g., Griffiths, Kuss, & Demetrovics, 2014; Pantic, 2014; Ryan et al., 2014). Therefore, some research has focused on developing a psychometrically robust instrument to assess addiction to social media, such as the six-item Bergen Facebook Addiction Scale (Andreassen, Torsheim, Brunborg, & Pallesen, 2012) and the Facebook Intrusion Questionnaire (Elphinston & Noller, 2011).

Although the two aforementioned scales have demonstrated good psychometric properties, a major limitation is that they do not assess all possible social media, i.e., they assess only one specific social media platform (i.e., Facebook). Moreover, the focus of both scales is on the medium itself rather than on the activities that can be carried out in social media (e.g., playing games such as Farmville; chatting with friends on Facebook; checking the status of a celebrity on Facebook). Given that people rarely become addicted to the Internet or social media platform itself, developing an instrument with accurate terminologies and sentence wordings is needed (Griffiths, 2013).

The Bergen Social Media Addiction Scale (BSMAS) is a newly developed scale that overcomes the aforementioned drawbacks, because it assesses the use of social media activities more generally rather than on one specific platform (Andreassen et al., 2016). However, to the best of our knowledge, the psychometric properties of this scale have only been examined in an Italian sample (Monacis, De Palo, Griffiths, & Sinatra, 2017a) and a nationally representative Hungarian sample (Bányai et al., 2017). Therefore, further psychometric testing of the BSMAS is needed. Both Monacis et al. (2017a) and Bányai et al. (2017) primarily applied classic test theory (CTT) methods to assess the psychometric properties of the BSMAS. Although their studies comprehensively applied a variety of methods to examine psychometric robustness of the BSMAS, they did not use Rasch models in their analysis.

To be confident that an instrument is psychometrically robust, its psychometric properties should be tested utilizing different statistical methods and across different populations (or ethnicities). Therefore, using Rasch models to test the psychometric properties of the BSMAS would provide additional support to the limited literature. The use of Rasch analyses can provide further understanding of the properties of the BSMAS including assessment of: (a) an interval scale to illustrate the BSMAS score (i.e., Rasch score with the unit of logit); (b) the psychometric between-item properties of the BSMAS (i.e., the BSMAS items) and person (i.e., the BSMAS raters); (c) the information about whether an item is redundant or out-of-concept; and (d) the threshold order of the response scales of the BSMAS (i.e., the 5-point Likert scale; Chang, Wang, Tang, Cheng, & Lin, 2014; Tractenberg, 2010).

This study was conducted to test the psychometric properties of the BSMAS in a Middle East country (i.e., Iran) to identify whether the BSMAS can be applied to such an ethnicity (i.e., Persian). First, the BSMAS was translated into Persian for high-school students. In addition, psychometric properties under CTT (e.g., internal consistency, composite reliability, construct validity, criterion-related validity, and floor and ceiling effects) and those under Rasch analyses (e.g., separation item and person properties and categorical functioning of the response scale) of the Persian BSMAS followed by its measurement invariance (using CTT method) and differential item functioning (DIF; using Rasch method) were examined.

Methods

Participants and procedure

The participants of this study were recruited from high schools (students aged 14–19 years) in Qazvin (Iran) using a two-stage sampling method. Thirty high schools were randomly selected from a total of 54 high schools. Then, in each school, three classes were randomly selected to participate in the study. Of 3,075 adolescents who were approached, 2,676 agreed to participate (response rate = 87%). All participants used social media.

Instruments

In addition to a series of standardized questionnaires, a background information sheet was used to collect data regarding participant demographic characteristics, including age, gender, parental education, current smoking status (yes vs. no), and hours spent on social media per day.

BSMAS

The BSMAS, which is adapted from the Bergen Facebook Addiction Scale (Andreassen et al., 2012), consists of six items rated on a 5-point Likert scale ranging from 1 (very rarely) to 5 (very often). The BSMAS applies the six core addiction elements (salience, mood modification, tolerance, withdrawal, conflict, and relapse) proposed by Griffiths (2005) to assess the experience of using social media over the last year (Andreassen et al., 2016). A higher score of the BSMAS indicates stronger addiction to the social media, and a BSMAS score over 19 indicates an individual is at-risk of developing problematic social media use (Bányai et al., 2017). Moreover, the structure of the BSMAS has been identified as a one-factor solution with measurement invariance supported across gender in an Italian sample (Monacis et al., 2017a).

The BSMAS was translated into Persian according to guidelines described by Beaton, Bombardier, Guillemin, and Ferraz (2000) for translation and cross-cultural adaptation of self-report measures. Consequently, several steps were taken. Two bilingual translators whose mother tongue was Persian independently translated the English version of the BSMAS. The two translators then met and resolved any discrepancies between the two translated versions to produce an interim version. The interim Persian version was then back-translated into English by two translators. The translators were not aware of the original English version. An expert committee including a methodologist, psychiatrist, psychologist, and a nurse consolidated all the versions of the BSMAS and developed a penultimate Persian version of the BSMAS for field testing. Finally, the Persian BSMAS was pretested on a sample of 67 high-school adolescents.

Internet Gaming Disorder Scale – Short Form (IGDS-SF)

Based on the concept of IGD proposed in the DSM-5, the IGDS-SF was designed to have nine items using the criteria for IGD in the DSM-5 (Pontes & Griffiths, 2015). All the IGDS-SF items are rated on a 5-point Likert scale ranging from 1 (never) to 5 (very often), and a higher IGDS-SF score indicates a higher level of IGD. The one-factor structure of the IGDS-SF has been supported, and the structure was found to be invariant across gender (Wu et al., 2017). The Persian IGDS-SF has gone through a rigorous method to ensure its linguistic validity. In addition, psychometric results indicated that the Persian IGDS-SF has excellent reliability and validity in a sample of Iranian adolescents (Wu et al., 2017).

Depression Anxiety Stress Scale (DASS)

The DASS comprises 21 items and assesses three psychiatric symptoms (i.e., depression, anxiety, and stress). Each symptom contains seven items that are rated on a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time; Lovibond & Lovibond, 1995). The internal consistency of the DASS was acceptable (α = .84 for depression, .91 for anxiety, and .86 for stress), and the factorial structure of the Persian DASS was previously confirmed (Asghari, Saed, & Dibajnia, 2008). Because of the previously demonstrated relationship between psychological health and social media addiction (Andreassen et al., 2016), the DASS was used for examining the criterion-related validity of the BSMAS.

Attention deficit hyperactivity disorder (ADHD) Rating Scale

The ADHD Rating Scale is an 18-item checklist based on DSM-IV criteria (Pappas, 2006). All the items are rated on a 4-point Likert scale ranging from 0 (never or not observed) to 3 (often or always). In addition to a total score, the ADHD Rating Scale contains two subscales: The inattention subscale contains all the odd-numbered items, and the hyperactivity subscale contains all the even-numbered items. A higher score indicates more ADHD-related behavioral problems. The two-factor structure of the ADHD Rating Scale has been confirmed, including its Persian version (Ghanizadeh & Jafari, 2010; Mercier et al., 2016). Moreover, the Persian ADHD Rating Scale showed satisfactory reliability and validity (Ghanizadeh & Jafari, 2010). Because of the revealed relationship between ADHD-related behaviors and social media addiction (Andreassen et al., 2016), the ADHD Rating Scale was used for examining the criterion-related validity of the BSMAS. The ADHD Rating Scale was completed by parents.

Data analysis

Descriptive statistics were used to demonstrate the characteristics of the participants and a Pearson’s correlation matrix was used to illustrate the relationships among collected variables. For psychometric testing, both CTT and Rasch methods were used to comprehensively evaluate the psychometric properties of the BSMAS, including the reliability, validity, categorical functioning in response scale, and measurement invariance (or DIF). More specifically, CTT was to investigate the ceiling/floor effects (a percentage of <20% suggests acceptable; Jette, Warren, & Wirtalla, 2005), internal consistency (Cronbach’s α > .7 indicates satisfactory; Cheng, Luh, Yang, Su, & Lin, 2016), corrected item-total correlation (a value of >0.4 is acceptable; Wang, Wang, & Shee, 2007), standard error of measurement (SEM) that assesses how the observed scores were attributed from the measurement errors (a value of <SD/2 is acceptable; Wuang, Su, & Huang, 2012), construct validity, and criterion-related validity.

In terms of the construct validity in CTT, confirmatory factor analysis (CFA) was used with diagonally weighted least squares (DWLS) estimator. A DWLS estimator was applied because it is suitable for Likert-type scales (Lin, Burri, Fridlund, & Pakpour, 2017; Lin, Oveisi, Burri, & Pakpour, 2017), such as the BSMAS. A series of fit indices were used: comparative fit index (CFI) and Tucker–Lewis index (TLI) > 0.9; root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) < 0.08; non-significant χ2 indicates the support of construct validity (Cheng et al., 2016; Lin et al., 2012). Moreover, the factor loadings were retrieved and the uniqueness values from the CFA were used to compute average variance extracted (AVE) and composite reliability (CR) to additionally understand the construct validity and reliability, respectively, of the BSMAS; AVE > 0.5 and CR > 0.6 are the recommended cutoffs (Bagozzi & Yi, 1988; Fornell & Larcker, 1981).

Multigroup CFA was carried out for testing the measurement invariance of the BSMAS at the scale level. There were three models in the multigroup CFA: configural model (Model 1; i.e., a first-order BSMAS framework with all six items loading on the underlying concept of social media addiction); metric invariance model (Model 2; i.e., a model based on Model 1 to constrain all factor loadings being equal across gender); scalar invariance model (Model 3; i.e., a model based on Model 2 to additional constrain all item intercepts being equal across gender). Measurement invariance in the multigroup CFA was assessed by comparing Model 1 with 2 for metric invariance and Model 2 with 3 for scalar invariance (Bagheri, Jafari, Tashakor, Kouhpayeh, & Riazi, 2014), and the recommendations indicating measurement invariant were ΔCFI > −0.01, ΔSRMR < 0.01, and ΔRMSEA < 0.015 (Chen, 2007).

Criterion-related validity was assessed by constructing several regression models with BSMAS or IGDS-SF as a dependent variable. For the regression model using BSMAS as the dependent variable, there were seven independent variables (IGD, depression, anxiety, stress, inattention, hyperactivity, and daily hours spent on social media) with two confounders (age and gender). For the regression model using IGDS-SF as the dependent variable, there were six independent variables (depression, anxiety, stress, inattention, hyperactivity, and daily hours spent on social media) with two confounders (age and gender). The standardized coefficient (β) was used to demonstrate the criterion-related validity.

The strongest association was expected to be IGDS-SF, followed by daily hours spent on social media, and other psychological (e.g., anxiety and depression) or behavioral problems (e.g., inattention and hyperactivity). These criteria were used because prior studies have demonstrated that social media addiction is weakly correlated to psychological health and ADHD-related behavioral problems, whereas strongly correlated to IGD and daily hours spent on social media (Andreassen et al., 2016; Monacis, De Palo, Griffths, & Sinatra, 2017b). Moreover, it was anticipated that the regression models using BSMAS as the dependent variable would have similar results of those using IGDS-SF as the dependent variable.

In terms of Rasch analyses, a Rasch partial credit model was used to report the item difficulty, and to examine item validity, categorical functioning in the response scale, item and person separation reliability, and item and person separation index. The item difficulty was presented using an interval unit (i.e., logit, a higher logit represents a more difficult item). Item validity was tested using information-weighted fit statistic (infit) mean square (MnSq), and outlier-sensitive fit statistic (outfit) MnSq with a recommended range between 0.5 and 1.5 suggested a good fit (Jafari, Bagheri, & Safe, 2012). A lower value in MnSq indicates an item being more redundant, and a higher value indicates being more out-of-concept (Khan, Chien, & Brauer, 2013). Categorical functioning was used to understand whether the Likert-type response scale of BSMAS had all categories for each item located in their expected orders (Jafari, Bagheri, Ayatollahi, & Soltani, 2012). More specifically, participants who are more addicted to social media are more likely to select a higher point in the BSMAS item (i.e., these respondents would be expected to select “very often” instead of selecting “often”). Item and person reliability with a value >0.7 suggests good reproducibility in the order of item difficulty and person’s underlying ability (Chang et al., 2014). Item and person indices with a value >2 suggest that the BSMAS has the ability to separate individuals or items into more than two distinct groups (Kook & Varni, 2008).

In addition, DIF in the Rasch models was used to test measurement invariance across gender at item level. Unlike multigroup CFA that tests the measurement invariance based on the entire BSMAS structure, DIF tests the measurement invariance for each item. Therefore, DIF can clearly identify which items for males are easier or harder than females to fulfill the item description (Lin, Yang, Lai, Su, & Wang, 2017). The commonly used cutoff for indicating substantial DIF is the DIF contrast (i.e., the difference of difficulty between two groups) >0.5 DIF (Shih & Wang, 2009).

IBM SPSS 23.0 (IBM Corp., Armonk, NY) was used to perform descriptive statistics and the CTT analyses except CFAs. CFAs were analyzed using the lavaan package (Rosseel et al., 2017) in R software; the Rasch models (including the DIF analyses) were carried out using WINSTEPS 3.75.0.

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki, and the study was approved by the Ethics Committee of Qazvin University of Medical Sciences. All participants were informed about the study, fully understood the study purpose, and all provided an informed consent. It should also be noted that parental consent was sought for those participants younger than 18 years of age.

Results

Table 1 shows the characteristics of the participants (N = 2,676). They had a mean age of 15.54 years (SD = 1.21). Slightly more than half were males (56.5%), and over one-quarter of them were currently smokers (28.5%). On average, they spent 3.75 hr/day on social media (SD = 1.13). The prevalence of being at risk to social media addiction in the sample was 22.4% for whole sample (15.1% for females; 28.1% for males). The correlation coefficients among the variables are shown in Table 2. BSMAS was highly correlated with IGDS-SF (r = .734), and both BSMAS and IGDS-SF were moderately correlated with daily time spent on social media (r = .583 and .476, respectively). Depression was moderately correlated with anxiety (r = .519). All other correlations were low or negligible (r < .25).

Table 1.

Participants’ characteristics (N = 2,676)

Mean ± SD or n (%)
Age (years) 15.54 ± 1.21
Gender (male) 1,511 (56.5)
Fathers’ educational year 7.74 ± 3.90
Mothers’ educational year 6.25 ± 3.59
Currently smoker (yes)a 763 (28.5)
Score in Bergen Social Media Addiction Scale 15.24 ± 4.83
Score in Internet Gaming Disorder Scale – Short Form 22.43 ± 7.91
Score in depressionb 7.70 ± 4.61
Score in anxietyb 8.39 ± 4.82
Score in stressb 7.79 ± 4.97
Score in inattentionc 3.58 ± 2.01
Score in hyperactivitiyc 5.06 ± 2.09
Total score in ADHD Rating Scalec 8.63 ± 2.89
Daily hours spent on social media (hours) 3.75 ± 1.13
Open in a new tab

Note. ADHD: attention deficit hyperactivity disorder; SD: standard deviation.

aFifteen participants did not report their smoking status. bMeasured using Depression Anxiety Stress Scale. cMeasured using ADHD Rating Scale.

Table 2.

Correlation matrix among tested variables

r (p value)
1. BSMAS 2. IGDS-SF 3. Depressiona 4. Anxietya 5. Stressa 6. Inattentionb 7. Hyperactivityb 8. Timec
1. – .734 (<.001) .214 (<.001) .167 (<.001) .125 (<.001) .222 (<.001) .131 (<.001) .583 (<.001)
2. – .206 (<.001) .128 (<.001) .088 (<.001) .145 (<.001) .108 (<.001) .476 (<.001)
3. – .519 (<.001) .068 (.001) −.010 (.622) .061 (.002) .108 (<.001)
4. – .016 (.426) .048 (.014) .002 (.931) .116 (<.001)
5. – .023 (.241) .011 (.584) .118 (<.001)
6. – −.001 (.970) .118 (<.001)
7. – .004 (.817)
Open in a new tab

Note. BSMAS: Bergen Social Media Addiction Scale; IGDS-SF: Internet Gaming Disorder Scale – Short Form.

aMeasured using Depression Anxiety Stress Scale. bMeasured using ADHD (attention deficit hyperactivity disorder) Rating Scale. cDaily time spent on social media.

The satisfactory psychometric properties of the BSMAS were supported by both CTT and Rasch analyses. At the BSMAS item level (Table 3), the results demonstrated that all six items had quite strong factor loadings in the CFA (0.64–0.83) and had acceptable corrected item-total correlations (0.56–0.69). No items had an infit or outfit MnSq outside of the recommended range (infit MnSq = 0.88–1.28; outfit MnSq = 0.86–1.22), and no items displayed substantial DIF across gender (DIF contrast = −0.48 to 0.24). Moreover, the 4-point Likert scale was functioned in order, and respondents more addicted to social media selected higher points on each BSMAS item (Figure 1); the higher Rasch score indicates more severe in the BSMAS (Figure 2).

Table 3.

Psychometric properties of the Bergen Social Media Addiction Scale (BSMAS) and endorsement rates of social media addiction in item level

Item no. Classical test theory Rasch analyses Endorsement rate (%)
Factor loadingsa Item-total correlation Infit MnSq Outfit MnSq Difficulty DIF contrastb Whole sample Males Females
BSMAS1 0.79 0.68 0.94 0.92 0.17 0.13 20.8 26.1 14.0
BSMAS2 0.64 0.56 1.28 1.22 0.64 −0.48 16.1 20.0 11.2
BSMAS3 0.83 0.69 0.88 0.92 −0.14 0.13 26.0 33.0 17.1
BSMAS4 0.72 0.65 1.00 1.00 0.04 0.12 19.2 24.5 12.4
BSMAS5 0.69 0.64 1.03 0.99 −0.25 −0.14 23.9 28.5 17.9
BSMAS6 0.82 0.69 0.88 0.86 −0.46 0.24 31.4 39.3 21.1
Open in a new tab

Note. Infit: information-weighted fit statistic; Outfit: outlier-sensitive fit statistic; MnSq: mean square error; DIF: differential item functioning.

aBased on confirmatory factor analysis. bDIF contrast across gender = difficulty for females − difficulty for males; and a value >0.5 indicates substantial DIF.

Figure 1.

Figure 1.

Open in a new tab

Ordered categorical functioning in the response scale of Bergen Social Media Addiction Scale (BSMAS)

Figure 2.

Figure 2.

Open in a new tab

The relationship between Rasch score and the Bergen Social Media Addiction Scale (BSMAS) score

At the BSMAS scale level (Table 4), both ceiling (2.2%) and floor effects (4.7%) were negligible. The internal consistency was very good (α = 0.86). AVE (0.51) and CR (0.86) were higher than the suggested cutoff, and SEM (1.81) was lower than the SD/2 in the sample (i.e., 2.415). All the fit indices in the CFA were excellent except the significant χ2 test. The separation reliability shown in Rasch analyses was satisfactory in both item (0.99) and person (0.80). Although the person separation index was slightly lower than expected (1.99), the item separation index performed extremely well (11.80).

Table 4.

Psychometric properties of Bergen Social Media Addiction Scale at scale level

Psychometric testing Value Suggested cutoff
Classical test theory
Ceiling effects (%) 2.2 <20
Floor effects (%) 4.7 <20
Internal consistency (Cronbach’s α) 0.86 >0.7
Confirmatory factor analysis
χ2 (df) 86.53 (9)* Non-significant
Comparative fit index 0.993 >0.9
Tucker–Lewis index 0.989 >0.9
RMSEA (90% confidence interval) 0.057 (0.046, 0.068) <0.08
SRMR 0.039 <0.08
Average variance extracted 0.51 >0.5
Composite reliability 0.86 >0.6
Standard error of measurement 1.81 <SD/2a
Rasch analysis
Item separation reliability 0.99 >0.7
Item separation index 11.80 >2
Person separation reliability 0.80 >0.7
Person separation index 1.99 >2
Open in a new tab

Note. RMSEA: root mean square error of approximation; SRMR: standardized root mean square residual.

a

The SD/2 was 2.415 based on our results (see Table 1 for the SD information).

*

p < .001.

As the item level of BSMAS displayed no DIF among all items, the measurement invariance at the scale level was further examined. The multigroup CFA (Table 5) concurred with the results of Rasch analyses that male and female adolescents interpreted the BSMAS similarly in both metric and scalar invariances.

Table 5.

Measurement invariance across gender on social media through confirmatory factor analysis

Model and comparisons Fit statistics
χ2 (df) or Δχ2 (Δdf) p value CFI or ΔCFI SRMR or ΔSRMR RMSEA or ΔRMSEA
Models
M1: Configural 109.849 (18) <.001 0.991 0.038 0.062
M2: Plus all loadings constrained 123.562 (23) <.001 0.990 0.041 0.057
M3: Plus all intercepts constrained 152.385 (28) <.001 0.988 0.046 0.058
Model comparisons
M2−M1 13.713 (5) .02 −0.001 0.003 −0.005
M3−M2 28.823 (5) <.001 −0.002 0.005 0.001
Open in a new tab

Note. M1 is a configural model; M2 is a model based on M1 to additionally constrain all factor loadings being equal across gender; M3 is a model based on M2 to additionally constrain all item intercepts being equal across gender. CFI: comparative fit index; SRMR: standardized root mean square residual; RMSEA: root mean square error of approximation.

The criterion-related validity of the BSMAS was tested with the criteria of IGDS-SF, DASS, ADHD Rating Scale, and daily hours spent on social media. After controlling for age and gender, it was found that all the aforementioned criteria were significantly correlated to BSMAS (p ≤ .01). The magnitudes were as expected. IGDS-SF showed the highest relationship (β = 0.542), followed by the daily time spent on social media (β = 0.281). It was also found that IGDS-SF shared similar results with the BSMAS (Table 6).

Table 6.

Criterion-related validity of the BSMAS using regression models with adjustment for age and gender

BSMASa IGDS-SFb
Criterion β p value β p value
IGDS-SF
Internet gaming disorder 0.542 <.001 – –
Depression Anxiety Stress Scale
Depression 0.038 .01 0.147 <.001
Anxiety 0.039 .01 −0.014 .49
Stress 0.039 .002 0.019 .29
ADHD Rating Scale
Inattention 0.101 <.001 0.087 <.001
Hyperactivity 0.061 <.001 0.085 <.001
Self-report
Daily hours spent on social media 0.281 <.001 0.427 <.001
Open in a new tab

Note. BSMAS: Bergen Social Media Addiction Scale; ADHD: attention deficit hyperactivity disorder; IGDS-SF: Internet Gaming Disorder Scale – Short Form.

a

Total R2 (adjusted R2) = .624 (.623); the R2 (adjusted R2) of age and gender was .075 (.075).

b

Total R2 (adjusted R2) = .262 (.260); the R2 (adjusted R2) of age and gender was .040 (.039).

Discussion

This study tested the psychometric properties of the BSMAS in Iran to investigate whether the BSMAS can be applied in Persian ethnicity. We found that the Persian BSMAS had a one-factor structure invariant across male and female adolescents. Other psychometric properties found in the Persian BSMAS include satisfactory internal consistency and promising criterion-related validity, as well as minimal (nearly negligible) floor and ceiling effects. Moreover, this is the first study to use Rasch models to investigate the BSMAS, and the BSMAS shared robust properties using CTT psychometric testing and Rasch models. More specifically, the categorical functioning in the 5-point Likert scale was supported for all the BSMAS items. Therefore, the BSMAS scoring method is arguably appropriate (i.e., adding up the BSMAS item scores to yield a total score). Although the study did not aim to recruit adolescents with online addiction problems, the sample had a high prevalence rate of being at risk to social media addiction using the Persian BSMAS. The main reasons for this are likely to be the popularity of the smartphone use with Wi-Fi access (Pew Research Center, 2016), which provides adolescents with 24/7 exposure to social media. In addition, adolescents appear to have great knowledge in using the modern technology but appear to have poor control in resisting its use. However, it should be noted that BSMAS scores are indicative and the only way of assessing a genuine addiction to social media would be through a clinical diagnosis from an appropriate practitioner.

The results of the acceptable reliability and satisfactory validity were comparable with the previous version of the BSMAS, Bergen Facebook Addiction Scale (Andreassen et al., 2012; Pontes, Andreassen, & Griffiths, 2016). Moreover, the results concerning the factorial structure being one-factor are consistent with the previous Italian findings (Monacis et al., 2017a), which is the only study that has tested the psychometric properties of BSMAS. This study extended the evidence of the one-factor structure of the BSMAS from CTT findings to the Rasch models. Moreover, the results agree with those reported by Monacis et al. (2017a) that males and females read the BSMAS items using similar interpretations. Again, DIF examinations were used in the Rasch models to support the measurement invariance results of BSMAS at item level. However, there were two results slightly different from the findings by Monacis et al. (2017a): First, the results did not correlate the uniqueness of Item 1 to that of Item 2, whereas Monacis et al. (2017a) did. Second, this study found both metric and scalar invariances in the BSMAS across gender, whereas Monacis et al. (2017a) only found metric invariance across gender.

Because the characteristics of the participants were directly compared with those of Monacis et al. (2017a), two major differences of different countries and different ages were identified. The sample in this study solely included Iranian adolescents, whereas the sample of Monacis et al. (2017a) included Italian adolescents and young adults. Because different countries may have different access to or using habits concerning the social media, it is possible that the BSMAS structure is slightly different between centuries. In addition, the cognitive ability between adolescents and adults is different (Lin, Scheerma, Yaseri, Pakpour, & Webb, 2017), which implies that these different age groups may have different interpretations of BSMAS items. Nevertheless, the one-factor structure of the BSMAS was confirmed, and the only difference was whether Items 1 and 2 additionally shared similar concept outside of the BSMAS. Moreover, by carefully examining the results reported by Monacis et al. (2017a), results from this study demonstrated that their CFA without relating uniqueness is supported, if somewhat loose cutoff for RMSEA is applied. That is, their RMSEA (0.094) is lower than 0.1, which is an alternative suggestion for RMSEA (Lin, Fung, Nikoobakht, Burri, & Pakpour, 2017).

The scalar invariance was not supported in the Italian sample (Monacis et al., 2017a), if the ΔCFI and ΔRMSEA values were used. Their ΔCFI was −0.02 and ΔRMSEA was 0.027, and CFI was 0.97 and RMSEA was 0.08 for the scalar invariance model. However, some argue that if the constrained model shows acceptable model fit (i.e., CFI > 0.9 and RMSEA < 0.08), the measurement invariance still can be supported even the ΔCFI and ΔRMSEA are not satisfactory (Wu, Chang, Chen, Wang, & Lin, 2015). Moreover, an alternative of ΔCFI (≥0.02), which supports the measurement invariance for the findings of Monacis et al. (2017a), has been proposed (Limbers, Newman, & Varni, 2008, 2009; Lin et al., 2012).

Only weak associations were found between the BSMAS and the criteria of psychological and behavioral problems, although the associations were significant. Therefore, the findings somewhat agree with the reported correlations among social media addiction, psychological health, and ADHD-related behavioral problems (Andreassen et al., 2016). More specifically, the BSMAS score was found to correlate with the IGDS-SF score (Andreassen et al., 2016; Monacis et al., 2017b). However, more evidence from other populations is needed. For example, the participants in this study were apparently healthy adolescents, and they might not have mental health issues or ADHD-related behavioral problems. Such a sample might develop the aforementioned problems after they become addicted to social media. However, they might not have had such problems during the data collection period as they might not have been exposed to social media for long enough. It was postulated that stronger relationships may be found in the sample with mental health problems or with ADHD-related behaviors.

There are some limitations in the study. First, no data were collected as to when the participants started to use the social media, and this might affect the relationships between BSMAS score, DASS score, IGD-SF score, and ADHD Rating Scale score. Second, no data were collected regarding psychiatric diagnosis of the participants. Confounding effects from psychiatric disorders cannot be ruled out. Third, high-school students were only recruited for testing the psychometric properties of the BSMAS; the generalization of the results to people in other age groups (e.g., adults) is therefore restricted. The present authors would encourage future studies to explore the psychometric properties of the BSMAS in other age groups. Fourth, because this study only recruited one ethnicity (i.e., Persian), there was no comparison of the BSMAS between Western and Eastern countries, which could have important implications for healthcare professionals. Future studies are warranted to explore the measurement invariance or DIF issues across countries (or cultures) for the BSMAS.

Conclusions

This study showed the unidimensionality of the Persian BSMAS in a large sample of Iranian students. The Persian BSMAS also had robust psychometric properties, including the supported categorical functioning in the response scale. Moreover, both the measurement invariance and DIF-free items were supported for the Persian BSMAS. Based on the aforementioned results, healthcare professionals and providers can easily use the summated score of the Persian BSMAS to compare the level of social media addiction between genders.

Funding Statement

Funding sources: None.

Authors’ contribution

AHP created and organized the study and collected the data. C-YL wrote the first draft, analyzed, and interpreted the data; AB and PN provided the directions of data analysis; MDG supervised the entire study and was responsible for all final editing. AHP, C-YL, AB, PN, and MDG critically reviewed the manuscript and provided constructive comments.

Conflict of interest

The authors declare no conflict of interest.

References

  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association. [Google Scholar]
  2. Andreassen C. S., Billieux J., Griffiths M. D., Kuss D. J., Demetrovics Z., Mazzoni E., Pallesen S. (2016). The relationship between addictive use of social media and video games and symptoms of psychiatric disorder: A large-scale cross-sectional study. Psychology of Addictive Behaviors, 30(2), 252–262. doi:10.1037/adb0000160 [DOI] [PubMed] [Google Scholar]
  3. Andreassen C. S., Torsheim T., Brunborg G. S., Pallesen S. (2012). Development of a Facebook Addiction Scale. Psychological Reports, 110(2), 501–517. doi:10.2466/02.09.18.PR0.110.2.501-517 [DOI] [PubMed] [Google Scholar]
  4. Asghari A., Saed F., Dibajnia P. (2008). Psychometric properties of the Depression Anxiety Stress Scales-21 (DASS-21) in a non-clinical Iranian sample. International Journal of Psychology, 2(2), 82–102. [Google Scholar]
  5. Bagheri Z., Jafari P., Tashakor E., Kouhpayeh A., Riazi H. (2014). Assessing whether measurement invariance of the KIDSCREEN-27 across child-parent dyad depends on the child gender: A multiple group confirmatory factor analysis. Global Journal of Health Science, 6(5), 142–153. doi:10.5539/gjhs.v6n5p142 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bagozzi R. P., Yi Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. doi:10.1007/BF02723327 [Google Scholar]
  7. Bányai F. Zsila Á., Király O. Maraz A. Elekes Z. Griffiths M. D. Andreassen C. S. Demetrovics Z. (2017). Problematic social media use: Results from a large-scale nationally representative adolescent sample. PLoS One, 12(1), e0169839. doi:10.1371/journal.pone.0169839 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Beaton D. E., Bombardier C., Guillemin F.,, Ferraz M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. doi:10.1097/00007632-200012150-00014 [DOI] [PubMed] [Google Scholar]
  9. Chang K.-C., Wang J.-D., Tang H.-P., Cheng C.-M., Lin C.-Y. (2014). Psychometric evaluation, using Rasch analysis, of the WHOQOL-BREF in heroin-dependent people undergoing methadone maintenance treatment: Further item validation. Health and Quality of Life Outcomes, 12(1), 148. doi:10.1186/s12955-014-0148-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Chen F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. doi:10.1080/10705510701301834 [Google Scholar]
  11. Cheng C., Li A. Y. (2014). Internet addiction prevalence and quality of (real) life: A meta-analysis of 31 nations across seven world regions. Cyberpsychology, Behavior, and Social Networking, 17(12), 755–760. doi:10.1089/cyber.2014.0317 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cheng C.-P., Luh W.-M., Yang A.-L., Su C.-T., Lin C.-Y. (2016). Agreement of children and parents scores on Chinese version of Pediatric Quality of Life Inventory Version 4.0: Further psychometric development. Applied Research in Quality of Life, 11(3), 891–906. doi:10.1007/s11482-015-9405-z [Google Scholar]
  13. Elphinston R. A., Noller P. (2011). Time to face it! Facebook intrusion and the implications for romantic jealousy and relationship satisfaction. Cyberpsychology, Behavior, and Social Networking, 14, 631–635. doi:10.1089/cyber.2010.0318 [DOI] [PubMed] [Google Scholar]
  14. Fornell C., Larcker D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. doi:10.2307/3151312 [Google Scholar]
  15. Ghanizadeh A., Jafari P. (2010). Cultural structures of the Persian parents’ rating of ADHD. Journal of Attention Disorders, 13(4), 369–373. doi:10.1177/1087054709332421 [DOI] [PubMed] [Google Scholar]
  16. Griffiths M. D. (2000). Internet addiction – Time to be taken seriously? Addiction Research, 8(5), 413–418. doi:10.3109/16066350009005587 [Google Scholar]
  17. Griffiths M. D. (2005). A ‘components’ model of addiction within a biopsychosocial framework. Journal of Substance Use, 10(4), 191–197. doi:10.1080/14659890500114359 [Google Scholar]
  18. Griffiths M. D. (2013). Social networking addiction: Emerging themes and issues. Journal of Addiction Research & Therapy, 4, e118. doi:10.4172/2155-6105.1000e118 [Google Scholar]
  19. Griffiths M. D. (2017). Commentary: Development and validation of a self-reported questionnaire for measuring Internet search dependence. Frontiers of Public Health, 5, 95. doi:10.3389/fpubh.2017.00095 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Griffiths M. D., Kuss D. J., Demetrovics Z. (2014). Social networking addiction: An overview of preliminary findings. In Rosenberg K., Feder L. (Eds.), Behavioral addictions: Criteria, evidence and treatment (pp. 119–141). New York, NY: Elsevier. [Google Scholar]
  21. Griffiths M. D., Szabo A. (2013). Is excessive online usage a function of medium or activity? An empirical pilot study. Journal of Behavioral Addictions, 3(1), 74–77. doi:10.1556/JBA.2.2013.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Jafari P., Bagheri Z., Ayatollahi S. M. T., Soltani Z. (2012). Using Rasch Rating Scale model to reassess the psychometric properties of the Persian version of the PedsQLTM 4.0 Generic Core Scales in school children. Health and Quality of Life Outcomes, 10(1), 27. doi:10.1186/1477-7525-10-27 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Jafari P., Bagheri Z., Safe M. (2012). Item and response-category functioning of the Persian version of the KIDSCREEN-27: Rasch partial credit model. Health and Quality of Life Outcomes, 10(1), 127. doi:10.1186/1477-7525-10-127 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Jette D. U., Warren R. L., Wirtalla C. (2005). Functional independence domains in patients receiving rehabilitation in skilled nursing facilities: Evaluation of psychometric properties. Archives of Physical Medicine and Rehabilitation, 86(6), 1089–1094. doi:10.1016/j.apmr.2004.11.018 [DOI] [PubMed] [Google Scholar]
  25. Khan A., Chien C.-W., Brauer S. G. (2013). Rasch-based scoring offered more precision in differentiating patient groups in measuring upper limb function. Journal of Clinical Epidemiology, 66(6), 681–687. doi:10.1016/j.jclinepi.2012.12.014 [DOI] [PubMed] [Google Scholar]
  26. Kook S. H., Varni J. W. (2008). Validation of the Korean version of the pediatric quality of life inventory™ 4.0 (PedsQL™) generic core scales in school children and adolescents using the Rasch model. Health and Quality of Life Outcomes, 6(1), 41. doi:10.1186/1477-7525-6-41 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kuss D. J., Griffiths M. D. (2012). Online gaming addiction in children and adolescents: A review of empirical research. Journal of Behavioral Addictions, 1, 3–22. doi:10.1556/JBA.1.2012.1.1 [DOI] [PubMed] [Google Scholar]
  28. Kuss D. J., Griffiths M. D., Karila L., Billieux J. (2014). Internet addiction: A systematic review of epidemiological research for the last decade. Current Pharmaceutical Design, 20, 4026–4052. doi:10.2174/13816128113199990617 [DOI] [PubMed] [Google Scholar]
  29. Limbers C. A., Newman D. A., Varni J. W. (2008). Factorial invariance of child self-report across age subgroups: A confirmatory factor analysis of ages 5 to 16 years utilizing the PedsQL 4.0 generic core scales. Value in Health, 11(4), 659–668. doi:10.1111/j.1524-4733.2007.00289.x [DOI] [PubMed] [Google Scholar]
  30. Limbers C. A., Newman D. A., Varni J. W. (2009). Factorial invariance of child self-report across race/ethnicity groups: A multigroup confirmatory factor analysis approach utilizing the PedsQL 4.0 generic core scales. Annals of Epidemiology, 19(8), 575–581. doi:10.1016/j.annepidem.2009.04.004 [DOI] [PubMed] [Google Scholar]
  31. Lin C.-Y., Burri A., Fridlund B., Pakpour A. H. (2017). Female sexual function mediates the effects of medication adherence on quality of life in people with epilepsy. Epilepsy & Behavior, 67, 60–65. doi:10.1016/j.yebeh.2016.12.012 [DOI] [PubMed] [Google Scholar]
  32. Lin C.-Y., Fung X. C. C., Nikoobakht M., Burri A., Pakpour A. H. (2017). Using theory of planned behavior incorporated with perceived barriers to explore sexual counseling services delivered by health professionals in individuals suffering from epilepsy. Epilepsy & Behavior, 74, 124–129. doi:10.1016/j.yebeh.2017.06.011 [DOI] [PubMed] [Google Scholar]
  33. Lin C.-Y., Luh W.-M., Yang A.-L., Su C.-T., Wang J.-D., Ma H.-I. (2012). Psychometric properties and gender invariance of the Chinese version of the self-report Pediatric Quality of Life Inventory version 4.0: Short form is acceptable. Quality of Life Research, 21(1), 177–182. doi:10.1007/s11136-011-9928-1 [DOI] [PubMed] [Google Scholar]
  34. Lin C.-Y., Oveisi S., Burri A., Pakpour A. H. (2017). Theory of Planned Behavior including self-stigma and perceived barriers explain help-seeking behavior for sexual problems in Iranian women suffering from epilepsy. Epilepsy & Behavior, 68, 123–128. doi:10.1016/j.yebeh.2017.01.010 [DOI] [PubMed] [Google Scholar]
  35. Lin C.-Y., Scheerma J. F. M., Yaseri M., Pakpour A. H., Webb T. L. (2017). A cluster randomised controlled trial of an intervention based on the health action process approach for increasing fruit and vegetable consumption in Iranian adolescents. Psychology & Health, 32(12), 1449–1468. doi:10.1080/08870446.2017.1341516 [DOI] [PubMed] [Google Scholar]
  36. Lin C.-Y., Yang S.-C., Lai W.-W., Su W.-C., Wang J.-D. (2017). Rasch models suggested the satisfactory psychometric properties of the WHOQOL-BREF among lung cancer patients. Journal of Health Psychology, 22(4), 397–408. doi:10.1177/1359105315603474 [DOI] [PubMed] [Google Scholar]
  37. Lovibond P. F., Lovibond S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335–343. doi:10.1016/0005-7967(94)00075-U [DOI] [PubMed] [Google Scholar]
  38. Mazzoni E., Iannone M. (2014). From high school to university: Impact of social networking sites on social capital in the transitions of emerging adults. British Journal of Education Technology, 45(2), 303–315. doi:10.1111/bjet.12026 [Google Scholar]
  39. Mercier C., Roche S., Gaillard S., Kassai B., Arzimanoglou A., Herbillon V., Roy P., Rheims S. (2016). Partial validation of a French version of the ADHD-rating scale IV on a French population of children with ADHD and epilepsy. Factorial structure, reliability, and responsiveness. Epilepsy & Behavior, 58, 1–6. doi:10.1016/j.yebeh.2016.02.016 [DOI] [PubMed] [Google Scholar]
  40. Monacis L., De Palo V., Griffiths M. D., Sinatra M. (2017. a). Social networking addiction, attachment style, and validation of the Italian version of the Bergen Social Media Addiction Scale. Journal of Behavioral Addictions, 6(2), 178–186. doi:10.1556/2006.6.2017.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Monacis L., De Palo V., Griffiths M. D., Sinatra M. (2017. b). Exploring individual differences in online addictions: The role of identity and attachment. International Journal of Mental Health and Addiction, 15(4), 853–868. doi:10.1007/s11469-017-9768-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Pantic I. (2014). Online social networking and mental health. Cyberpsychology, Behavior, and Social Networking, 17(10), 652–657. doi:10.1089/cyber.2014.0070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Pappas D. (2006). ADHD Rating Scale-IV: Checklists, norms, and clinical interpretation. Journal of Psychoeducational Assessment, 24(2), 172–178. doi:10.1177/0734282905285792 [Google Scholar]
  44. Pew Research Center. (2016). Smartphone ownership and Internet usage continues to climb in emerging economies Retrieved August 17, 2017, from http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-Internet-usage-continues-to-climb-in-emerging-economies/.
  45. Pontes H. M., Andreassen C. S., Griffiths M. D. (2016). Portuguese validation of the Bergen Facebook Addiction Scale: An empirical study. International Journal of Mental Health and Addiction, 14(6), 1062–1073. doi:10.1007/s11469-016-9694-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Pontes H. M., Griffiths M. D. (2015). Measuring DSM-5 Internet gaming disorder: Development and validation of a Short Psychometric Scale. Computers in Human Behavior, 45, 137–143. doi:10.1016/j.chb.2014.12.006 [Google Scholar]
  47. Rosseel Y., Oberski D., Byrnes J., Vanbrabant L., Savalei V., Merkle E., Hallquist M., Rhemtulla M., Katsikatsou M., Barendse M., Chow M. (2017). Package ‘lavaan’ Retrieved June 17, 2017, from https://cran.r-project.org/web/packages/lavaan/lavaan.pdf
  48. Ryan T., Chester A., Reece J., Xenos S. (2014). The uses and abuses of Facebook: A review of Facebook addiction. Journal of Behavioral Addictions, 3(3), 133–148. doi:10.1556/JBA.3.2014.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Shih C.-L., Wang W.-C. (2009). Differential item functioning detection using the multiple indicators, multiple causes method with a pure short anchor. Applied Psychological Measurement, 33(3), 184–199. doi:10.1177/0146621608321758 [Google Scholar]
  50. Tractenberg R. E. (2010). Classical and modern measurement theories, patient reports, and clinical outcomes. Contemporary Clinical Trials, 31(1), 1–3. doi:10.1016/S1551-7144(09)00212-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wang Y.-S., Wang H.-Y., Shee D. Y. (2007). Measuring e-learning systems success in an organizational context: Scale development and validation. Computers in Human Behavior, 23(4), 1792–1808. doi:10.1016/j.chb.2005.10.006 [Google Scholar]
  52. Wu T.-H., Chang C.-C., Chen C.-Y., Wang J.-D., Lin C.-Y. (2015). Further psychometric evaluation of the Self-Stigma Scale-Short: Measurement invariance across mental illness and gender. PLoS One, 10(2), e0117592. doi:10.1371/journal.pone.0117592 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Wu T.-Y., Lin C.-Y., Årestedt K., Griffiths M. D., Broström A., Pakpour A. H. (2017). Psychometric validation of the Persian nine-item Internet Gaming Disorder Scale – Short Form: Does gender and hours spent online gaming affect the interpretations of item descriptions? Journal of Behavioral Addictions, 6(2), 256–263. doi:10.1556/2006.6.2017.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Wuang Y. P., Su C. Y., Huang M. H. (2012). Psychometric comparisons of three measures for assessing motor functions in preschoolers with intellectual disabilities. Journal of Intellectual Disability Research, 56(6), 567–578. doi:10.1111/j.1365-2788.2011.01491.x [DOI] [PubMed] [Google Scholar]

Articles from Journal of Behavioral Addictions are provided here courtesy of Akadémiai Kiadó

RESOURCES