Testing the validity of the smartphone pervasiveness scale for adolescents with self-reported objective smartphone use data

Abstract

An ongoing and heated scientific debate pertains to the conceptualization and quantification of adolescents’ problematic smartphone use (PSU). To address the limitations of existing surveys, the smartphone pervasiveness scale for adolescents (SPS-A) has been designed to measure the subjective frequency of smartphone usage during significant moments within daily routines. Given the weak correlations in prior literature between self-reported PSU metrics and objective use data, this study investigates the relationships between diverse self-reported objective metrics of smartphone engagement—that is duration, frequency, and count of notifications—and the SPS-A scale, employing a cohort of Swiss adolescents (N = 1396; M_age= 15.8, SD_age= 0.81; 59% female). The findings reveal a substantial correlation between the total objectively measured duration of smartphone engagement and the SPS-A scale (r = .41 for iOS users and r = .42 for Android users). Moreover, a similar trend emerges as users are categorized by their level of objective use, with each category displaying a linear augmentation in smartphone pervasiveness levels. Instead, modest correlations emerge when considering the quantity of device unlocks and notifications. Noteworthy, no gender disparities emerged. These results add to our knowledge about the usefulness of the concept and measurement of smartphone pervasiveness: not only the SPS-A is a valid alternative to scales on “smartphone addiction” to capture non-pathological PSU, but it is also a better predictor of smartphone objective duration of use than self-reported measures. The correlation found between self-reported pervasiveness and actual use is discussed in light of the debate about the relevance of screen time in the study of PSU.

Introduction

Today, 95% of U.S. teens report having access to a smartphone, ¹ and one out of four children and adolescents report symptoms of problematic smartphone use (PSU). ² However, the definition of PSU, usually described as a compulsive pattern of usage resulting in negative consequences that harm user's daily functioning and relationships, ³ showed theoretical and methodological issues. These limitations include the conceptual differentiation with an addiction,^3,4 its overlapping with problematic social media use, ⁵ including its distinction from Facebook and Instagram overuse as well as its similarities with heavy WhatsApp use, ⁶ and the longitudinal link between use and problematic use. ⁷ Also, PSU scales correlate with some transdiagnostic (e.g. emotion dysregulation, fear of missing out) variables^8,9 and psychopathological (e.g. depression, anxiety) symptoms consistently,^10,11 while objective smartphone use does not associate with these constructs with the same consistency. ¹¹

To overcome these pitfalls, a recent approach ¹² proposed to measure what is called “smartphone pervasiveness.” The smartphone pervasiveness scale for adolescents (SPS-A) was first developed and validated in Italy, a country that has witnessed a substantial increase in the number of 9-to-16-year-olds who access the Internet daily from their smartphones^13,14 and a consequent increase in potential adverse effects of smartphone access on several life outcomes.^15,16 Similar trends and concerns can also be found in Canton Ticino, the Italian-speaking Region of Switzerland, where the current study is set.^7,17 SPS-A moves beyond the concept of addiction, in favor of a more detailed analysis of the specific moments of adolescents’ daily lives during which smartphone use could actually become problematic, without necessarily being perceived as such.^18–20

From a social science perspective, this latter approach has the advantage of focusing on uses that could affect adolescents’ functioning and, at the same time, be directly influenced by the social and cultural factors at the basis of digital inequality reproduction. Indeed, recent studies suggest that problematic digital media use is the most recent domain of digital inequality to be significantly influenced by individuals’ sociocultural resources and family background.^18,21 Based on this theory, smartphone pervasiveness has also been proposed as an indicator of a new form of digital inequality among youth, ¹⁸ thus being related to a lack of resources to channel and limit digital media use. The SPS-A scale also showed promising results as the outcome measure in the context of mobile media education interventions carried out in schools. ²²

We expect that the SPS is more relevantly correlated with objective metrics of smartphone engagement than what has emerged so far in PSU scales. First, SPS-A is not a measure of self-perceived discomfort related to smartphone overuse but a self-reported frequency of its use at certain times of the day. Such moments require the execution of demanding tasks and smartphone use can interfere with offline activities that are generally not compatible with online ones. ²³ Second, SPS-A does not indicate a pathological behavior that is typical of a specific segment of the youth population but measures smartphone-related time displacement that happens for every smartphone user. Consequently, SPS-A may be less susceptible to the impact of variations in the subjective perception of smartphone interferences than alternative PSU measures. Notwithstanding these advantages, the SPS-A still relies on adolescents’ self-reports, which may be subject to estimation biases. ²⁴ Self-report measures of smartphone use are indeed significantly affected by the difficulty of reporting frequent and habitual checking behaviors.^25,26

Considering that past self-reported measures of PSU showed little correlation with adolescent users’ log data, also depending on the type of data (e.g. duration vs. frequency),^24,27 it becomes crucial to evaluate the association between the SPS-A and objective smartphone use data. With the aim of filling this gap, this study wants to estimate the relationship between different metrics of smartphone use, such as duration, frequency, and number of notifications, and the SPS-A Scale in a group of Swiss Italian-speaking adolescents. It also aims to discuss the theoretical implications of such association for our understanding of concepts such as screen time, time displacement and the different dimensions of PSU.

Methods

Study design and sample

Data were collected from the first wave of the longitudinal “HappyB” project in Spring 2022, in a sample of Swiss adolescents living in canton Ticino, Italian-speaking Switzerland. The initial sample included 1662 students coming from 79 classes located in four high schools. Data were collected during school hours through an online questionnaire via Qualtrics in the presence of a teacher, after participants gave their consent within the platform. Of the initial sample, 145 participants were absent on the day of data collection or not willing to participate in the study, 28 did not complete their answers, and 57 reported an invalid value to the control question (i.e. “To check your attention, we ask you to select the number 3 from the list”). Based on z-scores > |3.5|, ²⁸ we excluded 27 participants as outliers. Also, we excluded students who reported an invalid answer to age (>18 years) (n = 5). The analytical sample included 1396 participants (M_age= 15.80, SD_age= 0.81), of whom 827 were female (59%). The study was approved by the regional education administration and by the ethics committee of USI Università della Svizzera italiana, Lugano (Switzerland).

Measures

For the present study, we used both self-reported and objective measures of smartphone use collected through a questionnaire.

Self-reported smartphone pervasiveness was assessed using the short version of the SPS-A, ²⁹ consisting of five items assessing daily-life moments relevant to adolescents’ well-being that might be influenced by smartphone overuse, i.e. dining with the family, spending time with friends, doing homework, watching television or movies, and awakening up during the night. Answer options ranged from 1 “Never” to 5 “Always” (M = 2.40, SD = .65, α = .60). The extended version of the SPS-A (7-item scale) was first validated on a sample of 3289 Italian high school students at grade 10, confirming its unidimensionality, factorial validity and measurement invariance by students’ ethnic origins, parental education, and gender. ¹² It showed a moderate correlation with students scores on the smartphone addiction scale (SAS), confirming its divergent validity from pathologic measures of problematic use, and negatively predicted students’ scores in literacy and math standardized tests. The short version of the scale (5 items) was instead administered and validated on a sample of 4675 Italian upper secondary school students (grades 9 to 13), obtaining satisfactory results both in terms of factorial validity and internal consistency. ¹⁸ The use of the short version is particularly useful for studies on public health topics involving large samples, hence we chose this version since it can be easily included in long surveys such as the HappyB study.

Objective smartphone use, including duration, number of unlocks (frequency), and number of notifications received. Objective smartphone data were collected for both iOS and Android users. For both iOS and Android users, the daily average duration of smartphone use (in hours and minutes), number of unlocks, and number of notifications received were captured by asking participants to go into their smartphone Settings while filling out the questionnaire and report the average estimate of the last week for the duration. For iOS users, the time window to refer to was the previous week, while Android users had to report two estimates of the previous week (one on a representative school day and one on a weekend day) that were combined to create a weighted week average.

Data analysis

Collected data were analyzed in R Studio 4.2.2. To test if self-reported and objective data were associated, we ran Pearson's and Spearman's correlation analyses and then tested whether objective smartphone use (categorized into three usage categories—low, medium, and high—based on interquartile ranges) would predict self-reported levels of smartphone pervasiveness. These categories included daily average duration, unlocks, and received notifications for iOS and Android users separately. After running descriptive analysis and a confirmatory factor analysis of the SPS-A scale, ANOVAs were used to test relationships between phone duration and pervasiveness scores, and Wilcoxon rank sum tests (non-parametric ANOVA) were used to test relationships between phone unlocks and the number of notifications and pervasiveness scores. Separate models were developed for Android and iPhone users, adjusting for sex. Tukey's HSD post hoc test was applied to compare different categories of smartphone duration. Dunn's test was applied to compare different categories among the non-normal distribution of the number of unlocks and notifications. To handle missing data, we used listwise deletion.

Results

The analytical sample (n = 1396) included 667 (47.8%) participants reporting a good financial status and 770 (55.2%) attending their first high school year (descriptive information is reported in Table 1). Results of the CFA of the SPS-A scale showed good fit indices (χ² = 10.221, p = .069, CFI = .994. RMSEA = .027, SRMR = .024) for one dimension of the scale. Means were calculated for each subdimension of the SPS-A scale (see Figure 1). Higher levels were reported for Item 5 “I use the smartphone as first thing in the morning, when I wake” and Item 1 “I use the smartphone while I am studying.” Correlations between single items of SPS-A can be found in the Supplementary Material (Table S1).

Table 1.

Descriptive results of the analytical sample.

	Boys (n = 587)	Girls (n = 836)	Overall (N = 1423)
Age
Mean (SD)	15.9 (0.829)	15.8 (0.792)	15.8 (0.809)
Median [Min, Max]	16.0 [14.0, 18.0]	16.0 [13.0, 18.0]	16.0 [13.0, 18.0]
Socio-economic status
Very good	175 (29.8%)	215 (25.7%)	390 (27.4%)
Good	279 (47.5%)	400 (47.8%)	679 (47.7%)
Average good	95 (16.2%)	175 (20.9%)	270 (19.0%)
Not so good	27 (4.6%)	29 (3.5%)	56 (3.9%)
Not good at all	1 (0.2%)	3 (0.4%)	4 (0.3%)
Missing	10 (1.7%)	14 (1.7%)	24 (1.7%)
High school
Bellinzona	139 (23.7%)	205 (24.5%)	344 (24.2%)
Locarno	151 (25.7%)	190 (22.7%)	341 (24.0%)
Lugano	157 (26.7%)	245 (29.3%)	402 (28.3%)
Mendrisio	140 (23.9%)	196 (23.4%)	336 (23.6%)
School year
First	321 (54.7%)	463 (55.4%)	784 (55.1%)
Second	266 (45.3%)	373 (44.6%)	639 (44.9%)
Smartphone type
iPhone	361 (61.5%)	670 (80.1%)	1031 (72.5%)
Android	225 (38.3%)	165 (19.7%)	390 (27.4%)
I don’t have a smartphone of my own	1 (0.2%)	1 (0.1%)	2 (0.1%)

Figure 1.

Bar plots for each SPS-A item clustered by gender.

The majority of participants (n = 1015, 72.7%) owned an iPhone. iOS users reported to spend an average of 285 minutes daily on the phone (SD = 123.7), unlocking the phone 102 times (SD = 51.6), and receiving 143 notifications (SD = 112) on average per day. Android users reported a weighted daily average of 205 minutes (SD = 102), unlocked the phone 52.2 times/day (SD = 33.5) and receive notifications 165 notifications daily (SD = 171) on average. All smartphone usage metrics significantly correlated with each other, with the number of unlocks and notifications being the most related. More information on smartphone usage metrics can be found in the Supplementary material Table S2 together with correlations (Table S3).

First of all, we checked the correlations between different metrics of objective use and students’ smartphone pervasiveness. Duration of smartphone use data was moderately correlated with smartphone pervasiveness levels (Corr_iOS= .41, p < .001; Corr_Android= .42, p < .001). The number of unlocks and notifications of smartphone data were less correlated with the scales's levels (Corr_iOSunlock= .26, p < .001, Corr_iOSnotif= .19, p < .001, Corr_{Androidunlock}= .19, p < .001, Corr_Androidnotif= .11, p = .05). Correlations among each SPS-A item and smartphone usage metrics can be found in the Supplementary Material (Table S4).

To test if the SPS is actually able to capture the differences of objective smartphone use, we divided the sample in low, medium, and high smartphone use categories and checked how these groups differed in their pervasiveness. To do this, we calculated the quartiles of the distribution of duration of use (Android_25th= 128.04, Android_50th= 195.9, Android_75th= 277.5; iOS_25th= 194, iOS_50th= 275, iOS_75th= 370), number of unlocks (Unlocks: Android_25th= 29.9, Android_50th= 47.9, Android_75th= 69.9; iOS_25th= 66, iOS_50th= 95, iOS_75th= 132.5), and number of notifications (Android_25th= 36.6, Android_50th= 107.3, Android_75th= 246.4; iOS_25th= 64, iOS_50th= 114, iOS_75th= 192).

Results of ANOVA models showed significant differences in smartphone pervasiveness by usage category for both iOS (see Figure 2) and Android users (F_Android= 32.9, df = 2, p < .001, η²= .150; F_iOS= 88.3, df = 2, p < .001, η²= .151). In particular, post doc comparisons showed significant differences among all three categories: medium-low, high-medium use, and to a larger extent, high-low (see Table 2). No differences were found between girls and boys.

Figure 2.

ANOVAs results for iOS (left) and Android (right) users for duration of smartphone use predicting SPS-A scores.

Table 2.

Group comparisons in SPS-A levels according to iPhone and Android metrics.

	Duration of use (minutes)		Number of unlocksa		Number of notificationsa
	Mean difference	p-Value	Z	p-Value	Z	p-Value
iPhone metrics
High - Low	0.70	<.001	7.31	<.001	5.05	.052
High - Medium	0.38	<.001	4.45	<.001	3.44	<.001
Low - Medium	0.32	<.001	−3.96	<.001	−2.38	<.001
Android metrics
High - Low	0.70	<.001	3.13	.05	NS	NS
High - Medium	0.45	<.001	2.14	.09	NS	NS
Low - Medium	0.25	.004	−1.47	.42	NS	NS

^aNon-parametric ANOVA.

In addition, the SPS-A category was also predicted by the number of unlocks (see Figures 3 and 4). The non-parametric ANOVA (H_diffAndroid= 9.96, df = 2, p < .001; H_diffiOS= 53.5, df = 2, p < .001) showed significant differences for a low versus medium number of unlocks, high versus medium, and high versus low (see Table 2).

Figure 3.

ANOVAs results for iOS (left) and Android (right) users for number of unlocks predicting SPS-A scores.

Figure 4.

ANOVAs results for iOS (left) and Android (right) users for number of notifications predicting SPS-A scores.

Additionally, the number of notifications received also predicted the levels of smartphone pervasiveness, but only for iOS users (H_diffAndroid= 3.59, df = 2, p = .17; H_diffiOS= 25.9, df = 2, p < .001). In particular, also in this case, significant differences were reported for low-medium number of notifications, high versus medium, but it was not significant for high versus low (see Table 2 and Figures 3 and 4).

Discussion

The SPS-A was originally built to go beyond existing measures of smartphone addiction, which are based on subjects’ perceptions of various types of discomforts related to smartphone usage. ⁸ The idea was to capture the potential impact of smartphone-related screen time on other relevant times of everyday life that have been highlighted in the literature as particularly problematic for adolescents’ social and physiological well-functioning. ¹² The SPS-A was then developed based on the concept of “pervasiveness.” ⁷

This study evaluated whether the SPS-A can effectively capture differences in objective smartphone uses. To do so, we administered SPS-A to a sample of Swiss adolescents and investigated its relationships with objective duration of use, number of unlocks, and number of notifications directly retrieved from their personal devices. As a result, SPS-A showed a medium size correlation with the objective duration of smartphone use (.41 for iOS users and .42 for Android users), while the correlation size was small for the number of unlocks and notifications. Similarly, when smartphone usage data were divided into low, medium, and high categories of smartphone usage, each category showed different levels of smartphone pervasiveness among both iOS and Android users (with the exception of Android notifications). SPS-A seems to be a better correlate of the actual duration of smartphone use than other measures of PSU. For example, Marciano and Camerini ²⁴ found a correlation of .150 between the Smartphone Addiction Scale for adolescents ³⁰ and the objective duration of smartphone use. SPS-A is also a good or even a better correlate of the actual duration of use compared to self-reported measures of use and measures of “smartphone addiction.” ²⁴ The positive correlation with duration of smartphone use highlights (i) the relevance of specific daily moments to predict the overall computation of device usage among youth; (ii) the relevance of duration of smartphone use to predict its time displacement potential during key moments of daily life. Notwithstanding justified calls to go beyond screen time in the analysis of youths’ smartphone usage, ³¹ our results indicate that objective usage duration is significantly correlated to a pervasive use in specific moments highlighted in the literature as problematic for adolescents’ social and physiological well-functioning.

The lower correlation emerging for notifications and unlocks may again be due to the nature of the smartphone pervasiveness construct itself. Pervasiveness is indeed a dimension of PSU primarily related to screen time in opposition to other daily life activities, while it was not designed to measure the frequency of checking behaviors or different types of use. Our results confirm that SPS-A, like other measures of PSU, ²⁷ gives better insight into the analysis of the amount of smartphone use as problematic instead of compulsiveness or modes of use. Also, this finding aligns with previous studies showing that the frequency of notifications themselves does not correlate with PSU ¹¹ ; however, the disrupted activities due to the frequency of notifications were associated with poorer mental health outcomes, like higher levels of depression, anxiety, and boredom proneness ³² on educational variables like a surface approach to learning ³³ —referring to a more fragmented knowledge with reduced ability to see relationships between ideas or concepts. ³⁴

Turning to the comparison by gender, the absence of differences between males and females in the results confirms that perceptions of discomfort—typical of addiction measures—could be more relevant in producing gender differences in PSU than objective measures of smartphone use. Girls probably manifest a higher sensitivity to discomforts connected to smartphone usage, but they do not seem to show significant differences neither in actual use nor in its pervasiveness during the day. Also, internalizing and externalizing problems are more frequently reported in girls than boys at that age. ³⁵ Finally, girls showed greater sensitivity to social comparison body image concerns, and they use social media apps and communication channels more intensively. ³⁶

Limitations of this study lie in the generalizability of the results beyond the Italian-speaking context. Although our sample belongs to a different nation (Switzerland) than the one in which the scale was originally validated (Italy), both samples are Italian-speaking and have a number of cultural traits in common. Future research will have to confirm the present results in different cultural and geographical contexts. Besides that, our study confirms that SPS-A can serve as a proxy of the total amount of time adolescents spend using smartphones, which currently seems to significantly go to the detriment of other relevant activities for their well-functioning and well-being.