Enemies and friends in high-tech places: the development and validation of the Online Social Experiences Measure

Short abstract

Background

Interpersonal positivity and negativity are separable dimensions both recognized as having important effects on health. Though online social interactions and research examining them are increasingly common, no validated instrument has heretofore been developed to examine social support and social negativity specifically in online contexts. The present studies describe the development and validation of the Online Social Experiences Measure (OSEM) to allow for assessment of online social positivity and negativity simultaneously.

Methods and results

In Study I, responses to a potential item pool were collected from participants (N = 557), who were then subdivided into two groups for the purposes of cross-validation via exploratory and confirmatory factor analyses. Results suggested the emergence of two factors: online social support (OSEM+ subscale) and online social negativity (OSEM− subscale). Across Studies I and II (N = 139), the OSEM showed good convergent and discriminant validity, and both subscales had high internal consistency. Study II included an ambulatory cardiovascular assessment and found the OSEM− subscale to be predictive of elevated diastolic blood pressure and heart rate. The OSEM+ subscale was predictive of more favorable cardiovascular readings for some types of participants (e.g. those with a greater number of online-only social ties).

Conclusions

Overall, results suggest online social positivity and negativity may have implications for physical health, and that the OSEM may be a useful tool in understanding online social processes.

Introduction

Social support, social negativity, and health

A broad literature links social support to psychological wellbeing.¹ For example, virtually all studies examining perceived social support and happiness find positive associations between the two.² Conversely, low perceptions of social support predict negative symptoms, such as those of major depression and post-traumatic stress disorder.^3,4

Strong evidence has also linked social support to physical health.^5–9 Meta-analytic evidence has found greater social support is associated with reduced risk of mortality across gender, age, initial health status, and cause of death.⁷

Positive aspects such as social support, however, are only one dimension of relationships; even many close relationships are characterized by some degree of negativity.⁸ A review of evidence finds that negative components of interpersonal interaction have been linked to dysregulation of the cardiovascular, endocrine, and immune systems,¹⁰ elevated risk of coronary heart disease,¹¹ diminished sleep quality,¹² and increased mortality risk.¹³ Importantly, relationship positivity and negativity appear to be separable dimensions, independently predictive of health outcomes,^14,15 underscoring the need to assess both.

Social relationships and interaction online

An increasing volume of social interaction takes place in online communities and social networks. For example, in 2014, Pew Internet researchers found that 71% of web users had Facebook accounts and 52% of online adults used two or more social media platforms such as Twitter or Instagram.¹⁶ This number has risen from 2010, when 47% of adults reported using at least one social network site (SNS).¹⁷ The rising popularity of such online SNSs is also reflected in the number of monthly active users—in 2012, Facebook was the first SNS to reach one billion monthly active users, but by the end of 2018, this figure surpassed 2.32 billion.¹⁸

Online social interactions have implications for psychological and physical health that are still under exploration. For example, while directionality or causality is not yet known, research has suggested that Facebook users who access the network service multiple times per day score eight points higher (83/100) on the MOS Social Support Survey than the United States Average.^17,19 The same researchers interpret their findings to mean that “someone who uses Facebook multiple times per day gets about half the boost in total support that someone receives from being married or living with a partner.”¹⁷ While such claims likely oversimplify the complex—and sometimes negative—associations between online experiences and wellbeing, it does seem clear the Internet can be a useful source of social support.^20,21 Meta-analytic evidence suggests general use of SNSs can promote some types of social support.²² Research has indicated simply receiving “likes” is perceived as a form of support as well.²³ For many people, online networks may serve as sources of general social support. For example, evidence indicates Facebook may be a useful in enhancing perceived support among deployed military personnel,²⁴ informational support among parents,²⁵ and can serve as a catalyst for social connectedness and cognitive benefits among older adults.^26,27

For other individuals, at least a portion of benefits may come from online peer support groups. As an example, such groups have been found to be used as sources of support for diverse purposes, including career counseling,²⁸ cyberbullying victims,²⁹ and for promoting physical activity.^30,31 Evidence spanning decades suggests that although such groups are also widely used in illness contexts, and may have benefits for individuals such as some patients and family caregivers,^32–34 the links to psychological and physical wellbeing remain largely unclear and underexplored,^35–39 an issue perhaps in part complicated by the absence of appropriate measurement instruments linking online networks and interactions to health.⁴⁰

Much research examining online social networks has considered positive relationships.⁴¹ However, online platforms are also used for aversive interpersonal interactions (e.g., cyberbullying).⁴² A growing body of research has started considering negative online interactions among friends as well.⁴³ For example, negative interactions on SNSs significantly predicted thwarted belongingness,⁴⁴ which the interpersonal theory of suicide identifies as a source of suicidal desire.⁴⁵ Similarly, a meta-analytic review found bidirectional associations between Facebook use and loneliness.⁴⁶ Importantly, much of this work examines only social positivity or negativity rather than considering both simultaneously.

Measurement of online social positivity and negativity

The specific purposes and affordances of SNSs and other digital platforms vary widely. This is true of the cultures that emerge among their users as well.⁴⁷ Further, within specific sites or platforms, affordances and usages can change instantly or over time.^48,49 Because of this heterogeneity, and due to the several subtypes of social support,^8,50 and negativity, a successful measurement instrument should not be bound to any one platform, pattern of usage, or type of support. Given the associations between various aspects of support and relationships and health,^7,51 and theoretical models linking social support to wellbeing via both psychological and behavioral pathways,^51,52 the present studies aimed to develop a measure encompassing a broad range of online social experiences, including capitalization, emotional, informational, and belonging social support, as well as negative experiences such as exclusion, rejection, interference, and stigmatization.

Examination of both social positivity and negativity online is critical, given related work in offline contexts. For example, a key principle underpinning the literature on social relationships and health is that positive and negative aspects of social ties are separable dimensions.^15,53 In other words, two individuals who experience identical levels of social positivity online may have radically different exposure to online social negativity. Positivity and negativity can have both separate and interactive effects on psychological and physiological wellbeing.^12,54–56 For example, Birmingham and colleagues found that individuals’ perceptions of their spousal behavior as supportive was associated with lower ambulatory blood pressure (ABP).⁵⁴ When both positive and negative behaviors were perceived, however, the benefits of the positive behavior were not observed. Related effects have been found in work on other outcomes, such as coronary artery calcification and inflammation.^56,57 Analysis at the network level has shown the importance of such a separable positivity and negativity framework in understanding psychological distress as well,⁵⁸ underscoring the importance of assessing both dimensions. The separability of positivity and negativity—both conceptually and in prediction of health and wellbeing—has implications in terms of measurement of online experiences. Specifically, a unidimensional measure cannot fully capture the possible range of experiences, as low positivity does not necessarily imply high negativity (see Figure 1 in Uchino et al.,).¹⁵ An understanding of how online interactions relate to both positivity and negativity is crucial to the study of social support and its effects on health in the twenty-first century.

Figure 1.

Systolic blood pressure decreases for married participants with increasing OSEM+ scores.

Existing measures of online social support

While online social environments are increasingly explored in the literature, such research generally utilizes ad hoc scales (which typically have weak or unknown psychometric properties),⁴⁰ or measures developed for offline contexts. While online social interactions mirror their offline counterparts in some ways, there are also many fundamental differences,⁵⁹ such that offline measures may not be suitable for or relevant to online work, underscoring the need for development of a measure specific to online contexts.

Laudably, some researchers have painstakingly constructed and validated online support measures for their own purposes. For example, the Online Social Support for Smokers Scale (OS4) was subjected to psychometric examination.⁶⁰ Specialized items such as “Using QuitNet helped me cope with cravings,” limit the application of such scales more broadly. Moreover, the considerable time and effort required for the development and validation of such purpose-built measures represents a substantial deterrent to many researchers in following suit. Other existing measures developed specifically for online contexts, such as the useful Internet Social Capital Scales (ISCS),⁶¹ target related but distinct concepts. Perhaps the psychometrically and theoretically strongest existing instrument assessing social support in a general online context is Nick and colleagues’ recently-published Online Social Support Scale.⁴⁰ The OSSS includes four subscales assessing emotional, instrumental, and informational support, as well as social companionship, making it well-suited to examining various types of social support. Notably, however, assessment of both positivity and negativity is outside the scope of the OSSS, as well as that of measures such as the OS4 and ISCS. Indeed, it appears no validated instrument exists that assesses both social positivity and negativity specifically in online networks.

As in the offline social relationships literature, the implications of online social support/negativity may not be uniform across individuals. They may differ as a function of network characteristics (e.g., number of online contacts whom an individual has not also met in offline contexts) or sociodemographic factors (e.g., marital status, which has been shown to have distinctive implications for the associations for offline social support and ABP).⁶² While early research examining SNSs found such services were primarily used to connect with existing social ties, it appears that some users are increasingly forming new relationships online.⁶³ Previous work indicates that, especially for individuals with weaker offline social support, SNSs may represent nonredundant sources of social support or social negativity.⁶⁴ Such online-only social ties may be particularly impactful for individuals facing social isolation or other social constraints offline. For example, SNSs are critical sources of informational and emotional support for parental caregivers of children with special health needs.⁶⁵

Even for individuals without health conditions, online-only social ties may complement or supplement offline support. For example, while it is often assumed that marriage increases individuals’ social resources by integrating them into the networks of a new spouse, meta-analytic findings did not support this, with some evidence tentatively pointing toward a decline in personal social network size following marriage.⁶⁶ This would be consistent with evidence from a lifespan approach to the study of relationships. Such work suggests that overall offline social network size tends to decrease as aging individuals shift toward a smaller number of more intimate relationships.⁶⁷ Relatedly, classic work has suggested that marital relationships may constrain access to support from others offline, particularly in times of conflict or in troubled marriages,⁶⁸ consistent with more recent work indicating that individuals in an unhappy marriage were not protected from negative cardiovascular effects by supportive offline networks.⁶² Online sources of support may, therefore, be particularly important in such instances.

Summary

Study I of the present article focused on the development of an instrument to assess social support and negativity derived from online social networks. The need for this is underscored not only by the aforementioned unresolved questions in the literature, but also by the constant development of new platforms connecting broad communities or niche subgroups.⁴⁷ The measure was designed to achieve both good convergent and discriminant validity. In other words, because social support and negativity in online contexts have much in common with their offline counterparts, some degree of association between the Online Social Experiences Measure (OSEM) and established, offline measures was expected. However, because on- and offline social experiences and networks also differ in important ways, the new instrument should not be entirely redundant with offline measures of social experience or personality (e.g., OSEM-assessed social negativity should not be entirely explained by emotional instability).

Study II focused on using this new instrument to link online social support and negativity to relevant health processes. The implications of technology-mediated social interactions and networks for physical health are only beginning to be explored, but in the offline context, the evidence linking social relationships to physiology is perhaps strongest for cardiovascular processes.⁸ This suggests cardiovascular health is a reasonable starting point for exploring technology-mediated communication and health. Research has shown that ABP is influenced by social support and relationship quality.^54,62,69,70 While lab-based assessments of cardiovascular reactivity have also been linked to health,⁷¹ decades of research suggest elevated ABP may more strongly predict a variety of cardiovascular outcomes, including mortality and morbidity, organ damage (e.g., left-ventricular wall thickness, hypertrophy), and severity of complications of primary hypertension.^72–75

Aims

1. The present studies intended to develop the OSEM to assess social support and social negativity in online contexts. (Note that the only authorized pronunciation of this acronym is “awesome.”)

2. A second, complementary aim was to examine the validity of the OSEM in predicting ABP.

3. An ancillary aim was to examine the possibility that the implications of online social support/negativity differ by participant sociodemographics or social network characteristics. Marital status and number of online-only social connections were examined as potential moderators of the effects of online social support.

Study I: scale construction and cross-validation

Study I hypotheses

1. Factor structure: positive and negative dimensions would emerge when the factor structure of the OSEM was explored in subgroup 1, and additional support for a two-factor structure would emerge in a confirmatory factor analysis (CFA) in subgroup 2.

2. Convergent and discriminant validity: the link between the positivity subscale and the ISEL (a measure of offline support; see below) was predicted to be moderate and higher than the link between the positivity subscale and the TENSE (a measure of offline social negativity) or relevant personality measures. In addition, the association between the negativity subscale and the TENSE was expected to be moderate and higher than the association between the negativity subscale and the ISEL or the personality measures.

Method

Participants and design

Participants (N = 557) ranged from 20 to 69 years of age (mean = 39.1 years, SD = 11.17). The majority of the sample identified as female (58.7%), white (83.7%), and heterosexual (88.4%). A plurality indicated they were married or living with a mate (49.7%). Most were college graduates (38.8%) or had attended some college (33.8%). Individuals were recruited from Amazon’s Mechanical Turk (MTurk) and were divided into two groups for the purposes of cross-validation.⁷⁶ All participants in both Studies I and II provided informed consent and both protocols were approved by an Institutional Review Board.

Procedure

In order to assess a wide variety of online social experiences, several existing measures of offline support and negativity were consulted. The pool of potential items began with the Interpersonal Support Evaluation List (ISEL),⁷⁷ Social Support Questionnaire 6,⁷⁸ as well as the Test of Negative Social Exchanges (TENSE).¹⁴ Other potential items were adapted from existing measures of social support, including the MOS Social Support Survey,¹⁹ Social Support Questionnaire (SSQ),⁷⁹ and previous research examining support.^80,81 Additionally, new items were written specifically for the OSEM item pool to reflect affordances or experiences specific to online social networks and interaction. Participants were instructed with the accompanying text:

For the following items, think about your online social networks (for example, Facebook, Instagram, Twitter, Snapchat, etc.). In addition to thinking about your friends and followers on those networks, consider others you have interactions with, such as friends of friends. To respond, indicate to what extent you have felt this way during the past month.

A five-point response scale was provided in which 1 = very slightly or not at all, 2 = a little, 3 = moderately, 4 = quite a bit, and 5 = extremely. Questions were also included in order to better understand participants’ use of SNSs (e.g., “Approximately how many total Facebook friends do you have; How many Facebook friends do you have whom you have not met in person?”).

The initial item pool was tested on a focus group of 10 individuals in order to identify ambiguous or confusing wording. Focus group members ranged in age from mid-teens to late middle age so as to solicit feedback informed by a variety of online experiences. Members of the focus group individually completed the item pool in questionnaire format, and were asked to flag items they felt were unclear or about which they had questions. Some small adjustments to item phrasing were subsequently made in order to enhance clarity and avoid possible misinterpretation. Focus group responses also provided information on approximate time required to complete the items, which was used in the recruitment of the main study sample.

Participants from the MTurk sample were divided into two groups. All completed the pool of potential questions, and responses were examined for internal consistency, proper completion of attention check items, completion time, and other markers of response quality. Exploratory factor analysis (EFA) was used to extract the number of underlying dimensions from the results of each group, and the results compared in a cross-validation analysis in order to examine the replicability of the extractions via CFA.⁸²

Convergent and discriminant validity were examined via correlational analyses of the strength of the association between the OSEM subscales (i.e., positivity, negativity) and the ISEL/TENSE and the TIPI extraversion and emotional stability subscales.

Measures

Interpersonal Support Evaluation List (ISEL)⁷⁷

The ISEL contains 40 questions and assesses offline social support and the specific dimensions of appraisal, self-esteem, belonging, and tangible support. In previous research, it has shown high internal consistency ranging from 0.60 to 0.92,⁸³ with a 4-week test–retest reliability of 0.87 for the total scale. Internal consistency in the present studies was high (Study I α = .97, Study II α = .85). The reliability of the ISEL has also been established over a 6-month period.⁸³

Test of Negative Social Exchanges (TENSE)¹⁴

Comprising 18 items, the TENSE assesses global interpersonal stress (e.g., interference, inconsideration, anger). In the present study, it was used to examine convergent validity with the OSEM negativity subscale. In previous work, the scale has shown good psychometric properties and internal consistency.^14,84 Internal consistency was very high in the present studies (Study I α = .96; Study II α = .94).

Ten Item Personality Inventory (TIPI)⁸⁵

This 10-item measure is a brief five factor measure of personality and was used to demonstrate discriminant validity. The instrument has shown good test-retest reliability in past work, as well as acceptable convergent and discriminant validity during its development.

Demographics measure

Standard background information (e.g., race, marital status, income, etc.) was also collected.

Study I results

Subgroup 1: exploratory factor analysis

An EFA was conducted in SPSS (v20.0.0) using data from subgroup 1 (n = 2 79). Principal axis factoring was used as the extraction method (with results remaining similar under maximum likelihood extraction, as well as in principal components analysis). Oblique (oblimin with Kaiser normalization) rotation was used in order to allow factors to correlate with each other (though results remained similar when solutions used orthogonal [varimax] rotation). Sampling adequacy was examined using the Kaiser–Meyer–Olkin measure, yielding a value of .925. Bartlett’s test of sphericity was also favorable, χ²(5050) = 21,309, p < .001.

Scree plot analysis indicated two factors should be retained. This was consistent with examination of initial eigenvalues (factor 1 = 25.97; factor 2 = 16.91) and rotation sums of squared factor loadings (factor 1 = 24.45; factor 2 = 17.93). The inter-factor correlation was not significant, r = −.063, p = .29. As shown in Table 1, which gives standardized loadings for the rotated two-factor solution, factor 1 was clearly interpretable as online social support, whereas factor 2 clearly indicated online social negativity. In order to balance psychometric considerations with response burden, 40 total items were retained. (The following items included in the final OSEM were adapted from these existing sources: TENSE: OSEM items 2 and 9; ISEL: 20, 21; SSQ6: 12, 35; MSSS: 23; Abbey et al: 11.⁴⁴) Items were examined for the highest factor loadings, and 20 items were selected for factor 1, which were summed to constitute the OSEM Social Positivity Subscale (OSEM+). The process was repeated to select 20 items from factor 2, constituting the OSEM Social Negativity Subscale (OSEM−). Both subscales showed high internal consistency (OSEM+ α = .968; OSEM− α = .944).

Table 1.

Study I EFA estimated standardized factor loadings.

Item number	Item text	OSEM+ Loading	OSEM− Loading
OSEM+ subscale
3	There are people in my online social network who have faith in me and my abilities.	.730	−.214
4	Someone in my online social networks has encouraged me when I felt like quitting.	.655	.011
6	Members of my online social networks care about me as a person.	.773	−.188
10	I have felt supported by someone in my online social networks who agreed with my point of view.	.784	.042
12	There are people in my online social network who care about me no matter what is happening to me.	.782	−.154
14	My online social networks respond positively to the things I post.	.718	−.223
15	There is someone in my online social networks I could confide in or trust with a secret.	.698	−.142
17	People in my online social networks praise me for my accomplishments.	.795	−.126
19	People in my online social networks make me feel better about my life.	.752	−.064
20	When I feel lonely, there are several people I can talk to online.	.803	−.074
21	When I need suggestions on how to deal with a personal problem, I know someone in my online social networks I can turn to.	.819	−.015
23	There is someone in my online social networks who is available to listen if I just need to talk.	.812	−.087
27	Members of my online social networks give me useful information and advice when I want it.	.768	−.021
28	Members of my online social networks reassure me about choices I've made.	.782	.044
30	There are people in my online social networks who make me feel good on special occasions, like my birthday.	.733	−.113
32	Members of my online social networks show appreciation for what I do.	.815	−.044
34	Members of my online social networks listen when I want to confide about things that are important to me.	.834	−.075
35	Someone in my online social networks has cheered me up when I was feeling down.	.769	.085
37	There is someone in my online social networks I can turn to for advice about handling problems with my family.	.724	−.044
38	There is someone in my online social networks I could turn to for advice about making career plans or about changing my job.	.758	.046
OSEM− subscale
1	I have felt ignored or unimportant because others didn't respond to something I posted online	−.180	.671
2	Interactions with someone in my online social networks prevented me from working on my goals or other important things.	.044	.586
5	I have felt sad or rejected when someone criticized, downvoted, or disliked a post I made.	−.034	.767
7	Someone in my online social networks has dismissed my opinion or my beliefs.	−.031	.630
8	Someone in my online social networks has made me feel unwanted.	−.063	.792
9	Someone in my online social networks has been inconsiderate to me (for example taken me for granted, taken advantage of me, taken my feelings lightly).	.024	.785
11	Members of my online social networks act in unpleasant or angry ways toward me.	−.088	.709
13	I have felt excluded by others online. (For example, seeing friends at an event without me.)	−.125	.676
16	Someone in my online social networks has made me feel discouraged.	−.087	.746
18	I have been unable to fall asleep while thinking about a negative online interaction I had.	.089	.701
22	I have been upset by someone important to me being too friendly with someone else online.	.080	.569
24	I have felt uncomfortable because of unwanted attention I received from someone in my online social networks.	.106	.580
25	I have felt ashamed of myself because of content or comments someone posted online (even if they weren't targeted at me).	.073	.659
26	I have felt rejected when someone in my online social networks ignored me (for example ignored a friend/follow request or message I sent, or a post I tagged them in).	−.076	.683
29	I have felt like an outsider because no one in my online social networks shares my beliefs.	−.165	.683
33	I have felt I would be rejected by important members of my online social networks if I expressed myself freely online	−.107	.572
36	Someone has made me feel embarrassed or foolish online.	.024	.735
39	I have been unable to focus because of thoughts about a negative online interaction I had.	.094	.710
40	I have been hurt when someone unfollowed or unfriended me.	.071	.635
41	I have worried what people think of me because of content posted by my family or friends or others associated with me.	.060	.671

Convergent and discriminant validity

As expected, the OSEM+ subscale correlated with the ISEL, r = .623, p < .001 and showed a smaller inverse association with the TENSE, r = −.308, p < .001. The OSEM+ subscale showed a small correlation with extraversion, r = .131, p = .034, and was uncorrelated with emotional stability, r = .010, p = .874. The OSEM− subscale, as expected, showed a moderate correlation with the TENSE, r = .436, p < .001. The OSEM− subscale also showed a moderate inverse association with ISEL scores, r = −.465, p < .001. Consistent with hypotheses, the OSEM− subscale was uncorrelated with extraversion, r = −.024, p = .693 and emotional stability, r = .099, p = .109.

Subgroup 2: confirmatory factor analysis

The subscales were next used as the basis for a CFA in AMOS (v22.0.0), drawing on data from subgroup 2 (n = 278). The 20 items selected for the OSEM+ subscale during the preceding EFA were input as observed variables, and set to indicate a latent variable representing online social support. This common factor was allowed to correlate with a second unobserved variable representing online social negativity, which was indicated by the 20 items selected for the OSEM− subscale in the EFA. Estimates for item regression weights are presented in Table 2. Fit indices provided somewhat mixed evidence, as some indicated acceptable fit, χ²/df = 2.52, RMSEA = .074, 95% CI [.07, .078], SRMR = .071, while others were less favorable, χ²(739) = 1859.86, p < .001, TLI = .844. Internal consistency was high (OSEM+ α = .96; OSEM− α = .94).

Table 2.

Study I CFA estimated regression weights.

Item number	b	SE	β	Variance
OSEM+ subscale
3	.764	.048	.755	.463
4	.832	.064	.665	.915
6	.855	.048	.804	.418
10	.816	.052	.749	.547
12	.876	.055	.754	.610
14	.636	.049	.660	.549
15	.939	.061	.743	.750
17	.844	.055	.739	.621
19	.779	.054	.709	.629
20	.952	.055	.790	.572
21	1.023	0.54	.833	.484
23	.952	.055	.796	.552
27	.838	.049	.787	.452
28	.838	.054	.746	.589
30	.841	.057	.722	.681
32	.875	.051	.791	.482
34	1.000	/	.878	.313
35	.869	.062	.693	.856
37	1.021	.057	.807	.588
38	.800	.057	.700	.698
OSEM− subscale
1	1.055	.091	.671	.587
2	.819	.090	.543	.696
5	1.135	.092	.715	.533
7	1.027	.089	.670	.558
8	1.140	.084	.772	.380
9	1.045	.082	.734	.406
11	.789	.063	.722	.248
13	.878	.079	.652	.451
16	1.149	.082	.796	.330
18	.710	.074	.571	.450
22	.697	.077	.539	.514
24	.689	.073	.558	.453
25	.775	.072	.627	.401
26	1.047	.086	.706	.477
29	.933	.077	.705	.381
33	.831	.090	.546	.704
36	1.000	/	.759	.319
39	.761	.067	.664	.317
40	.990	.087	.665	.533
41	.986	.082	.693	.454

Study II: validity in predicting ambulatory blood pressure

Having completed construction and preliminary examination of the OSEM in Study I, Study II was conducted in order to re-examine the measure’s psychometrics in a new sample, as well as to examine its validity in predicting ABP. Study II also investigated the moderating role of online-only social ties and marital status on ambulatory cardiovascular outcomes, given their importance as sociodemographic considerations.

Method

Participants and design

Analyses of ABP involved multi-level models, given that a one-day assessment typically yields approximately 18 valid readings.⁵² Based on a sample size of 140 participants, power for these analyses were estimated a priori using G*Power with the sample size appropriate for the nested structure (i.e., N_effective).⁸⁶ Assuming an association between ABP measurements of .5, power for Study II primary aims was estimated to be >.99.

One participant took part in the ambulatory protocol but did not provide any other data, leaving an overall sample of N = 139, comprising 95 females (68.4%) and 44 males. (As no participants reported a nonbinary gender, this variable was coded as male = 1, female = 2.) Participants ranged from 18 to 55 years of age (mean = 21.97 years, SD = 5.85). The majority of the sample identified as white (64.8%), single/never married (82%), and heterosexual (84.8%). Most had attended some college, but were not yet college graduates (87.1%). Participants were recruited for this study from undergraduate psychology courses and through posted advertisements across the community. Individuals were required to be at least 18 years of age, free of serious health complications (e.g., cancer),⁸⁷ to have no existing hypertension, cardiovascular medication use, history of disease with cardiovascular component (e.g., diabetes), or recent psychological disorder (e.g. major depressive disorder), and to consume no more than 10 alcoholic beverages weekly.⁷⁰ Standard demographics and health behaviors were also assessed to allow for statistical control of these factors during analysis. Participants were compensated with course credit or $40.

Procedure

Three days prior to the ambulatory protocol, participants were provided with a number of demographic, health, and psychological questionnaires to be completed online in advance of their laboratory appointment. In the laboratory on day 1 of the ambulatory protocol, trained researchers demonstrated proper placement of an ABP monitor and observed as participants repeated the process, continuing until the participant had correctly placed the monitor themselves. Blood pressure readings taken at this time also served as baseline screenings to further ensure the participant did not exhibit irregular or clinically concerning values for blood pressure and heart rate. Consistent with previous work, an interval scheduling system initiated a random reading once every 30 min during the ambulatory protocol over the course of a “typical day.”⁵⁴ Participants were instructed to, immediately after the readings, complete a brief diary entry to provide details about the circumstances under which the readings were taken. Participants were able to complete the diary entries online via smartphones or using paper copies. Participants were instructed to wear the monitor throughout the day, removing it before sleep, and to again wear the monitor the following day until their return to the lab. During the appointment on day 2, participants returned the ABP monitor (and physical diary sheets, if applicable) to the lab and were debriefed by study staff.

Measures

As in Study I, the ISEL, TENSE, TIPI, and demographics questionnaire were also administered to participants in Study II.

Online Social Experiences Measure (OSEM)

The scale developed and cross-validated in Study I assessed social support and negativity from online sources. One item was also included as an attention check; this item was not set to load on either subscale. The final OSEM scale (see Supplemental Material) alternated between the 20 OSEM+ items and 20 OSEM− items in a randomly determined sequence that was manually adjusted in order to preclude more than three consecutive items from the same subscale.

Ambulatory blood pressure (ABP)

The Oscar 2 (SunTech Medical, Morrisville, NC) was used to obtain ambulatory readings of systolic and diastolic blood pressure (SBP/DBP) and heart rate (HR). The Oscar 2 is specifically intended for ambulatory assessment and has been well validated via intra-arterial blood pressure measurement. The device assesses blood pressure noninvasively using oscillometry with step deflation by means of an occluding cuff placed on the participant’s upper, nondominant arm. Once an assessment is initiated, the results are stored automatically without necessitating action on the part of the participant. The Oscar 2 can store and timestamp 250 such readings before data retrieval is required, rendering it more than suitable for a one-day ambulatory protocol.

The Oscar 2 tags problematic readings with trouble codes to indicate difficulties in generating reliable ambulatory assessments. ABP readings with codes 1 and 2 (1001 total measurements indicating measurement timeout, artifact/erratic signal, no signal, cuff leak, or measurement canceled) were discarded, consistent with prior research.^70,88 Data, which were also checked for outliers against the criteria outlined by Marler et al.,⁸⁹ were discarded if SBP > 250 mmHg (n = 0 readings), SBP < 70 mmHg (n = 1), DBP < 45 mmHg (n = 17), DBP > 150 mmHg (n = 8), SBP/DBP > 3 (n = 1), SBP/DBP < (1.065 + [.00125 × DBP]) (n = 2), HR < 40 (n = 1), or HR > 200 (n = 0). This resulted in a total of 3561 valid ABP readings across 134 individuals (five additional participants provided survey data, but did not complete the ambulatory protocol).

Ambulatory diary

Participants were instructed to complete a brief (1–2 min) diary entry following each ABP assessment. Designed for ease of completion and compliance maximization, the diary obtained general information such as the date and time of the reading (automatically recorded by the electronic version of the diary) to facilitate matching with ABP readings, and collected information on basic variables with the potential to influence cardiovascular function, such as posture (sitting, standing, lying down), activity level (1 = no activity, 4 = strenuous activity), location (work, home, etc.), talking (yes, no), relative temperature (too hot, comfortable, too cold), recent consumption of nicotine, caffeine, alcohol, or food, and recent exercise.^88,90

Study II hypotheses

1. Convergent and discriminant validity: it was predicted that the OSEM+ (positivity) subscale would show a moderate correlation with the ISEL. Correlations with the TENSE and the TIPI extraversion and emotional stability subscales were expected to be weaker. The association between the OSEM− (negativity) subscale and the TENSE was predicted to be moderate and stronger than associations between the negativity subscale and the ISEL or TIPI subscales.

2. The positivity subscale was expected to predict lower ABP.

3. The negativity subscale was expected to predict higher ABP.

4. Consistent with the separability of positive and negative aspects of relationships, the subscales were expected to be independent predictors of ABP when both were entered in the same model.

5. Ancillary hypotheses predicted differential ABP as a function of online support and participant sociodemographic factors. Specifically, marital status was identified as a potentially important factor in the association between online social support and ABP. It was also hypothesized that online network characteristics play an important role. Specifically, it was hypothesized that OSEM+ scores may be more predictive of favorable ABP among participants with larger numbers of uniquely online contacts (i.e., individuals to whom the participants are connected online, but have not met offline, reflecting a source of social support nonredundant with offline networks).

Study II Results

Convergent and discriminant validity

As expected, the OSEM+ subscale was moderately correlated with ISEL scores, r(129) = .42, p < .0001, while showing a weaker inverse association with the TENSE, r(129) = −.20 p = .0207, and also showed smaller associations with extraversion, r(129) = .27, p = .0020, and emotional stability, r(129) = .18, p = .0427 (see Table 3 for correlations and means for main study variables). Also consistent with hypotheses, the OSEM− subscale was more strongly correlated with the TENSE, r(129) = .56, p < .0001, and had a smaller inverse association with the ISEL, r(129) = −.31, p = .0003, and emotional stability, r = −.39, p < .0001, as well as smaller inverse associations with extraversion, r = −.07, p = .4237. The inter-factor correlation was not significant, r = .08, p = .36. Internal consistency was high for both subscales (OSEM+ α = .95; OSEM− α = .93).

Table 3.

Correlations and means for study variables (n = 131).

Variable	1	2	3	4	5	6	7	8
1. ISEL	–
2. TENSE	−.36***	–
3. OSEM+ subscale	.42***	−.20*	–
4. OSEM- Subscale	−.31***	.56***	.05	–
5. TIPI extraversion	.30***	−.21*	.27**	−.07	–
6. TIPI emotional stability	.33***	−.30***	0.18*	−.39***	.35***	–
7. Age	.11	0.04	.15	.01	.11	-.03	–
8. Gender (male = 1; female = 2)	.20	.03	.07	.004	.008	−.16	.13	–
Mean	28.36	.95	66.76	36.49	3.98	4.46	21.76	68.4%
SD	5.87	.73	16.51	13.78	1.53	1.56	5.80	Female

^*p < 0.05. **p < 0.01. ***p < 0.001.

OSEM predictive validity: ABP

Random regression models were constructed using SAS (v.9.4; SAS Institute) Proc Mixed, with time as a repeated measure. An autoregressive covariance structure was specified using the type = ar(1) option, in order to account for measurement occasion dependency in ABP readings. Predictors were centered at their grand mean before entry into models.⁹¹

In base models controlling for participant age and gender, the OSEM+ subscale was not predictive of ABP or heart rate (p-values > .47). Separate models using the OSEM- subscale, however, found that increased online social negativity was associated with greater ABP assessments of DBP (b = .06, SE = .03, p = .0421, 95% CI [.0022, .1172]), and HR (b = .07, SE = .03, p = .0324, 95% CI [.0056, .1274]), but not SBP (b = .08, SE = .05, p = .1203, 95% CI [−.0218, .1860]); see Table 4. This pattern remained stable in models with the OSEM+ and OSEM− subscales were included together as simultaneous predictors; higher OSEM− scores were still predictive of higher DBP (b = .06, SE = .03, p = .0435, 95% CI [.0018, .1172]), and HR (b = .07, SE = .03, p = .0362, 95% CI [.0043, .1264]).

Table 4.

Results for random regression models predicting ambulatory cardiovascular outcomes.

	Systolic blood pressure
	b	SE	p	95% CI
Age	.4284	.1239	.0007	.1834, .6735
Gender	−11.75	1.54	<.0001	−14.81, −8.70
OSEM+	−.01459	.04381	.7396	−.1013, .07029
Age	.4279	.1233	.0007	.1839, .6720
Gender	−11.84	1.5380	<.0001	−14.89, −8.80
OSEM−	.08213	.05252	.1203	−.02179, .1860
	Diastolic blood pressure
Age	.1776	.06805	.0102	.04291, .3122
Gender	−1.695	.8491	.0481	−3.38, −.015
OSEM+	.00614	.02430	.8009	−.04194, −.05421
Age	.1798	.06771	.0089	.04587, .3138
Gender	−1.726	.8454	.0433	−3.40, −.053
OSEM−	.05970	.02907	.0421	.002176, .1172
	Heart rate
Age	−.1200	.07218	.0989	−.2628, .02283
Gender	4.385	.8999	<.0001	2.60, 6.17
OSEM+	.0184	.02569	.4749	−.03242, .06924
Age	−.1154	.07176	.1103	−.2574, .02661
Gender	4.367	.8954	<.0001	2.60, 6.14
OSEM-	.0666	.03076	.0324	.005687, .1274

In addition to age and gender, more conservative analyses also controlled for the following variables on the basis of previous ABP work: posture (lying-sitting, sitting-standing), relative temperature (comfortable-cold, comfortable-hot), activity level at the time of the ABP reading, talking at the time of the reading, and consumption of food or alcohol in the interval since the previous reading. The predictive power of the OSEM− subscale was attenuated in these models for DBP (b = .05, SE = .03, p = .1394, 95% CI [−.0160, .1135]), but remained relatively robust for HR (b = .07, SE = .03, p = .0417, 95% CI [.0027, .1366]). A similar pattern emerged when controlling for TENSE scores in predicting DBP (b = .03, SE = .03, p = .3467, 95% CI [−.0347, .0981]) and HR (b = .07, SE = .04, p = .0476, 95% CI [−1.5118, −1.1331]).

Sociodemographic moderators of OSEM+ scores in predicting ABP

Additional models controlling for age, gender, and ISEL scores were constructed to examine interaction effects. Models including marital status revealed significant interactions with OSEM+ scores. As shown in Figures 1 and 2, results suggested married participants showed lower levels of SBP (b = −.46, SE = .14, p = .0012, 95% CI [−.7368, −.1859]) and DBP (b = −.20, SE = .08, p = .0105, 95% CI [−.3578, −.0484]) as OSEM+ scores increased, a pattern that was not evident for participants who had never been married. The interaction of OSEM+ scores and marital status did not predict HR (b = −.03, SE = .08, p = .7652, 95% CI [−.1908, .1407]). In full models (controlling for age, gender, ISEL scores, and additionally for posture [lying-sitting, sitting-standing], relative temperature [comfortable-cold, comfortable-hot], activity level at the time of the reading, talking at the time of the reading, and consumption of food or alcohol in the interval since the previous reading) these interaction effects remained predictive for both SBP (b = −.37, SE = .17, p = .0283, 95% CI [−.7044, −.0404]) and DBP (b = −.21, SE = .10, p = .0350, 95% CI [−.3963, −.0147]).

Figure 2.

Diastolic blood pressure decreases for married participants with increasing OSEM+ scores.

Models controlling for age, gender, and participants’ number of online social ties examined the role of online only social ties—that is, individuals to whom participants are connected online, but have not met offline. For individuals with relatively high numbers of online only social ties, ambulatory measures were more favorable as OSEM+ scores increased, an effect evident for SBP (b = −.07, SE = .03, p = .0331, 95% CI [−.1342, −.0057]), DBP (b = −.06, SE = .02, p = .0013, 95% CI [−.0951, −.0238]), and HR (b = −.04, SE = .02, p = .0302, 95% CI [−.0796, −.0041]; see Figures 3–5, which contrast individuals with fewer than 50 online only social ties with individuals who have 101–175 such ties). In full models additionally controlling for posture (lying-sitting, sitting-standing), relative temperature (comfortable-cold, comfortable-hot), activity level at the time of the reading, talking at the time of the reading, and consumption of food or alcohol in the interval since the previous reading, and ISEL scores, the pattern of results remained similar for SBP (b = −.07, SE = .04, p = .0720, 95% CI [−.1459, .0063]), DBP (b = −.08, SE = .02, p = .0004, 95% CI [−.1208, −.0363]), and HR (b = −.04, SE = .02, p = .0526, 95% CI [−.0899, .000510]).

Figure 3.

Systolic blood pressure decreases for participants with many online only social ties with increasing

Figure 4.

Diastolic blood pressure decreases for participants with many online only social ties with increasing OSEM+ scores.

Figure 5.

Heart rate decreases for participants with many online only social ties with increasing OSEM+ scores.

Discussion

This set of studies aimed to develop and validate an instrument for the measurement of social support and social negativity in online contexts. As hypothesized, and consistent with separability of social positivity and negativity, a two-factor structure emerged and was replicated, suggesting the OSEM captured the target dimensions of the online experience. In Study II, results suggested the OSEM has predictive validity; specifically, greater online social negativity, as assessed by the OSEM− subscale, was predictive of increased diastolic blood pressure as well as heart rate in a one-day ambulatory assessment. OSEM+ scores were predictive of cardiovascular outcomes for some types of participants. In factor analysis, oblique rotations were used to allow the OSEM+ and OSEM− factors to correlate freely. Despite this, inter-factor correlations were not significant, suggesting the measure uniquely captures online social positivity and negativity. Across Studies I and II, results were also consistent with hypotheses in terms of suggesting convergent and discriminant validity. Notably, correlations used to examine validity were remarkably stable across the Study I subgroups and the Study II sample.

As predicted, greater online social negativity was suggestive of less favorable ambulatory cardiovascular assessments. This is consistent with work linking negativity in offline social relationships to a variety of adverse health outcomes.^11,12,92 The results of Study II suggest that such social negativity in online contexts also has deleterious associations with health. Previous work has suggested increased technology-mediated communication or online social negativity are associated with a variety of undesirable psychological outcomes such as stress,⁹³ depressive symptoms and anxiety,^94,95 and decreased mood.⁹⁶ The findings of the present studies harmonize with this work on psychological wellbeing and offer evidence that online social negativity may have unfavorable ramifications for physical health as well through daily ambulatory cardiovascular function.

In more restrictive analyses controlling for age, gender, posture, relative temperature, activity level, and recent consumption of food or alcohol, the OSEM− subscale remained a significant predictor of heart rate, but not of diastolic blood pressure. Several possibilities arise from this pattern of results. First, it may be that the social factors under consideration are more predictive of HR than of DBP. This would be somewhat consistent with a meta-analysis of social support in laboratory stress research, the findings of which included a larger average effect size for heart rate versus diastolic blood pressure.⁹⁷ Secondly, it may be that these more conservative analyses represent a case of statistical overcontrol to some degree.⁹⁸ The possibility of systematic unmatching on relevant variables is of particular interest given previous work suggestive of links between some such variables and social negativity.⁹⁹ For example, social rejection has been linked to increased alcohol consumption,¹⁰⁰ unhealthy eating,¹⁰¹ and depression,¹⁰² which is in turn linked to lower activity. Unintentional unmatching on such variables through overcontrol may distort results. Finally, of course it may be that the association between online social negativity and ambulatory cardiovascular outcomes is attributable to or mediated by one or more of these control variables. Additional research will be needed to investigate these various possibilities.

Contrary to predictions, greater online social positivity was not predictive of more favorable cardiovascular assessments in the Study II primary analyses of main effects. This is somewhat in line with other work using ABP to examine interpersonal relationships and health. For example, Holt-Lunstad et al. found that interactions with supportive network members were not associated with lower SBP, DBP, or HR.⁷⁰ It may be that positive online interactions or experiences have more marked benefits depending on the population in question. Participants in Study II were generally healthy and reported relatively high perceptions of offline social support. More geographically isolated or otherwise vulnerable individuals may be more likely to show benefits relating to positive online experiences. Similarly, Study II participants were fairly young on average. Laboratory work indicates the links between social support and resting blood pressure may strengthen with age,^103,104 suggesting OSEM may be more able to predict SBP or show larger effect sizes for DBP in samples including older adults. Another potential explanation for this pattern of results is suggested by prior work in which negative aspects of relationships seemed to prevent individuals from experiencing the benefits of positive aspects of those same relationships.⁵⁴ Indeed, research on the negativity bias provides evidence that, compared to positivity, negative information provokes larger and faster physiological responses;^105,106 this is also manifest in work examining social negativity and physical health outcomes.¹⁰ Thus, it may be that the effects of online social negativity are more easily detectible or more influential on ABP than are those of online social support. Consistent with this, a study assessing social media use and other forms of technology-mediated communication found that both positivity and negativity were respectively associated with fewer or more depressive symptoms at a three-week follow-up.⁹⁴ While both positive and negative interactions were predictive of depressive symptoms, the magnitude of these associations was noticeably larger for negative interactions (average r = .48 vs. −.33 for positive interactions).

However, largely consistent with hypotheses, interaction analyses were suggestive of an association between online social positivity and more favorable cardiovascular readings among certain segments of the present sample. For example, results indicated that married participants showed lower SBP and DBP as OSEM+ scores increased, a pattern that did not emerge for participants who had never been married. These results may indicate that online social positivity may be more beneficial for certain types of individuals. Consistent with this, analyses also indicated that higher OSEM+ scores were predictive of more favorable cardiovascular readings for individuals with greater numbers of online only social ties, in contrast to participants whose online and offline social networks were more redundant. This may indicate that online positivity is more beneficial to the extent that it reflects a unique source of social support, rather than simply overlapping with offline sources. The state in which this research was conducted shows the region’s highest percentage of residents born in-state,¹⁰⁷ and has been ranked as having the highest levels of social support in the US.¹⁰⁸ These factors may mean that on average, individuals in the Study II sample were unusually embedded in supportive offline social networks (indeed, ISEL-assessed offline support was 10 percentage points higher in the Study II sample vs. both Study I subgroups), limiting the ability of the OSEM+ subscale to predict ambulatory values for all participants. These issues should be explored in greater detail in future work.

Though the results of the exploratory factor analysis in Study I clearly suggested a two-factor structure underlying the initial OSEM item pool, the confirmatory model yielded somewhat mixed fit indices. For example, while RMSEA was acceptable, the chi-square test was significant. Despite its enduring popularity, the chi-square test is considerably limited as a measure of model fit by the required assumptions, as well as issues with both small and large samples.¹⁰⁹ RMSEA is generally agreed to be a more informative fit index.^109,110 Further evidence may be provided by future work in additional work with varying sample size and characteristics, as well as various study designs.

The emergence of a two-factor structure in the present studies is consistent with research in offline social networks indicating that positive and negative aspects of relationships are separable dimensions rather than opposite ends of the same continuum.^15,54,70 By providing a means of measuring social support and social negativity unique to online contexts, the OSEM may help expand and reconcile conflicting findings on the positive and negative correlates of interaction with online social networks. These data agree with an emerging body of work suggesting that in terms of beneficial outcomes, the quality of online social interactions and networks may be more important than the frequency with which they are transacted or accessed.⁹⁴ The findings of the present studies are consistent with review work on the distinctions between active and passive use of social media,^22,111 and the related, elegantly-articulated interpersonal-connection-behaviors framework.¹¹² This theory suggests that SNSs are beneficial to users when they facilitate meaningful connection, but are harmful when they contribute to unfavorable social comparisons or social isolation. Examination of the OSEM items suggests the subscales largely align with this distinction, with OSEM+ items generally assessing behaviors reflecting formation or facilitation of social connection, while OSEM− items largely relate to social rejection or exclusion. The OSEM may facilitate use of the interpersonal-connection-behaviors framework as a theoretical foundation in future work.

The prediction of cardiovascular outcomes from OSEM+ scores for some types of participants is intriguing as the literature examining technology-mediated communication and resulting coverage in the popular press move beyond overly-simplistic questions such as “Is social media use harmful or helpful?” Consistent with the present study and with previous work indicating the importance of online sources for individuals with less offline support,⁶⁴ it is conceivable that online support may be especially important to individuals living in socially isolating circumstances. For example, persons in rural locations or with chronic, rare, or stigmatized medical conditions may particularly benefit from online social support and positivity, which may not be available from offline sources. Family care partners of individuals with such conditions may also benefit from online support.⁶⁵ For future researchers considering use of the OSEM in their own work, the following principal is emphasized: Consideration of the broader social context—both online and offline—is critical to understanding the psychological or physiological implications of any specific form of technology-mediated communication.

The present studies address several recommendations made by Nick and colleagues.⁴⁰ For example, the authors stated testing the ability of online social experiences to predict a broader variety of outcomes is a critical step. Prediction of ABP in the present research provides some such data. The authors also called for testing the moderation of the effects of online social support by individual differences, another contribution of the present study.

A major innovation introduced by the OSEM is the simultaneous assessment of both online positivity and negativity. Consideration of these separable dimensions together is key to understanding associations between social processes and health, as these dimensions have been shown to have individual and interactive links to health and wellbeing.^{12,15,54,113,114} In addition to helping explore the implications of online experiences for wellbeing, the OSEM may be helpful in informing future intervention work aimed at increasing positivity and decreasing negativity in online experiences, or in replacing relatively unhealthy social media use with more adaptive behaviors.

Future research can build on the findings of the present studies by addressing limitations. For example, the present research did not directly examine the possible health effects of interaction with social media platforms in real time. Future work could monitor the online activities of participants and use OSEM items to characterize participant experiences and contemporaneous physiological data. The ABP data collected in Study II were observational; future work could experimentally manipulate the online actions of participants or curate specific social media feeds to reflect varying mixes of online social support and negativity. The present studies examined online networks at an aggregate level; future work could use the OSEM to examine specific social network platforms or individual social ties, an important consideration given that previous work suggests such specificity may have implications for the detection of health-relevant outcomes.¹² Importantly, future work using longitudinal designs may help indicate to what extent individuals’ reports of online social positivity and negativity are stable or shifting over time. Changes over time may occur in response to a number of factors, such as the provision of new affordances or the elimination of existing ones on various online social networks and other technology-mediated communication platforms. Changes in an individual’s life circumstances and offline social relationships may also alter perceptions of or participation in online social interaction and involvement.

Conclusions

The present studies suggest the Online Social Experiences Measure (OSEM) captures both positive and negative dimensions of social interactions in online contexts, and begin to explore the OSEM’s validity in predicting health-relevant outcomes such as ambulatory cardiovascular functioning. Additionally, the results of Study II suggest unfavorable health outcomes may be associated with online social negativity, and that online social positivity may be beneficial for individuals in certain circumstances. While much remains unknown regarding the specific ways in which online social positivity and negativity relate to psychological and physical wellbeing, the development of this instrument gives researchers an additional tool with which to continue investigation of technology-mediated social phenomena.

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Contributorship

RGKdG and AEH researched literature and all authors contributed to conception of the study. RGKdG and BNU were involved in protocol development. RGKdG oversaw gaining ethical approval, participant recruitment, and data collection. RGKdG, BNU, and BRWB were involved in data analysis or statistical approach. RGKdG drafted the manuscript. All authors reviewed and edited the manuscript and approved its final version.

Ethical approval

The Institutional Review Board of the University of Utah approved this study (IRB #s: 00089349 & 00089350).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors gratefully acknowledge generous support from the Consortium for Families & Health Research at the University of Utah and the B. Jack White Memorial Award.

Guarantor

RGKdG

Note

This research and development of the manuscript were conducted using renewable energy and carbon offsets.

Peer review

This manuscript was reviewed by reviewers who has chosen to remain anonymous.

Supplemental material

Supplemental material for this article is available online.