Feasibility, usability, and acceptability of personalized web-based assessment of social network and daily social support interactions over time

Abstract

Purpose

The purpose of this study was to test the feasibility, usability, and acceptability of implementing a web-based method for collecting social network and longitudinal daily interaction data from cancer survivors and their caregivers.

Methods

Young adult and sexual/gender minority cancer survivors and their informal caregivers were recruited as dyads. Feasibility data, including enrollment and retention, were captured. Individual social network data were collected at baseline and used to individualize daily electronically delivered surveys assessing characteristics of daily social support-related interactions with identified network members for 14 days. Follow-up questionnaires assessing usability and exit interviews assessing acceptability were completed at the end of the 2-week study period.

Results

Fourteen survivor-caregiver dyads (28 individual participants) were enrolled and completed all baseline and final measures. Participants completed 85.2% of daily diary reports and reported excellent usability ratings. Acceptability was also high. In qualitative interviews, participants reported enjoying the daily reflection on social support facilitated by our methods.

Conclusions

Our method has been shown to be highly feasible, usable, and acceptable.

Implications for Cancer Survivors

Developing better data collection tools can lead to better understanding of the social support cancer survivors and their caregivers receive, and how the social network structure facilitates or creates barriers to accessing this support.

Social support is generally viewed as a interpersonal process that occurs within the context of one’s social network. In cancer research, there is strong evidence that social support plays a critical, protective role in cancer survivors and informal caregiver well-being [13, 37, 41, 42, 44]. Previous research also indicates that survivor and caregiver cancer experiences and outcomes are linked [23, 27, 39]. For example, caregiver social support is linked to patient health, and vice versa [26]. A much smaller body of literature focuses on the structure of social networks for cancer survivors and their caregivers and the characteristics of these networks that facilitate the exchange of support [28, 29]. Differences in social networks and interaction patterns can lead to differences in how social support is accessed and activated, and its effectiveness in buffering stress [2, 30, 46]. Some work indicates that larger social networks can protect against mortality in cancer survivors [31] and ease burden for caregivers [18]. Size of network alone, however, is unlikely to explain the buffering influence of social support independent of network membership and quality and quantity of interactions [25].

Social network assessment measures used in oncology populations are secondary analyses of retrospective data, or prospective studies that collect information about support at baseline only [16, 25] and tend to rely on general measures of social integration or participation (e.g., marital status, volunteering, church attendance) and/or static measures of functional social support (e.g., global ratings of perceived support) [19]. While informative, these studies are limited by a lack of robust egocentric network data that are necessary for drawing conclusions about the structure and compositions of real-world support networks, including information about relationships and ties between personal support network members [16]. Members of survivors’ and caregivers’ close personal support networks may fulfill different roles and provide different kinds of support at varying times [3, 6, 34, 43]; this level of detail is rarely addressed or captured systematically and prospectively. Current models of social network data collection fail to capture sufficient data to accurately model the cancer survivor-caregiver experience of acquiring support.

Many data collection methods are also structured in ways that reproduce culturally normative assumptions (e.g., traditional heteronormative spousal relationships centered in nuclear, biological family structures). This can leave out particularly vulnerable populations, such as young adult cancer survivors and/or members of sexual and gender minority groups (e.g., LGBTQ + including gay, lesbian, bisexual, transgender, nonbinary), both underrepresented groups in social support research [35]. Young adults often have increased mobility and thus their social networks may be less physically proximal, more diffuse, and support may come through new channels, such as social media [4, 45]. Differences in social network structure may be even more apparent among people who are LGBTQ + , who may be more likely to identify a “chosen” or “logical” family rather than biological/legal family [10, 15, 17]. Traditional social network and social support data collection methods can fail to adequately capture these flexible and adaptive kinship and social networks. Kinds of support needed, timing and form, medium, and fit between individuals’ preferences for how they want or need support are all areas that need closer examination.

Study purpose

The purpose of our pilot study was to design and test the feasibility, acceptability, and usability of a web-based, person-centered method for collecting and integrating social network and daily support interaction data. This approach integrated the collection and use of social network data with online daily longitudinal, repeated surveys delivered via text or email. Our overall aim was to develop data collection methods that will reduce barriers to participation and promote more inclusive study design, particularly for groups who are under-represented in cancer and cancer caregiving research including young adults (YA, aged 18–39) and people who are LGBTQ + .

Methods

Data collection tool and methodology development

Our study database was developed in REDCap [20, 21], a secure online data capture and management software, making extensive use of item branching logic to create a participant-facing survey that would include both baseline, daily response, and follow-up survey measures while allowing for personalization via participants’ specific social network data, as explained below. We delivered daily survey links to each participant by text or email on a predetermined schedule. The data collection tool featured inclusive survey design (e.g., language, having range of options in forced-choice items, allowing for free text responses, attention to participant burden), and more broadly, the study offered opportunities for participant feedback and reflection and compensation for time and contribution. We also sought feedback on this approach—and particularly on reducing barriers to research participation and increasing inclusive practices—from LGBTQ + cancer survivors and caregivers, national leaders, and stakeholders of advocacy groups (e.g., oncology clinicians, advocates) in oncology care and services, cancer research, survivor, and caregiver advocacy. (Data on this are available elsewhere: [5].)

Study design

We conducted an iterative pilot study in two US National Cancer Institute-designated Comprehensive Cancer Centers, one in the Southeast and one in the Intermountain West. We recruited four survivor-caregiver dyads at one site (n = 8 individuals) to complete baseline questionnaires, daily assessments of social interactions for 2 weeks, and follow-up questionnaires at 2 weeks, including a semi-structured interview designed to obtain qualitative feedback. After the initial 4 dyads, we reviewed our methods and findings. We made minor adaptations, including changing some of our questionnaire instructions, and recruited a second wave of 10 survivor-caregiver dyads across both sites for a total of 14 dyads (28 individuals). We intended to recruit eight dyads at both sites, but we closed our study early due to the COVID-19 pandemic.

Data for all participants across both sites are presented here. All study activities were conducted with approval by the Institutional Review Board at each site (exempt determination). Verbal informed consent was obtained from all individual participants.

Study participants

Potential survivor participants were required to have a cancer diagnosis within the previous 3 years (YA) or 5 years (SGM) and receive some form of treatment, including follow-up care or ongoing monitoring at one of the participating cancer centers. Eligible survivors had to identify a caregiver/primary support person who also agreed to participate in the study; this was most often a spouse/partner or a parent. All participants were required to each be over age 18, have the ability to speak and read English, and have a device capable of internet access and receiving email or text messaging.

Since our goal was to collect feasibility data and test our methods with a targeted sample of cancer survivors and caregivers (YA and SGM), each site had different additional inclusion criteria. One site targeted recruitment of survivors between the ages of 23–38 years old (“Millennials”) and their caregivers, while the second site recruited LGBTQ + survivors aged 18 + and their caregivers.

Recruitment

The first wave of recruitment occurred between August and November 2019; the second wave occurred between December 2019 and May 2020. Potential YA participants were identified by the clinic staff at one site; potential LGBTQ + participants were recruited through outreach to community groups, social media, and chain referral sampling. Interested cancer survivors were contacted by study staff to explain the study, determine interest, formally screen for eligibility, and to obtain contact information for caregivers. Caregivers of interested survivors were similarly contacted to determine interest and eligibility and obtain consent.

Procedures

Recruitment and enrollment data were recorded. Demographic data and a short collection of self-report and psychosocial assessments were assessed at baseline using REDCap. Measures were repeated at the conclusion of the study. As the focus of this present report is on the feasibility of our social network and daily support data collection method, data from these psychosocial self-report measures are not reported here.

Measures

Baseline measures

To capture social network characteristics and interaction with social network members, each participant individually participated in a brief interview (10–15 min) at baseline. Using an interactive web-based Social Support Network mapping tool (https://ssnm.ctl.columbia.edu/map/about/) as a guide, participants were asked to identify up to 10 alters, using first names, nicknames, or initials. Alters are individuals who participants perceived as sources of support related to their cancer and/or caregiving experience (e.g., relatives, friends, co-workers, case managers). Participants were asked to provide social network analysis information about each alter, including demographics, relationship to the participant, whether the alters knew each other, strength of ties between the participant and network members, and the types of support each alter generally provided the participant (instrumental, informational, emotional).

Daily social interaction surveys

Once the baseline questionnaire and social network interviews were completed by both survivor and caregiver dyad members, each participant was sent daily links by text or email to a brief online daily survey for 14 days.

Each day, participants were presented with a personalized list of the support network members they named at baseline and were asked who they interacted with in the last 24 h. For each person indicated, the following information was assessed: the purpose of the interaction, the mode of interaction (e.g., in-person, phone, email, text, social media), whether the interaction was supportive (yes/no), a rating of the level of supportiveness, and the type(s) of support represented (informational, instrumental, emotional, appraisal). Participants were also able to add up to five additional interactions with people not originally named as network members. Participants were also provided a free text option to add any other support-related experiences or thoughts.

Assessment of usability and acceptability

After 14 days, participants individually completed the 10-item System-Usability Scale (SUS) [9, 36] assessing the usability of daily surveys. Two additional items were added to assess overall satisfaction with the system and perceived relevance of the daily items to participants’ support experiences. We conducted semi-structured interviews with each individual participant to gather acceptability feedback, including how much they liked their experiences with the data collection method, suggestions for improvement, and other information they wished to share about their support experiences. Interviews were audio-recorded and transcribed verbatim.

Analysis

Descriptive statistics were used to describe study sample demographics. Recruitment and enrollment numbers for all participants and completion rate of daily surveys were calculated to assess feasibility. Descriptive statistics were used on System Usability Scale scores to assess usability. Descriptive qualitative content analysis summarized interview data relevant to acceptability.

Results

Fourteen survivor-caregiver dyads (28 individual participants) were enrolled and completed all baseline and final measures. Eight dyads were recruited at one site (YA cancer survivors) and six dyads at another site (LGBTQ + cancer survivors). Data are merged here as all participants completed the same research procedure. Table 1 presents participant demographics.

Table 1.

Demographics

		All	Survivor	Caregiver
		Yrs (range)	Yrs (range)	Yrs (range)
	Age	40.7 (21–81)	33.2 (21–79)	48.3 (24–81)
	Length of relationship	–	16.4 (2–36)	–
	Time since diagnosis	–	4.0 (1–16)	–
Relationship		N (%)	n (%)	n (%)
	In-law	2 (7.1)	1 (7.1)	1 (7.1)
	Parent	4 (14.3)	1 (7.1)	3 (21.4)
	Spouse/partner	16 (57.1)	8 (57.1)	8 (57.1)
	Child	4 (14.3)	3 (21.4)	1 (7.1)
	Sibling	2 (7.1)	1 (7.1)	1 (7.1)
Gender
	Male	8 (28.6)	6 (42.9)	2 (14.3)
	Female	19 (67.9)	8 (57.1)	11 (78.6)
	Non-binary	1 (3.6)	0 (0.0)	1 (7.1)
Transgender
	Yes	3 (10.7)	2 (14.3)	1 (7.1)
	No	25 (89.3)	12 (85.7)	13 (92.9)
Sexual orientation
	Heterosexual	17 (60.7)	8 (57.1)	9 (64.3)
	Lesbian/gay	3 (10.7)	1 (7.1)	2 (14.3)
	Bisexual	4 (14.3)	3 (21.4)	1 (7.1)
	Queer	2 (7.1)	1 (7.1)	1 (7.1)
	Pansexual	2 (7.1)	1 (7.1)	1 (7.1)
Race
	Black	4 (14.3)	2 (14.3)	2 (14.3)
	White	24 (85.7)	12 (85.7)	12 (85.7)
Ethnicity
	Hispanic/Latinx	1 (3.6)	0 (0.0)	1 (7.1)
Level of education
	High school	3 (10.7)	2 (14.3)	1 (7.1)
	Some college or vocational school	9 (32.1)	4 (28.6)	5 (35.7)
	College graduate	1 (3.6)	0 (0.0)	1 (7.1)
	Some graduate or professional schooling	4 (14.3)	1 (7.1)	3 (21.4)
	Graduate or professional degree	11 (39.3)	7 (50)	4 (28.6)
Employment*
	Full time	10 (35.7)	5 (35.7)	5 (35.7)
	Medical leave	2 (7.1)	1 (7.1)	1 (7.1)
	Not working for pay	5 (17.9)	4 (28.6)	1 (7.1)
	Student (full or part-time)	5 (17.9)	3 (21.4)	2 (14.3)
	Part-time	7 (25)	4 (28.6)	3 (21.4)
	Retired	4 (14.3)	1 (7.1)	3 (21.4)

*participants could select more than one option for this question

Feasibility

Site 1 (Southeastern United States) primarily recruited through clinical referral. While the study included both survivors and caregivers, survivors were targeted for enrollment first, then were asked to nominate a caregiver. In total for site 1, 30 survivors were assessed for eligibility based on clinic referrals, 6 were ineligible, 5 declined participation, and 8 enrolled with a caregiver; all 8 completed the study. Site 2 (Intermountain West Region of the USA) primarily sought participants from the LGBTQ + community and potential participants self-referred to the study from clinic advertising, community outreach, and social media posts advertising the study; as such, eligibility data are not available. One dyad from site 2 completed consent, and dropped from the study prior to completing the baseline data collection. Six other dyads enrolled at site 2 and all completed the study.

Study participants from both sites completed 85.2% of daily diary reports. At site 1, participants completed 11.4 days on average with a range of 5–14, and at site 2, participants completed 12.7 days with a range of 8–14 days. Our study was halted due to institutional restrictions on research recruitment related to the COVID-19 pandemic. However, we were able to continue to collect data from five participants who were already enrolled, eliminating any in-person contact. See Table 2 for additional completion information.

Table 2.

Daily diary completion

	All	Site #1	Site #2
Days completed (n, range)	11.9 (5–14)	11.4 (5–14)	12.7 (8–14)
Percent completion (%, range)	85.2 (35.7–100)	81.3 (35.7–100)	90.55 (51.1–100)
Completed all days	46.4	31.3	66.6
Completed 70%	82.1	75.0	91.7

Usability

The mean total SUS score for survivors was 89.8 (SD = 8.4); the mean total score for caregivers was 78.0 (SD = 15.9). While any score greater than 70 is considered good, a score in the high 70 s indicates excellent usability and in the 90 s superior. Individual item responses are shown in Table 3.

Table 3.

System usability score [9]

	All		*Survivors		Caregivers
	M	(SD)	M	(SD)	M	(SD)
I think that I would like to use this system frequently	3.2	(0.8)	3.4	(0.9)	3.1	(0.7)	Higher scores indicate more usability
I thought the system was easy to use	4.5	(0.8)	4.9	(0.3)	4.1	(0.9)
I found the various functions in this system were well integrated	4.2	(0.8)	4.4	(0.8)	4.1	(0.7)
I would imagine that most people would learn to use this system very quickly	4.5	(0.6)	4.8	(0.4)	4.3	(0.7)
I felt very confident using the system	4.3	(1.0)	4.7	(1.1)	4.0	(0.8)
I found the system unnecessarily complex	1.7	(0.9)	1.2	(0.4)	2.1	(0.9)	Lower scores indicate more usability
I think that I would need the support of a technical person to be able to use this system	1.3	(1.0)	1.0	(0.0)	1.6	(1.3)
I thought there was too much inconsistency in this system	1.4	(0.8)	1.2	(0.4)	1.6	(1.0)
I found the system very cumbersome to use	1.5	(1.1)	1.7	(1.4)	1.4	(0.8)
I needed to learn a lot of things before I could get going with this system	1.2	(0.4)	1.0	(0.0)	1.4	(0.5)
Total SUS score	83.7	(13.9)	89.8	(8.4)	78.0	(15.9)
Satisfaction	4.3	(0.9)	4.6	(0.8)	4.0	(0.9)
Relevance	4.1	(0.7)	4.2	(0.8)	3.9	(0.6)

Total scores are transformed to scale of 100; individual items are raw scales, 1–5

^*SUS scores missing for one survivor

Acceptability

Participants were asked to rate their satisfaction with data collection and whether the tool captured data that was relevant to them using a 1–5 scale. On average, participants rated high levels of satisfaction (M = 4.30, SD = 0.87) and high levels of relevance (M = 4.07, SD = 0.73).

Interview data suggests that most participants enjoyed reflecting on their daily interactions and felt it increased their awareness of both who was in the support network and the type of support they tended to receive from each person. These reflections also served as reminders of where support could be accessed when needed.

I think that it was really helpful and not only did it make me realize that I had a little bit more support than I thought, but helped me realize what type of support. Like friend support versus family support or more empathy than practical support. (Caregiver, 26 y/o)

It’s a very calming moment to just look back and try to think about it practically…I think it’s going to continue to be done in my own head with or without the survey. (Survivor, 24y/o)

Some participants noted that there were differences between how they reported their support at baseline and what that support actually looked like in daily interactions with their network members. Specifically, some participants indicated that family—a traditional staple in social networks for cancer survivors and caregivers—were less important for support than friends or other social network members. Additionally, our tool helped them understand the type of support they were receiving from different sources and how to tap into that support when necessary. Some participants felt it would be better to complete over a longer period than 2 weeks because they are not in continuous contact with all people in their network.

Some people, they are in my support system and they are on my chart, but I don’t contact them a lot, it’s like every couple of weeks I check in on them and so I wasn’t really expecting to get too much communication from them. But I wasn’t really surprised with the amount of communication I had between everyone. (Caregiver, 24y/o)

I think that mostly I was surprised how much of my family wasn’t included on there. That it was mostly friends and just kind of disappointed I can’t lean on my family more. (Caregiver, 26y/o)

In response to questions about what could be improved, some participants felt that some aspects of the daily questionnaires were vague and required clarification. For example, one participant commented that despite being provided definitions and examples, it was difficult to know what kind of interactions were supportive in nature, and where to “draw the line” between supportive and other kinds of interactions, especially within a large network. A few participants commented that they were uncertain about who to include on the daily survey—all of their contacts including coworkers and acquaintances, or just those from their networks. Other participants struggled with defining what was meaningful support within a given interaction or relationship.

It was kind of fun to go through and think back through each person. I didn’t feel like there was a lot of room. I mean obviously I interact with a lot more people through work but not – you know but I wasn’t sure whether to answer [about everyone]. (Survivor, 23y/o)

Discussion

Few studies of cancer survivors and their caregivers capture robust prospective personal social network data, which limits researchers’ ability to understand how social support is exchanged in a real-world context prospectively across time. Additionally, the types of interactions and relationships assessed often fail to include the experiences of more diverse survivors and caregivers, including young adults and people who are LGBTQ + , who may have more flexible social networks that are not as defined by traditional biological or legal definitions of family or kinship [8, 10, 15, 17]. Our team found that our method of social network and social support data collection had high levels of usability, and participants liked the flexibility and inclusivity of our tool, which allowed them to capture their social network interactions in ways that reflected their lived experience.

Our recruitment rates were in line or somewhat higher than other studies targeting survivor-caregiver dyads [7, 22, 40]. While our study did not achieve enrollment goals, we were on track to have completed enrollment in our projected timeframe prior to COVID-19 research recruitment shutdowns. The research methods and data collection strategies themselves are relatively “pandemic-proof,” such that the majority of our participant interactions after initial recruitment were conducted via video-conference or through electronic data collection. These methods are additionally beneficial for young adult and LGBTQ + participants, who may prefer the accessibility of participating in research within their own spaces, on their own time, and without the need to travel. The oldest participants in our sample, who were also LGBTQ + , also noted their appreciation of these features given the time and energy costs of research participation on top of both cancer and caregiving.

In addition to wanting to capture social network data in a more ecological context, our remote data collection processes were also developed in response to feedback from participants and stakeholders prior to the pandemic, suggesting that even in “normal” times, in-person participation [38] often presents logistical challenges and burdens for cancer survivors and their caregivers. Survivors (and the caregivers who accompany them) often have long days of back to back appointments, getting to facilities can be complicated or expensive [32], and immunocompromised survivors may see in-person visits as risky. LGBTQ + participants who are unable to bring their caregiver with them to clinic appointments may feel the loss of having an advocate with them to help them navigate potentially heteronormative encounters with providers [12]. While there is currently discussion about increasing accessibility of clinical trials [33] for cancer treatment, there is also a need to make behavioral and observational research more accessible to cancer populations. Using technological solutions such as video conferencing and online surveys can be important ways to increase this access, especially when paired by research team availability to navigate questions and technical issues. Better outreach helps capture a more robust picture of a broader range of survivor and family caregiver experiences.

The vast majority of our participants reported our study tools were easy to use; the majority of our participants reported that they were comfortable with our data collection tools. National data suggests that an overwhelming majority of Americans are becoming more comfortable with technology [11]; likely, the pandemic will further accelerate this as well, given the increasing technology [14, 24] use in health care, business, education, and even personal relationships. Ensuring that technology is developed using user-centered-design concepts [1]—in which tools are developed with the user and the context in mind, and often in collaboration with key stakeholders—facilitates processes that are intuitive. Because they are often part of the process and can provide feedback, the tools developed in this way are often highly usable and in fact desired by participants.

Finally, a major finding of our work is that participants liked having a tool to log their social network interactions that was inclusive and flexible enough to accommodate their personal networks and capture their natural experience. We purposively chose to recruit young adults and LGBTQ + cancer survivors because they more often have non-traditional social networks compared with the middle-aged, older, heterosexual, and cisgender/gender binary groups that comprise most cancer research participants reported in the literature, and participants noted that they liked being able to include whoever fit, not just parents or romantic partners. In fact, many participants reported that participating was helpful to them, by allowing them to catalogue and reflect on their social networks and social support.

Limitations

Although we targeted young adults and LGBTQ + participants, our sample was limited in racial/ethnic diversity, and, as a pilot study, our sample was relatively small and may be biased due to our recruitment methods. Our recruitment was also impacted by research shutdowns related to the COVID-19 pandemic; however, the majority of our methods have shown to be relatively robust to distance participation by design. Finally, while the flexibility of our methods and measures may allow for inclusivity, some participants reported wanting more guidance on what “counted” as support or a close relationship. It can be difficult to objectively define support, particularly when so much of the benefit is derived from individual perception, but more strict definitions or concrete examples may be useful in future research.

Conclusion and implications

New methods are needed to capture more inclusive and real-world data on social networks of cancer survivors and their caregivers to better understand social support processes. Our method has been shown to be highly usable, and our feasibility and likeability data suggest that diverse groups of cancer survivors and caregivers not only will use these methods, but will like them and find benefit. Data collection techniques that are more accessible, personalized, and available to complete when participants have time may also increase participation, especially for the YA and LGBTQ + groups that are under-represented in cancer research. These methods can be implemented in larger studies to capture important information about survivor interactions with their social support networks, including change over time. Ultimately, our work will inform the development of interventions and clinical services to leverage or augment this support.

Author contributions

Drs. Cloyes, Guo, Jones and Reblin contributed to the study conception and design. Material preparation, data collection and analysis were performed by Rachael McCormick, Kelly Mansfield, Sarah Wawzynski, Dana Ketcher and Karrin Tennant. The first draft of the manuscript was written by Dr. Reblin and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

Dick and Tammy Burton Foundation Pilot Grant Award, University of Utah College of Nursing. Moffitt Cancer Center Adolescent and Young Adult Program with support from Swim Across America and the Bay Area Advisors.

5T32CA090314-16 funded Dr. Ketcher’s effort.

F31NR018987 funded Ms. Wawrzynski’s effort.

K01NR016948 funded Dr. Guo’s effort.

The REDCap application reported in this publication was supported in part by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR001067 for period 5/1/13–3/30/18 and UL1TR002538 for period 4/1/18–2/28/23.

Availability of data and material

Data is available upon reasonable request to the corresponding author.

Code availability

Not Applicable.

Declarations

Ethics approval

This study was determined to be exempt by the Advarra IRB (Moffitt Cancer Center) and the University of Utah IRB under the Revised Common Rule. The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Consent to participate

Informed consent was obtained from all individual participants.

Consent for publication

No identifying information is presented in this publication.

Conflict of interest

The authors declare no competing interests.