Development and Usability Testing of a Web-based Self-management Intervention for Oral Cancer Survivors and their Family Caregivers

Abstract

Oral cancer (OC) survivors experience debilitating side effects that affect their quality of life (QOL) and that of their caregivers. This study aimed to develop and evaluate a dyadic, web-based intervention to improve survivor self-management and survivor/caregiver QOL. A qualitative needs assessment (semi-structured interviews) with 13 OC survivors and 12 caregivers was conducted to discern information and support needs as well as preferences regarding website features and tools. Results using Grounded Theory analysis showed that OC survivors and caregivers: 1) want and need practical advice about managing side effects; 2) want to reach out to other survivors/caregivers for information and support; and, 3) have both overlapping and unique needs and preferences regarding website features. Usability testing (N=6 survivors; 5 caregivers) uncovered problems with the intuitiveness, navigation, and design of the website that were subsequently addressed. Users rated the website favorably on the dimensions of attractiveness, controllability, efficiency, intuitiveness, and learnability, and gave it a total usability score of 80/100. Overall, this study demonstrates that OC survivors and caregivers are interested in using an online program to improve QOL, and that providing tailored website content and features based on the person’s role as survivor or caregiver is important in this population.

INTRODUCTION

Oral cancer (OC) is a type of head and neck cancer that is characterized by malignant tumors of the oral cavity and oropharynx. It accounts for 2% of all cancers diagnosed in the United States (National Institute of Dental and Craniofacial Research 2013). Although the proportion of individuals diagnosed with OC is small, the population is distinct with regard to debilitating side effects that persist long after treatment has ended (Neville and Day 2002). Most patients with OC undergo intensive radiotherapy treatments, either alone or in combination with surgery or chemotherapy. Radiotherapy results in persistent hyposalivation and dry mouth (Rankin et al 2008). If salivary flow is compromised, problems including decreased remineralization of enamel and decreased salivary antimicrobial capacity can occur, resulting in rampant dental caries. Other problems include deglutition, mastication and speech issues, taste alterations, and malnutrition (Silverman 2003). Living with these side effects often takes an emotional toll. OC survivors experience social withdrawal and embarrassment while eating in public (Rankin et al 2008), and are at increased risk for suicide compared to survivors of other cancers (Zeller 2006). Given the devastating impact of radiotherapy on QOL (Langendijk et al 2008, Katsura and Aoki 2015) and the fact that QOL has been associated with survival after OC (Meyer et al 2009), interventions are needed to address the multifaceted QOL needs of this population.

This paper describes the development and formative evaluation (user and usability testing) of an innovative web-based intervention called CARES (Computer Assisted oral cancer REhabilitation and Support). The program is grounded in Self-Determination theory (SDT) which is a motivational theory based on the idea that the fundamental psychological needs for competence, autonomy, and relatedness are essential for promoting the internalization of new behaviors and well-being (Deci and Ryan 2011). Unlike the vast majority of web-based interventions in cancer (Badr et al 2015a), CARES targets both OC survivors and their family caregivers, who play a critical role in home care and supporting adherence (DiMatteo 2004).

Self-management Challenges after OC

To control side-effects and reduce the likelihood of long-term problems after radiotherapy for OC, physicians often encourage survivors to follow intensive self-management protocols. For example, survivors are instructed to use salt-soda rinses 8–10 times per day and drink copious amounts of water throughout the day (even though they have difficulty swallowing), alter their diet significantly to include high-protein and soft/liquid foods, practice daily repetitions of multiple types of swallowing exercises, and engage in intensive oral care routines (Silverman 2003). Adherence can improve swallowing function (Mittal et al 2003), facilitate return to a normal diet (Lazarus 2009, Crary and Groher 2006), help control side effects, and decrease the likelihood of dental caries (Rankin et al 2008). However, non-adherence rates are extremely high. As many as 81% of survivors do not adhere to oral care recommendations (Thariat et al 2012) and 87% do not adhere to swallowing exercise recommendations (Shinn et al 2013). This is problematic because poorly managed side effects are associated with treatment interruptions, social and emotional problems, and a more complicated and costly rehabilitation (Terrell et al 2004).

Although reasons for non-adherence have not been explicitly examined in OC, one possibility is that standards for self-management after treatment do not exist in many institutions or are inconsistently implemented (Epstein et al 2007). There is also a dearth of materials available to teach OC survivors and their families the skills needed for self-management and the coordination of care (McGuire 2003). Research has shown that adherence increases with regular follow-up medical visits (Ozeki 2015); however, most OC survivors see their multidisciplinary healthcare teams on an intermittent basis once treatment has ended.

Unlike healthcare providers, family caregivers see survivors every day. They are thus in an excellent position to encourage self-management (Williams et al 2006) and to support adherence (DiMatteo 2004). Unfortunately, families are often ill-prepared for caregiving, and can display poor communication (e.g., critical or controlling) and model unhealthy behaviors that can undermine survivor adherence to self-management protocols (Homish and Leonard 2005). Moreover, research has shown that the prevalence of distress among caregivers is comparable to that of patients (Verdonck-de Leeuw et al 2007). Oral and head and neck cancers can also be challenging for couples, resulting in modified life plans and physical sequelae that can compromise adaptive communication and the flow of social support (Sterba et al 2015, Badr et al 2015b).

Use of interactive health communication technologies (IHCTs) may facilitate survivor and caregiver QOL

IHCTs involve the interaction of an individual with an electronic device to access, transmit, or receive health information or support (Prochaska et al 2000). Most IHCTs are Internet-based applications and their use has been associated with improvements in disease self-management and QOL across different health conditions, including cancer (Gustafson et al 2001, McKay et al 2002). Home-based behavioral interventions that are delivered via IHCTs have many advantages over other channels of intervention delivery (Lewis 2003, DuBenske et al 2010). Specifically, they: 1) are a convenient on-demand resource; 2) offer features to protect the anonymity of the user; 3) connect people through message boards and other interactive features for social support and information; 4) provide interactive features like videotaped sequences to demonstrate and reinforce behaviors that need to be acquired (e.g., regular practice of swallowing exercises); and, 5) have greater reach to geographically dispersed populations like cancer survivors and their caregivers (Stull et al 2007). Given that the needs and experiences of cancer survivors and caregivers are interdependent (Nightingale et al 2014), IHCTs that provide information, support, and skills training for the dyad (not just for the survivor) may facilitate coping with the physical and psychosocial side effects of illness. In turn, this could foster positive outcomes including better symptom management, faster rehabilitation, and improved QOL for both the survivor and caregiver (DuBenske et al 2010).

This study aimed to develop and evaluate a dyadic, web-based intervention to improve survivor self-management and survivor/caregiver QOL. Toward this end, we: (1) conducted a qualitative needs assessment (semi-structured interviews) with OC survivors and their caregivers to discern their information and support needs as well as their preferences regarding website features and tools; (2) developed the web-based prototype with our multidisciplinary team; and, (3) formally evaluated the web-based prototype (CARES program) by conducting usability and user testing. The study was approved by The Mount Sinai Institutional Review Board. Given the dearth of dyadic web-based interventions in cancer that target both the individual with cancer and his or her caregiver (Badr et al 2015a), each step of the CARES development and evaluation process as well as relevant findings is described in detail below. Our hope is that these steps may serve as a guide for others who are considering developing similar interventions.

Step 1: Conduct Qualitative Needs Assessment

Methods

Given that a clear understanding of the unmet needs and preferences of the end user is paramount to creating relevant content and a user-centered design (Kinzie et al 2002), semi-structured interviews were conducted with OC survivors and their family caregivers.

Eligibility criteria

OC survivors who had completed radiotherapy within the last 12 months, and lived with a spouse/partner or other family member who served as their primary caregiver were eligible to participate. In addition, both the survivor and caregiver had to be age 18 or older, speak/read English, and be able to provide informed consent.

Procedures

Survivors and caregivers were approached during routine clinic visits to participate in separate 60-minute semi-structured interviews. Questions focused on information and symptom management needs and potential design features that could be incorporated to increase user engagement with the website and facilitate communication and support. Participants received $25 gift cards upon completion of the interview.

All interviews were digitally recorded and transcribed. Transcripts were individually validated by the staff person who conducted the interview to ensure accurate and complete transcription. The analysis of transcripts was performed using the technique outlined by Grounded Theory methodology (Strauss and Corbin 1998, Strauss and Corbin 1990). Briefly, two independent coders examined the data for key words and statements that captured respondents’ experiences. Through comparative analysis, patterns underlying survivor and caregiver information and support needs in the setting of oral cancer gradually emerged, and the data were conceptually ordered. For example, for the topic of information needs, we began by identifying types of information that survivors and caregivers thought were important and found either easy or difficult to obtain. We then identified website features that survivors/caregivers thought could facilitate access to information or information exchange. Coder differences were discussed and resolved through continued comparative analysis of the data.

Results

Twenty-four consecutive survivors were approached to participate. If the survivor consented and his/her caregiver was present, the caregiver was also approached to participate at that time. Caregivers who were not present were approached by phone. Sixteen out of 24 survivors agreed to participate (67%) and 12 out of 16 caregivers (75%) agreed to participate. Reasons for survivor refusal included: feeling fatigued or unable to vocalize well enough to complete the interview (N=5), not interested (2), and did not use computers (1). There were no significant differences between survivors who participated and those who declined with regard to available data at the time of recruitment for age, time since diagnosis, gender, or race. Reasons for caregiver refusal were: too busy/no time (2), and not interested (2). Survivors were mostly male (69%), white (75%), and ranged in age from 55 to 79 (M=65, SD=7.2). Caregivers were mostly female (92%), white (75%), and ranged in age from 38 to 68 (M=57, SD=8.3). Caregivers were either a spouse (67%) or adult child of the survivor.

Survivors and caregivers liked the convenience and anonymity of the Internet. One survivor remarked, “A website is very accessible. You can just log on right away and not have to wait till the next day to see when you can schedule coffee with someone to talk about something important.” Echoing this sentiment, a caregiver said, “Sometimes you wanna talk about what happened that day on your own time. When you have time. When you are in your pajamas late at night…Also, you feel freer to share more because it’s anonymous.”

Survivors and caregivers also emphasized the need for a website specific to oral cancer. One woman said, “Everything I found online was general and that didn’t do me any good. I wanted information about how to do specific things that are related to caregiving for my husband who is going through oral cancer.” Pointing out the dearth of support groups for caregivers, another woman said, “I couldn’t find any caregiver-specific support groups and felt uncomfortable going to a group without my dad. I read some discussion forums -- none of them were specific to oral cancer and I think that there’s a stigma associated with it which I got really angry about in the beginning. People would say, ‘Oh, was he a smoker? Did he have HPV? ‘ If it were breast cancer or prostate cancer…It would also be nice to interact more with caregivers who are facing a more similar experience than those who are caregiving for someone with another cancer or another life threatening disease.” Going one step further, a survivor talked about his experiences reading blogs about oral cancer and the importance of knowing the specifics of people’s stories, “I found them [the blogs] interesting, but not necessarily helpful. A lot of cases were worse than mine and almost depressed me, actually.” He continued, “Success stories are inspiring, but you have to pick the stories carefully because you can have someone who might be misled by a rosy story and their situation is not as rosy. If you provide information that is relevant so a person can weigh the success, then it could be really helpful.”

Table 1 details some of the survivor and caregiver information needs that emerged from the qualitative interviews. Overall, survivors and caregivers expressed many similar needs. Both wanted a clearer explanation of side effects and a clearer timeline for recovery. In fact, 10 out of 13 survivors (77%) and 9 out of 12 (75%) of caregivers said they would use a program like CARES primarily to learn more about how to manage treatment side effects and find out more about other survivors’/caregiver’s experiences with managing side effects. One survivor talked about how he creatively used information on the Internet to gauge what his recovery would be like, “On the forums you can track people’s progress, even if it’s in the past. You can select a person and see how they recovered and how long it took by looking at their posts.”

Table 1.

Information Needs of Oral Cancer Survivors and their Family Caregivers (N=25)

	Survivors (N=13)	Caregivers (N=12)
What to expect	Side effects: “I think the website should say clearly what the side effects are going to be for different treatments … and to warn people about the swallowing problems.” Timeline: “I want a site that looks like a big calendar and that shows the day when you’re 100 percent better [laughs]. I don’t know…a timeline of some sort would be nice – just so people would know what to expect.”	Side effects: “I would have liked an honest account of what the reality of the side effects are and what that looks like on a daily basis.” Timeline: “They can give you, you know, a little bit of a heads up…a better understanding of what’s gonna happen, what might happen, and how long things will last.”
Managing nutrition and swallowing issues	Swallowing: “One time I didn’t chew a piece of steak enough. And my husband took me to the emergency room. It’s not fun and games. It’s trial and errors. And I still need to be conscious about chewing my food well and trying to swallow it. That’s one of the very important things in my life right now. I MUST CHEW.” Nutrition: “I would find it helpful if there were discussions regarding the feeding tube. It appears to be somewhat basic – just pour your liquids down, but that can be done at various speeds. I tended to do it a little quicker than others which caused some regurgitation. Having food come back into your throat from your stomach is a really weird sensation because you kind of taste it.”	Swallowing: “The number one thing would be really clear information on how to get your family member swallowing again.” Nutrition: “Information on just like the feeding tube, how to use it properly, keeping it clean so it won’t get infected… “ Nutrition: “It would have been helpful to have more recipes for drinks, smoothies...because you feel at a loss when they don’t eat and you want to mix it up for them so they don’t have to drink the same thing every day.”
Pain management	None	Guidelines: “One of the things that I really struggled with was when is too much? The doctors wanted him to have as much as he needed to control the pain. But I was giving him huge doses and he was pretty out of it and I kept calling my son who is a physician and I was checking with my friend whose husband had cancer…They would say the same thing, ‘You don’t want the pain to get too bad’…but…what about some guidelines??! One day I called my son and said I think I’m giving too much -- I think he’s overdosing. He said to check his pulse and blood pressure, which I didn’t even think about. So things like that you know so you might be aware of when you think you’re overdoing it. That was a really hard thing for me as a caregiver.”
Self-care	None	Taking time out for self: “I think maybe the biggest thing is to really for the caregiver to have some time that is for themselves. I mean you know it has to be limited because you don’t really have the time to do that but sometimes you need an escape. There were days when there was really nothing I could do to help my husband feel better. I could only do what I could do. My hands were really tired even though I was doing all that I was doing. On days like that, people need a break.” Time management: “I would say more advice on how to manage your time so that when you find your free moments -- maybe the only time you have is the 20 minutes on the train between work and home – you would know how to use those 20 minutes.”
HPV	More information: “HPV. It’s a very difficult subject for some people to discuss. On a forum with other individuals in the same boat, it might be easier to share questions and information.”	None
Living a healthy lifestyle after cancer	Exercise: “When you are exercising, the dry mouth is a problem. It’s not even about fatigue in your body, it’s about dry mouth because when you’re exercising, a lot saliva is changing. So how do you strike a balance between trying to life a healthy lifestyle and dealing with the physical changes?” Weight management: “Maintaining health…maintaining…you know…eating habits…I’m trying to keep myself from gaining the weight back… because obviously going through this there is a lot of weight loss and uh prior to this whole thing happening, I was diagnosed as a type 2 diabetic, borderline type 2 diabetic.”	None

Nutrition/swallowing issues were the biggest informational concern for survivors and caregivers. Both expressed feeling ill-prepared for dealing with feeding tube issues at home. Caregivers wanted more information about self-care and pain management. Survivors wanted more information about HPV and making/maintaining lifestyle behavioral changes (i.e., physical activity and weight loss) after treatment.

Table 2 details survivor and caregiver support needs. Survivors were interested to learn more about managing emotions and expressing feelings; caregivers spoke primarily of their experiences with depression and fear of recurrence. Both survivors and caregivers expressed the desire to learn how to better communicate/coordinate care with each other and to connect with other survivors and caregivers for social connection and social support. Caregivers additionally expressed concerns about holding back feelings and strained communication with other family members. Survivors wanted more support for their caregivers.

Table 2.

Support Needs of Oral Cancer Survivors and their Family Caregivers (N=25)

	Survivors (N=13)	Caregivers (N=12)
Emotional Reactions	Managing emotions: “I just think the main thing is how to handle it mentally. The emotional or psychological components…handling the anger and fears.” Managing emotions: “Well, a good portion of the time I keep things in and I’ve noticed over the past couple of months that I’m reaching out more to even just my friends. Just talking about it more, and expressing my fears. So I think it’s important to convey the idea that you can express that and there’s nothing wrong with it. It’s okay to be, “weak” or vulnerable.”	Depression: “I was alone taking care of my husband and I became emotional… I fell into a slump and I felt sorry for all of us.” Fear of recurrence: “I worry that there’s more [cancer] there. I try not to think about it, but it’s there with me. I don’t want him to go through it all again.”
Communication	Communicating with caregiver: “Learning how to communicate better with your caregiver when they don’t do what you expect. How to not expect too much or put too much on a caregiver, stuff like that.” Communicating with caregiver: “Sometimes caregivers get too close and offer sympathy rather than being upbeat, positive. Let’s go for a walk today, let’s go for a movie today. You want your normal boring everyday life back that you took for granted. So I guess how to let your caregiver know that you need that more than their sympathy or pity.”	Coordinating care: “There were times when I would be frustrated because I thought he should be doing something and he didn’t want to do it and we would argue. I remember there was a period where he kept calling me the drill sergeant.” Holding back concerns: “It was really hard knowing how to act, because I felt in many ways like he [her husband] was watching me. I just felt like I had to maintain something because I knew on a big level that he needed me [to be strong]” Communication with family: “Sometimes there are two caregivers…my mother and I did not see eye to eye and were at odds a lot of the time. It would have helped to have someone to talk with us and talk us through…”
Social Connection / Social support	Social connection: “At 2 or 3 o’clock in the morning when you are sitting there in agony, you can go to one of those forums or blogs and you post something and people respond right away. It relieves tension to know that people out there care.” Emotional support: “You can get a lot out of strangers, strangely enough. You kind of hook up with people that are going through the same things [and] you don’t have that emotional pile…They’re not judging you or have ulterior motives. Unlike your family or friends, with strangers, you get – it’s not support, but it’s not criticism either – it’s just listening.” Support for the caregiver: “For the first few weeks after…it would be helpful for caregivers to have some sort of advice on what to do and what not to do. Be calm. Don’t rush somebody. Make them feel good. Tell them they can take it easy and take their time and do whatever they have to do at their pace. Things like that.”	Social connection: “I see people on there [the Internet] praying. Even if they don’t know my mom, they know me [through message boards], and they are praying to make sure that she is doing better.” Emotional support: “Sometimes I need to get cancer off my chest but I don’t want to burden [my husband] because he has already gone through so much.”

In terms of website features, more caregivers (N=8) than survivors (N=6) were interested in tools to help them set up and track rehabilitation goals or to remind them to engage in self-care activities or take/dispense medication. One caregiver said, “It would be nice to set goals and watch the progress as they recover. It’s something you [the caregiver] and the patient could do together, like a project.” In contrast, some survivors expressed skepticism regarding tracking. “I really don’t think a patient who is going through cancer is going to sit down and say, ‘Okay, let me sit down at the computer and track my eating,’” said one man. “I really wasn’t thinking about tomorrow. I just did it all day-by-day. I brushed when I thought it was necessary. I tried to eat when I could.”

The majority of survivors (N=9) and caregivers (N=6) were interested in social media tools that would connect them with other survivors and caregivers. Survivors wanted to reach out to obtain a more realistic picture of the recovery process, “Family members tell you what you want to hear, but other survivors tell it like it is.” In contrast, caregivers wanted to obtain support and understanding, “I have one friend who would call me several times. She was so good but she didn’t really empathize. She couldn’t really understand exactly what I was going through. She was very available but she hadn’t gone through it. So there is something different talking with someone who is going through it with you that can be very helpful.” Comparing what she felt she got from other caregivers that she could not get from medical personnel, one caregiver added, “When you ask the medical people, they keep it more about health, but when you talk with other caregivers on the Internet, it’s more anecdotal. What you get is how to have a comfortable life while you are going through hell.”

Both survivors and caregivers had ideas about how to use the website to improve communication with one another. Caregivers talked about the website as a potential vehicle for reminding the survivor to engage in self-care. One said, “If there was a way you could communicate…and not get into the defensive stuff…Maybe you could write it down online and say you have to have at least five cans (liquid nutrition) a day. If you don’t want to take it now, when are you going to do it? How do you want to do it? If you blow up and express it verbally it makes it bigger, but if you do it in a funny or removed way online - maybe through an emoticon or picture even - you could still express but it wouldn’t be as threatening or anger provoking.”

Another caregiver wanted to use the website as an alternate means of communication with her husband, “Part of the problem is him not being able to talk…there are also times when he’s so fatigued that he can’t listen to me when I want to talk about things. I could write it down [online] and then when he is feeling up to it, he could maybe go and read and respond.” Similarly, a survivor who was having difficulty sharing his feelings felt that online communication with his spouse might make things easier, “When you’re in front of each other, both of you are trying to hold up and be strong, but online you are not seeing each other, so you might be more inclined to be honest because you know your wife is not there with you and isn’t going to respond right then and there. Having that kind of space in the conversation could be really useful.”

Other website features that survivors were interested in were a glossary that explained medical terms, and a feature where they could either chat in real time with a healthcare professional or submit questions that would be answered in a timely manner. “Having access to maybe not your doctor but a professional and saying, ‘This is happening to me. Is this normal?’ Something as simple as that can relieve a lot of stress for the person.” Survivors and caregivers were also interested in recipes for drinks, smoothies, and soft, high-calorie foods, as well as inspirational quotes that conveyed the importance of perseverance and hope.

Step 2: Develop CARES Prototype

Development of the CARES prototype was informed by our needs assessment, best practices for managing oral and swallowing complications following radiotherapy for OC (Broadfield and Hamilton 2006, Mittal et al 2003, Silverman 2003), national healthy lifestyle guidelines/recommendations for cancer survivors (Rock et al 2012, Hanna et al 2013), and Self Determination Theory (SDT). According to SDT, fulfillment of the fundamental psychological needs for competence, autonomy, and relatedness is essential for promoting the internalization of new behaviors and well-being (Deci and Ryan 2011). Competence (which is similar to self-efficacy (Schwarzer 2014) represents the degree to which people feel able to achieve desired outcomes and has been shown to predict health behavior change and disease self-management. Autonomous or intrinsic motivation has been shown to facilitate behavior change whereas controlled motivation (i.e., behaving to avoid guilt or because of a demand from an external agent) has been shown to impede change (Ng et al 2012). This distinction is relevant because controlled motives often follow an illness diagnosis and treatment; survivors may initiate health behaviors not because they want to but because of fear or to please others (Patrick and Williams 2012). Finally, SDT posits that relatedness -- a sense of being respected, cared for, and understood -- is key for the internalization of new behaviors. Experiences of connection or relatedness are developed through interactions with important others such as peers (other survivors), healthcare providers, and family members (Patrick and Williams 2012). By virtue of their connection to the survivor, these individuals are poised to positively influence disease management and adherence. In fact, research has shown that when health care providers and family members support survivor autonomy (e.g., acknowledge feelings, minimize pressure to behave, offer choices, and provide a meaningful rationale for suggested behaviors), it boosts survivor autonomy and competence for adhering to medical recommendations (Ryan et al 2008).

In light of the SDT research base, the CARES prototype was designed to provide information, skills-building, and support services to bolster survivor and caregiver autonomy, competence, and relatedness. The main interface with the program is through a central dashboard that is divided into three sections: ACT, TRACK, and SHARE (see Figure 1). Under ACT, users can access the information services described below. Consistent with SDT which emphasizes the importance of autonomy, the term “act” was chosen because it connotes a more active and volitional role in self-management, as opposed to “learn” which suggests a more passive role. Under TRACK, users can access skills-building services, and under SHARE, they can access support services. Although survivors and caregivers have access to all the services offered by CARES; they log on separately so they can access their own tailored content based on their role. Rather than requiring that survivors and caregivers log on together and sit side by side at the computer while completing the program (Zulman et al 2012), opportunities for collaboration and interaction are facilitated through the skills-building and support services features described below.

Figure 1.

Information Services

Five educational modules were developed to provide survivors and caregivers with information on improving oral health and QOL after OC. Topics are: 1) taking charge after OC; 2) managing oral care and oral side effects; 3) managing nutrition and swallowing problems; 4) managing social and emotional concerns; and, 5) maintaining a healthy lifestyle after cancer. The content of the modules was informed by a comprehensive literature review and evidence-based content review of best practices for symptom management and rehabilitation following radiotherapy for OC, the literature on survivor and caregiver psychosocial adjustment to cancer, and the information and support needs that were identified in the qualitative needs assessment. To assure that module content was medically accurate and consistent with SDT, written drafts were reviewed by a multidisciplinary advisory panel of experts in the areas of head and neck surgical, medical, and radiation oncology, survivor and caregiver QOL, SDT, multimedia production/web design, social work, speech pathology, and nutrition.

Consistent with SDT, each module provides: (1) a clear rationale for recommendations (e.g., evidence base); (2) a variety of behavioral strategies and options to encourage choice and elaboration; and, (3) shared and tailored content based on the person’s role as a survivor or caregiver. For example, shared content included information about symptom management, coping as a team (e.g., joint problem-solving), and cognitive-behavioral strategies for managing depression and anxiety symptoms (e.g., cognitive reframing, relaxation). Tailored content for survivors included strategies for self-management, balancing autonomy with soliciting/accepting support, disclosing care/support needs, and supporting/acknowledging the caregiver. Tailored content for caregivers included strategies to minimize burden, overprotection, and negative interaction patterns (e.g., nagging, criticizing), and for supporting the survivor’s self-care goals.

All modules have quizzes and multimedia features (e.g., videos) to facilitate processing of the materials covered. To facilitate engagement and focus user experience, users also receive tailored email digests based on their role and requested information needs that summarize and reinforce key concepts, direct users to relevant services, and remind them to complete activities/homeworks. Consistent with health communication best practices, short sentences, chunking, and visual cues were employed (National Institutes of Health 2002, Centers for Disease Control and Prevention 2009). Professional editors were also hired to ensure that content was written and laid out in a manner appropriate for the Web (U.S. Department of Health and Human Services Office of Disease Prevention and Health Promotion 2010), and on a 8^th grade reading level.

Skills-building services

Skills-building services were designed to reinforce the information presented in the modules and to enhance competence for self-management, caregiving, and the coordination of care. They include action planning and behavioral monitoring tools (e.g., trackers), as well as alerts via text or email that users can set up to remind them to engage in or log a desired behavior. Other examples include video segments illustrating swallowing exercises that survivors can practice along with (see Figure 2), an audio guide that survivors listen to in order to practice relaxation techniques, exercises based on Cognitive-Behavioral Therapy principles to identify and challenge irrational thoughts that can contribute to affective distress, and joint homeworks and activities that are designed to encourage teamwork and adaptive communication.

Figure 2.

Support services

Support services were designed to enhance relatedness by allowing survivors and caregivers the opportunity to solicit and offer support to peers and each other through limited access, facilitated bulletin boards and a survivor-caregiver sharing function (see Figure 3). The sharing function allows survivors to link to their caregiver through the site. Once connected, both have access to a portal where they can leave messages, work together on joint homeworks, and provide feedback on individual action plans and trackers that they have opted to share. They can also share news and information that they access on the website. To facilitate engagement, users receive email alerts and notifications when they log on to CARES that their partner left them a message on the site.

Figure 3.

Step 3: Conduct Formative Evaluation

Methods

Formative evaluation of the CARES prototype consisted of usability testing based on the U.S. Department of Health and Human Services usability guidelines (U.S. Dept. of Health and Human Services 2006) and user testing consisting of an evaluation survey and interviews regarding users’ impressions of the prototype. Based on these guidelines, a sample size of 5–8 subjects is sufficient to evaluate the usability of a website, and to determine navigation and design problems.

Usability Testing Procedures

Eligibility criteria were the same as for the needs assessment described above. Participants reviewed the CARES prototype using the “task analysis” method whereby they were instructed to look for specific information and to use specific tools in the program while being videotaped (Maguire 2001). A facilitator moderated the testing sessions asking users to “think aloud” while completing a set of representative task scenarios (Monique et al 2004). Sessions were videotaped so that screenshots and participants interactions with the program could be captured. A note-taker additionally monitored users’ interaction with the application and documented feedback and problems. Two research team members reviewed the video clips to identify navigational difficulties. Final program modifications were implemented to correct problems that were detected.

User Testing Procedures

Following the usability test, participants were interviewed regarding the extent to which the CARES prototype and materials were personally relevant, helpful, and easily interpreted. They also completed a 25-item evaluation survey. The survey was adapted from several existing scales (Brooke 1996, Chin et al 1988, Lewis 1995) and assessed attractiveness (i.e., whether the website is pleasant to look at and offers functionality/content that is relevant to the user; six items); controllability (i.e., the degree to which respondents feel confident navigating the website; five items); efficiency (i.e., the degree to which respondents feel they can quickly locate and do what is of interest to them; four items); intuitiveness (i.e., the degree to which the website corresponds with one’s expectations about its content/structure; five items); and, learnability (i.e., whether respondents feel they can start using the website with minimal instruction; five items). All items were rated on a 5-point Likert type scale with 1=strongly disagree to 5=strongly agree. Scale internal consistency reliability (Cronbach’s alpha) was high (α=.73 to .81). Mean scores for each scale were calculated, summed, and then multiplied by 4 to yield a total usability rating score out of 100.

Results

Usability Testing Results

Six OC survivors (83% male, 83% white, age range = 44–70 years old; Mean = 55 years) and five caregivers (80% female; 60% white; age range = 52–65 years, Mean = 58 years) completed the usability testing. Testing revealed 35 system errors/navigation problems that needed improvement. Table 3 lists 10 examples and the resulting modifications.

Table 3.

Usability Testing Feedback and Revisions Made

Topic	Observation feedback from users	Revisions made
Design	Users felt that the light blue font color made some text hard to read and wanted a larger font.	Used a darker, larger font.
Design	Users felt that overall, there was a lack of scannability on key paragraphs	Incorporated more subheadings and bulleted lists that users could click on to display more detailed information if they were interested.
Features: Message Board	Some users did not understand how to post to the message board, or edit/delete threads.	More detailed directions were provided. We added an edit and delete function so that users could modify or delete message board posts.
Features: Progress Bar	Participants were not sure what the progress bar was indicating	The progress bar was redesigned to be more visually prominent and to indicate the percent of the module that is complete as opposed to the percent of the overall program.
Features: Quick Access Menu	Users were not clear what “Act”, “Track” and “Share” stood for on the quick access menu.	Added captions and more descriptive icons to inform users where clicking on each button would take you on the site.
Features: Tracking Function	Users did not understand how to create trackers and add data to the trackers	More detailed directions were added on the trackers page along with a step-by-step graphic to demonstrate how to create and add data to the trackers.
General	Some users wanted more information about who was behind the site so they could judge the credibility of the information presented.	Added an About page and About info to footer, Login, and Home page
General	Some users had privacy concerns	Added privacy language to the communication boards and login.
Links	Users noted that some links were the same color as body content, there were some missed opportunities for cross-linking, and that visited links didn’t change color.	Made all links and buttons a unique color, added links to relevant pages on site, and made visited links a different color.
Navigation	In order to return home, users had to click on the CARES logo, but this was not obvious to many users.	Added home link and icon to main navigation.

User Testing Results

Participants unanimously reported that they found the content to be relevant and helpful and would suggest that other OC survivors and caregivers use the CARES website. Examples of their suggestions for improvement and the resulting modifications are in Table 4. In most cases, we were able to make small changes but had to defer larger more substantive changes due to resource constraints. Table 5 shows the means and SDs for each of the scales on the evaluation survey. Since there were no significant differences between survivor and caregiver ratings, we combined them for simplicity. Overall, users rated the CARES website favorably on each of the usability dimensions we assessed and gave the site a total usability score of 80/100.

Table 4.

User Testing Feedback, Revisions Made, and Future Plans

User feedback	Revisions	Future plans
Users noted the site felt a little impersonal and “cold.”	Added pictures of clinicians, survivors, caregivers, and families together to “warm up” the site. Added more personal examples/stories of how other survivors/caregivers handled specific challenges during their rehabilitation journey.	Add profile photo upload capabilities and more detail in the user profile so that users can write about themselves and their cancer journey. Users then have the option to make their picture and/or profile private or allow other survivors/caregivers to access their profile. Users will have the option to post about strategies that have worked for them for managing symptoms/coping with challenges. Posts will be moderated and healthcare experts (i.e., physicians, psychologists, nutritionists, speech pathologists) on the research team will endorse or comment where appropriate.
Users wanted the ability to communicate in real time.	Added a chat feature so that survivors and caregivers can chat in real time.	We are currently exploring other communication features to allow users to link with other survivors/caregivers through the site to create a broader sense of community beyond interacting solely via message boards.
Users wanted a quick and easy way to access and return to information without having to wade through an entire module to find it.	Added a button on the home screen that allowed users to return to their most recently accessed pages.	Content will be categorized in the information architecture to be pulled and displayed programmatically based on the symptoms selected by the user to allow quicker and easier access. Users will have the ability to bookmark content that they return to repeatedly via profile/history. Setting up a system to allow users to effectively create “quick links” that are most useful for them will enhance and simplify their experience in using the application to manage their health more effectively.

Table 5.

Results of User Testing Evaluation Survey

	MEAN	SD
Attractiveness	4.0	.67
Controllability	4.2	.78
Efficiency	4.1	.67
Intuitiveness	3.9	.89
Learnability	3.8	.88
Total Usability Score	80.0	12.42

Note: Items were rated on a 5-point Likert type scale with 1=strongly disagree and 5=strongly agree. Individual scale scores were then summed and multiplied by 4 to yield a total usability rating score out of 100.

Discussion

IHCTs for cancer survivors and their caregivers offer an opportunity to deliver tailored information in a potentially more efficient, accessible, and cost-effective manner than in person methods. Several aspects of our development and evaluation strategy were critical to the success of the acceptability of the CARES program. First, the multidisciplinary team included individuals with experience in the medical aspects of OC, development of dyadic interventions, web design and application development, and program usability evaluation. Second, we used an iterative process to determine user needs and gather feedback. This stakeholder involvement and feedback provided valuable insight about content and features of the site and was crucial in improving the prototype, resulting in a more user-centered application.

Results of the qualitative interviews showed that OC survivors and caregivers: 1) want and need practical advice about managing side effects; 2) want to reach out to other survivors/caregivers for information and support; and, 3) have both overlapping and unique needs and preferences regarding website features. Even though survivors and caregivers were both concerned about managing side effects, they often approached the problem from different perspectives. Survivors wanted to learn from other survivors about what self-care strategies worked and didn’t work (possibly as a means of organizing or prioritizing medical recommendations). Caregivers wanted to learn how to cope with the challenges of caregiving (e.g., coordinating care and support, managing time, coping with their own emotions) and were interested in goal tracking features that would help them monitor the survivor’s rehabilitation. Thus, our findings suggest that tailored content and features based on role (survivor/caregiver) are appropriate for this population.

The finding that survivors and caregivers were interested in using online tools (e.g., social media) to communicate with other survivors/caregivers as well as with each other deserves further attention. A major research gap has been the lack of integration of social media despite the fact that social support and communication are frequently targeted components of dyadic interventions in cancer (Badr and Krebs 2013). Indeed, social networking is recognized for its potential to provide new opportunities for social engagement and connection (Amichai-Hamburger and Furnham 2007) and research has demonstrated the benefits of social networking sites for mental health and well-being (Ellison et al 2007). Social media represents a challenge for dyadic interventions because intervention focus is on improving social support and communication between patients and caregivers (Badr and Krebs 2013), whereas social media platforms primarily enhance the ability to connect with others outside the family unit. Complicating things further, research has shown that for cancer patients, receiving social network support does not compensate for a problematic relationship or a lack of partner support (Pistrang and Barker 1995). Thus, one challenge for dyadic interventions will be to leverage social media to bring survivors and caregivers together. Future avenues of research include examining whether online communication between survivors and caregivers stimulates in-person communication and whether survivors and caregivers use technology as a substitute for working through communication difficulties (and if that matters).

Overall, results of the qualitative interviews with survivors and caregivers suggest some opportunities to integrate IHCTs in dyadic interventions. For example, integrating content sharing applications may make it easier for survivors and caregivers to reach out for support and express their concerns (Walther and Boyd 2002). In addition, sites that connect survivors and caregivers to others dealing with the same cancer may enhance overall perceptions of social support and augment information from the healthcare team by providing practical tips for managing the day-to-day aspects of life after cancer treatment (Eysenbach et al 2004). However, more research is needed to determine whether online interventions can adequately address the wide range of OC survivor and caregiver unmet needs that were identified and whether IHCT use has discernible effects on survivor and caregiver outcomes relative to other intervention formats. Indeed, how an IHCT system is used is likely to reflect user needs and may be the most important factor in determining intervention efficacy (Han et al 2009). Moreover, one meta-analysis showed that Internet-based cognitive-behavioral interventions that included a therapist were more effective for relieving anxiety and depression than those that did not (Spek et al 2007). Thus, there may be circumstances where IHCTs need to be supplemented by face-to-face contact with a mental health or medical professional.

Overall, users rated the usability of the site favorably with a usability score of 80/100 which is comparable to the only other dyadic web-based intervention development study that has been published in cancer (Zulman et al 2012). Usability testing uncovered some problems with the intuitiveness of the site. Users wanted clearer instructions for interacting with site features (e.g., how to post to the message board, create trackers), and clearer definitions of what labels indicated (e.g., on the progress bar or quick access menu). Some navigation and design problems were also identified and addressed to improve functionality.

Preliminary user testing findings were promising. All the participants found the content to be both relevant and helpful and indicated that they would recommend that other survivors/ caregivers to use it. Many of their suggestions to add warmth to the site (e.g., add pictures, personal stories) and improve functionality were incorporated but other suggestions were deferred due to resource constraints.

This study had some limitations. First, the sample size for the usability testing was small. Although the recommendation is five to eight subjects to determine navigational problems, more subjects might be required for more complex web-based tools such as this one. Second, even though the caregivers who completed the usability testing were the same age on average as the caregivers who completed the qualitative interviews, survivors who completed the usability testing were younger. Thus, it is possible that age may have played a factor in the acceptability of the CARES program among survivors. Third, the study samples were predominantly Caucasian and thus may not be representative of the entire OC population. Finally, because this is an ongoing study, we are not yet able to report on preliminary usage or process and outcome variables. Additional analyses will provide insight into survivor and caregiver characteristics that impact the value of the CARES intervention and a future randomized trial will provide more objective results regarding the impact of CARES on the SDT constructs of autonomy, competence, and relatedness, as well as survivor and caregiver QOL.

Overall, this study demonstrated that OC survivors and their caregivers are interested in using an online program designed to improve their QOL and that the CARES program that we developed is acceptable to the target population. This study also outlined an iterative systematic development and user testing process that can serve as a prototype for other researchers who are considering developing similar dyadic interventions that target cancer survivors and their caregivers. The impact of the CARES intervention will be determined when data collection from the randomized trial is complete. If proven effective, CARES could be disseminated with the potential of improving self-management, coordination of care, and survivor and caregiver QOL.