Development and Usability Testing of the e-EXCELS Tool to Guide Cancer Survivorship Follow-Up Care

Abstract

Objective:

The aim of this study is to describe a user-centered (e.g., cancer survivors and clinicians) development process of an eHealth tool designed to facilitate self-management of cancer survivorship follow-up care.

Methods:

Guided by Cognitive-Social Health Information Processing Model (C-SHIP) and informed by core self-management skills we engaged in a user centered design process. In phase I we conducted depth interviews with survivors of breast (n=33), prostate (n=4), and colorectal (n=6) cancer, and (n=9) primary care providers to develop content and design of the web tool. Phase II utilized iterative user testing interviews (n=9) to test the web-based tool prototype. Data from both phases were independently coded using a template/content analytic approach.

Results:

The top 5 functions identified in phase I for the web based platform included: (1) educational materials to learn and prepare for health encounters (80%); (2) questions for health providers (74%); (3) ability to track contact information of providers (67%); (4) provide general information (64%); and, (5) support information (62%). Users of the prototype reported patient burden, tool fatigue, introduction timing of the tool, relevance, and security/privacy as concerns in Phase II.

Conclusion:

This study demonstrates the value of using a theoretically informed and user-centered design process to develop relevant and patient centered eHealth resources to support cancer survivorship. A larger study is needed to establish the efficacy of this eHealth tool as an intervention to improve adherence to follow-up care guidelines.

BACKGROUND

Survivors of breast, colorectal and prostate cancer represent approximately one-third of the 16.9 million cancer survivors in the United States.(1) Due to the impact of breast, prostate and colorectal cancers and their various treatments, survivors are at increased risk for health complications including cancer recurrence and subsequent primary cancers.(2) Survivors experience a variety of symptoms, including pain, fatigue, distress, and anxiety that can impact their health and quality of life if poorly managed.(3) The chronicity of symptoms post-treatment have led to the promotion of a chronic disease model of long-term cancer care, including self-management strategies to empower survivors.(4) Self-management is the ability to manage the symptoms and consequences of living with a chronic condition, including physical, social and lifestyle changes.(5) Cancer survivors’ self-management tasks are those that support effective communication exchanges between the multiple healthcare providers to manage symptoms, make health-related decisions, and support behavior change.(4, 6)

For survivors to adhere to follow-up care guidelines they need information, tools and support to promote these activities in partnership with their healthcare teams. A recent literature review concluded patient-centered self-management interventions specifically targeting the core skills and information needed among long-term cancer survivor are needed.(7) Self-management interventions for survivors that use in-person or telephone based support have demonstrated improvements in overall quality of life, self-efficacy, cancer-related distress, depression and anxiety; but, require significant resources to run and are not continually available to survivors.(7, 8) Self-management tools that utilize web-based supports, also characterized as electronic health (eHealth) have been demonstrated to be acceptable and feasible in non-U.S. breast cancer survivor populations (9) and among other chronic condition populations. ^6,37

Presently, there are limited eHealth resources available to guide cancer survivors’ activities for longer-term follow-up. Therefore, we developed the Extended Cancer Education for Longer-term Survivors eHealth self-management tool (e-EXCELS), a mobile optimized website. This study describes the development and usability testing of e-EXCELS, that is currently being compared to other self-management strategies (e.g., telephone health coaching) in an ongoing randomized controlled trial. (10)

METHODS

Overview of procedures

The development of e-EXCELS used an iterative, seven-step approach (see Figure 1). These included: (1) a comprehensive literature review to draft an initial content map (11, 12); (2) the identification of credible virtual information; (3) the development of a relational database to organize content specific to each cancer type that aligned with American Cancer Society Primary Care Cancer Survivorship guidelines; (13–15) (4) production of preliminary wireframes; (5) collection of user data (Phase I); (6) development a prototype and, (7) user/usability testing (Phase II) and final production of e-EXCELS.

Figure 1.

Overall development and preliminary testing of e-EXCELS

Guiding theory

The content and design of e-EXCELS was guided by the Cognitive-Social Health Information Processing Model (C-SHIP), which addresses how individuals process information about cancer threats and prevention options based on their perceived vulnerability, disease characteristics, and self-efficacy expectations, goal/values, and, coping strategies.(16, 17) e-EXCELS supports the processing of tailored cancer survivorship information that is mobile optimized for ease of use during point-of-care interactions. The objectives of e-EXCELS were to: (1) promote accurate processing of recurrence risk and risk of secondary cancers; (2) identify emergent or ongoing treatment symptoms; (3) reframe beliefs and expectations about need for risk-based preventive care for existing sequelae; and (4) identify action steps to encourage guideline concordant follow-up.

Preliminary content map development

We identified preliminary content domains based on the existing literature and preventive health services guidelines, and C-SHIP constructs tailored by cancer site. Domains include: (1) follow up care procedures; (2) communicating with healthcare teams; (3) emotional concerns; (4) healthy lifestyle; and (5) skills and resources.

The content map guided the writing process for e-EXCELS materials. This process included the identification of source materials, developing new content and revising existing materials. All text content was written to conform to best practices used in communicating health messages.(18) Design of text content aimed to reach an 8^th grade reading level.(19)

Development of Wireframes (Website Interface Design)

e-EXCELS was designed in collaboration with NotSoldSeparately (NSS), a healthcare software production company that has previously collaborated with the team in the development of cancer-related e-Health tools.(20) The research team met weekly with NSS to refine the conceptualization, presentation, functionality, and tailoring scheme of e-EXCELS.

The NSS team produced a wireframe of the web interface, the research team provided feedback on preliminary versions to guide refinement around content map domains. The research team provided input on the visual design of the website (e.g., page layout, content formatting, etc.). After four iterations, the wireframes for Phase I data collection were produced.

Target User Information Needs and Design Preferences

Phase I focused on target user input (e.g. survivors and primary care physicians) about website content and functional design. Phase II focused on feedback about the design and content based on survivors’ experience navigating the e-EXCELS prototype.

Phase I Participants

Between November 2014-April 2015 a purposive sample of early stage (I-III) breast, colorectal and prostate cancer survivors were recruited through community-based survivorship organizations, local oncology and primary care practices, and the Rutgers University Faculty and Staff on-line bulletin. Eligible survivors: (1) were post-treatment (except maintenance therapy); (2) had breast, colorectal, or prostate cancer; (3) could read and speak English; (4) were age 35 or older; and, (5) were able to provide informed consent. Interested participants phoned the research office and were screened. A purposive sample of practicing primary care clinicians were recruited through the New Jersey Primary Care Research Network. At the interview appointment, graduate-level trained interviewers obtained written consent. Interviews were audio-recorded and transcribed verbatim. Participants received a $20 gift card. The study was approved by the IRB at Rutgers Biomedical and Health Sciences (Protocol number 2013003309). The data that support the findings of this study are available on request from the senior author. The data are not publicly available due to privacy or ethical restrictions.

Data Collection—Phase I—Needs Assessment and Design Preferences

Phase I patient interviews were semi-structured, ~70 minutes, and conducted in-person in the research office or in the community. The interview schedule included a base interview with a subsection focused on e-EXCELS development. Domains in the subsection included: information seeking, tool utilization and, reactions to the e-EXCELS wireframes. Interviewers described the functionality, showed participants the associated wireframe and probed about information organization and health related activities. Each participant was asked an open-ended summary question, “What types of information would be the most helpful in a website/app for cancer survivors?” with a list of probes (e.g., screening reminders, emotional support, etc.).”

Phase I physician interviews were semi-structured, ~50 minutes, and were conducted in-person at practice sites. Domains of interest included: physicians’ clinical experiences with the cancer survivor population and their perceptions about patient self-management support needs.

Based on findings from Phase I e-EXCELS was refined and a video orientation was developed to be tested in Phase II.

Phase II—Usability/User Testing

Phase II user testing was conducted using a guide based on usability engineering principles to refine the content, design and perceived utility of e-EXCELS. (21) The observation and interview took ~50 minutes and ascertained needed formatting, design (e.g., layout and flow), and content changes. Interviewers described the purpose of e-EXCELS, provided brief instructions and temporary log in details. Patients were asked to “think aloud” as they navigated through the website (i.e., profile page, orientation video, my healthcare team, learn and prepare, and action items) and to give overall impressions. At each page, interviewers used prompts to gauge users’ impressions.

Analyses

All data were analyzed using a content/template based analytic approach to reduce and synthesize patient preferences for content preferences and functionality. We coded both the open-ended and probed responses of the summary content question to discern top functions for e-EXCELS. Our analysis was aided by ATLAS.ti version 7 software. Additionally, an ongoing, immersion crystallization approach to identify patient concerns about the e-EXCELS tool was conducted. Two coders, who jointly coded 50% of the cases, completing the remaining cases after the codebook was established. At weekly research team meetings key themes were discussed and coding discrepancies were resolved through consensus.(22)

RESULTS

Participant characteristics

Table 1 presents the demographics and overall characteristics of patient participants from both phases of the study (N =51). For Phase I interviews, there were cancer survivors (n=42), with breast (69%), prostate (16.7%), and colorectal (14.3%) cancer histories. The majority of survivors were Caucasian (64%), with one-third of the sample identifying as African American. Survivors represented the continuum of survivorship with 17% less than two years from diagnosis, 33% between two and five years, 26% greater than five from diagnosis and 24% more than 10 years from treatment. The majority were college educated with most having some college or a bachelor’s degree (66.7%) or a graduate degree (21.4%).

Table 1.

Characteristics of the Patient Sample

	Total (N=51)		Phase 1 (n=42)		Phase 2 (n=9)
	Frequency (n)	Percent (%)	Frequency (n)	Percent (%)	Frequency (n)	Percent (%)
Age
Median	61		63		55
Range	39-84		39-84		44-65
Race
White	32	62.7	27	64.3	5	55.6
African-American	18	35.3	14	33.3	4	44.4
Other	1	2.0	1	2.4	0	0.0
Ethnicity
Hispanic	1	2.8	1	3.7	0	0.0
Non-Hispanic	35	97.2	26	96.3	9	100.0
Cancer Type
Breast	34	66.7	29	69.0	5	55.6
Colorectal	9	17.6	6	14.3	3	33.6
Prostate	8	15.7	7	16.7	1	11.6
Cancer Stage
1	31	60.8	29	69.0	2	22.2
2	10	19.6	6	14.3	4	44.4
3	10	19.6	7	16.7	3	33.3
Years Post-Treatment
< 2	11	21.6	7	16.7	4	44.4
2 – 5	16	31.4	14	33.3	2	22.2
6- 10	14	27.5	11	26.2	3	33.3
10+	10	19.6	10	23.8	0	0.0
Marital Status
Single	9	17.6	8	19.0	1	11.1
Divorced	14	27.5	7	16.7	7	77.8
Married	26	51.0	25	59.5	1	11.1
Widowed	2	3.9	2	4.8	0	0.0
Employment *
Full-time	28	54.9	21	50.0	7	77.8
Part-time	7	13.7	7	16.7	0	0.0
Unemployed	15	29.4	14	33.3	1	11.1
Income
≤ $39K	5	10.0	5	11.9	0	0.0
40K- 79K	13	26.0	12	28.6	1	11.1
80K+	32	64.0	24	57.1	8	88.9
Education
HS or less	6	12.0	5	11.9	1	12.5
Some college or BA	33	66.0	28	66.7	5	62.5
Graduate +	11	22.0	9	21.4	2	25.0
Device Ownership
Desktop or Laptop	46	90.2	37	88.1	9	100.0
Tablet	33	67.4	25	59.5	8	89.9
Smartphone	40	78.4	31	73.4	9	100.0
Internet for cancer search
Weekly	7	13.7	6	14.3	1	11.1
Monthly	18	35.4	14	33.3	4	44.4
< 1x month	12	23.5	10	23.8	2	22.2
Never	14	27.4	12	28.6	2	22.2

^*Missing data

Physician participants (n=9) in Phase I were mostly female (55.6%), Caucasian (55.6%) and none were Latino. All were family physicians (n=9) with four practicing in single specialty practices and the others in multi-specialty practices. Length of practice tenure ranged from 1-35 years.

RESULTS

Content analyses of the survivor and physician interviews indicate survivors wanted a tailored, interactive tool to manage their care (see Table 2). The top 5 functions endorsed by more than half of survivors included: (1) educational materials to learn and prepare for health encounters (80%), (2) questions for providers (74%), (3) ability to track contact information of providers (67%), (4) provide general information (64%), and, (5) support information (62%). There were variations based on cancer types in terms of desired functionality. Breast cancer survivors wanted educational learn and prepare materials (86%), questions (83%), general information (76%), contacts (76%) and support (73%). Colorectal cancer survivors wanted information on additional resources (100%) followed by support and tools (67%). Prostate cancer survivors were most interested in questions for health care providers and educational materials to learn and prepare for health encounters (57% respectively).

Table 2.

Results of Content Analysis, Selected Quotes and Summary Feedback for Top Functions of e-EXCELS

	Breast		Colorectal		Prostate		Total		Cancer Survivor Perspectives	PCP perspectives	Summary Feedback
	n	%	n	%	n	%	n	%
Learn and Prepare	25	86.2	2	66.7	4	57.1	31	79.5	“… if you have it right here then it’s like, oh, you know, maybe this is an issue for me.” -, CA, age 63 (BC 03)	“I guess I’m thinking … like, “Things you can do,” or, “Things to watch out for…” MD, 18 years in practice	Users preferred to see only actionable information related to their specific issues.
Questions for Health Care Team	24	82.8	1	33.3	4	57.1	29	74.4	“…Being able to remember the question to ask my doctor when I get there…[otherwise I]…can’t remember the questions.— AA, aged 54 (BC 53)>	“… I would suggest … ‘Here are some suggestions of questions to ask… your doctor,’… make it like a list….” MD, unk years in practice	Patients wanted to add custom questions Physicians wanted to help prime patient questions
Health Care Team Contact Information	22	75.9	1	33.3	3	42.9	26	66.7	“Just helps you manage your whole health situation, all your doctors, …” –CA, aged 68 (CRC 006)	“… who’s the doctor, who’s the nurse, who’s the …contact… that becomes my team…” MD, unk years in practice	Input in this section problematic Wanted interaction with calendar software on devices
General information on how to stay healthy	22	75.9	1	33.3	2	28.6	25	64.1	“You turn on your computer and you’ll find something there…”— CA, aged 84 (PC 02)	“…. once they’re really cured they should really be worrying about their blood pressure …and all those things.” MD, 35 years in practice	Guidance to credible sources of information Clinicians emphasize broader health concerns
Emotional Support	21	72.4	2	66.7	1	14.3	24	61.5	“…Back then, I really didn’t know anybody else that really had cancer to even say…It was always such a hush-hush, that people didn’t want to discuss it.” –CA, Aged 68 (BC 54)	“… So some of the psycho-social, the emotional aspects of growing older are all part of that [survivorship] process. .” MD, 1 year in practice	Patients describe changing support needs Clinicians describe similar needs among aging chronic illness populations

AA=African American, BC=Breast Cancer; CA= Caucasian American; CRC=Colorectal Cancer; MD=Medical Doctor; PC=Prostate Cancer; PCP=Primary Care Physician

Phase I Feedback.

Survivors and physicians provided feedback and expressed opinions on the presentation style, modality and user burden related to specific functions. The website wireframes included a profile page, a tab to track the healthcare team, an interactive list of action items, questions for health providers, and, a tab with educational materials focused on symptom management.

Survivors focused most on the “learn and prepare” function, a standardized educational presentation of cancer-related symptom-specific (e.g., fatigue, etc.) and self-care topics (e.g., nutrition, etc.). This function includes a brief description of the topic and its relevance to cancer survivors, questions for providers, and links to additional topic-specific resources. In their review, survivors underscored the importance of being prepared for interactions with providers; but, did not like the initial title --“health alerts.” Physicians agreed with survivors’ concerns and we retitled this “learn and prepare.” Questions for providers were designed to prime the patient-clinician interaction to develop a symptom management or behavioral change plan. Based on overall feedback, we added a function that enabled survivors to select questions and link them to a provider and a custom question feature.

Due to survivors varying levels of comfort using e-EXCELS during a clinical exam we added an option to print or email questions. Based on overall feedback, general information about health maintenance was tailored based on age, gender and cancer type. In our design, we both address the patient expressed needs for specific enough information to inform a self-management plan and provide access to links of information beyond e-EXCELS for more in-depth exploration of topics using credible information sources.

Phase II User/Usability Testing.

Refinements based on Phase I feedback were made to the prototype and an additional 9 survivors were recruited for usability testing of e-EXCELS. Feedback on the content, length, and presentation of the short video orientation were assessed and the video was refined based on user suggestions. The major user concerns identified were: (1) patient burden, (2) tool redundancy/fatigue, (3) timing in survivorship trajectory, (4) relevance, and (5) security/privacy issues (see Table3).

Table 3.

Patient Concerns and Design Responses for e-EXCELS Development

Concern	Patient Concerns	Design Response	Implementation Implication
Patient burden	– Too much information input – Information needs manual updates	– Minimal data for functionality	– Patient wanted tailorable notifications (e.g. content/mode) – Integration with health care setting so less input required
Tool Redundancy	– Too many passwords/profiles	– Single profile data entry at initial sign on	– Different portals across settings fatigue end users
Timing in survivorship trajectory	– Survivors already figured out a system that “works”	– Orientation to relevance for longer-term survivors – Information temporally tailored based on diagnosis date	– Introduce e-EXCELS earlier in survivorship
Relevance	– Survivors issues changed over time	– Tailored information by age, gender, cancer site, time since diagnosis – Learn and prepare section specific to cancer site – Custom topics can be added	– Survivors express the salience of different issues over time, topics need updating for newly identified issues
Security/Privacy	– Concerns about privacy	– Hosted in protected HIPAA compliant website	– Cross organization interoperability constrained by privacy/security

DISCUSSION

eHealth tools have focused on the development of apps and tools targeting health behavior change (23) and symptom management;(24, 25) however, the development of tools to promote survivorship self-management lags significantly behind.(10) Survivors are increasingly using technology to communicate health issues, manage chronic diseases, and search for health information.(26) The present study suggests that survivors desire information to help them be and feel prepared for their health encounters and want access to targeted, credible information. Patients in our study expressed preferences for questions to prime interactions, around specific topics and strategies to organize this information (e.g., contact information, question lists). (27) Advantages of eHealth interventions include access to appropriate information, ability to tailor content based on survivor’s post-treatment needs, the inclusion of a variety of learning modalities, and the option for survivor’s to interact in a dynamic, self-directed manner optimal for patient-centered usage.(28) The potential for eHealth tools to support cancer survivors to feel and be prepared during provider interactions over long-term cancer survivorship may be more beneficial than static documents, like survivorship care plans.

To maximize eHealth tool implementation for cancer survivors in primary care the broader challenges of interoperability need further examination.(29) In this study, cancer survivors expressed wanting tools that are easy to use, interoperable across health systems, and interface with the technology they use (e.g., contacts/calendars). Users expressed concern that the tool might require “too much work” in terms of data entry. During tool development we minimized mandatory field requirements for functionality; however, the tool requires some input from patients to be functional. Several survivors expressed the desire for the tool to protect their health information and to interface with their multiple providers to decrease manual data input. Some users expressed general fatigue with healthcare dashboards and multiple systems that are not interoperable. The team noted patient preference for interoperability and its congruence with the concern of tool fatigue; however, developing interoperability across multiple settings impacts costs, security, and technical complexity.

Key findings from this study suggest that survivors are aware of the potential of eHealth tools in managing their long-term follow-up needs but some survivors are skeptical about whether they would routinely use the tool. E-health interventions are increasingly used for cancer patient care to enhance interactions with providers, enhance understanding of post-treatment care, address unmet psychosocial needs and positively affect health.(30, 31) The potential of eHealth tools to empower patients with tailored information related to cancer survivorship needs has been understudied. As a result, much remains unknown about the potential of eHealth tools to promote adherence of survivorship follow-up guidelines. Studies of eHealth interventions to promote adherence in chronic conditions suggest the effectiveness of eHealth tools for long-term condition guideline adherence remains mixed.(32, 33)

Study Limitations

There are some limitations in the present study to consider. We recruited a sample to represent the target age group of end users, and the average age of cancer survivors in Phase I was 61 years old. In recent years, access to technology has risen significantly among older adults. Access to the internet via phone, computer or tablet was an inclusion criteria, however, recent studies suggest phone ownership is not sufficient to determine the utility of eHealth applications among underserved populations and mobile phone literacy and utilization of health apps need further consideration.(34) We used an inclusive design and 35% were African American; however, only 10% of the sample were low-income and 12% were among the low educational group. This limits our generalizability; however, our ongoing clinical trial aims to diversify the study sample to evaluate utility in these populations. We had a small sample of user testers, and were testing a prototype not the final e-EXCELS. Small sample sizes in eHealth user testing limits our generalizability; (35) however, our development process was similar to published research on web tool development using between 8 to 12 participants to access usability. (20, 36)

Clinical Implications

Efforts to implement personalized survivorship pathways models of survivorship care are underway in the U.S., and strategies to promote self-management will be needed.(37) In the UK, low risk survivors triaged to the least intensive follow-up in primary care account for 67% of breast, 53% of colorectal, and 47% of prostate cancer survivor populations.(38) e-Health interventions may have a role in risk-based health service delivery innovations, and have been shown to be acceptable and convenient for patients.(39) Our findings suggest that survivors are aware of the potential of eHealth tools in managing their long-term follow-up needs remain skeptical about whether they would routinely use them.

Conclusions

A larger study is needed to establish the efficacy of the e-EXCELS as an intervention to improve adherence to follow-up care guidelines. Future research focused on the intersection of factors that drive use of eHealth (e.g. behavioral intentions, self-monitoring style) and populations characteristics (e.g., monitoring style) to discern which interventional elements achieve clinical benefits is needed. Additionally, research and development strategies that aim to integrate eHealth self-management tools with electronic health records across providers and systems would advance patient-centeredness of e-Health tools.