Development and Implementation of an Internet-Based Survivorship Care Program for Cancer Survivors Treated with Hematopoietic Stem Cell Transplantation

Abstract

Introduction

The internet provides a widely accessible modality for meeting survivorship care needs of cancer survivors. In this paper we describe the development and implementation of an internet site designed as a base from which to conduct a randomized controlled trial to meet psycho-educational needs of hematopoietic stem cell transplantation (HSCT) survivors.

Methods

A cross-disciplinary team designed, wrote content and programmed an internet site for online study registration, consent, assessment, and study implementation. All 3–18 year survivors of HSCT for hematologic malignancy treated at one transplant center were approached by mail for participation. All study activities could be conducted without study staff contact. However, participants had options for phone or email contact with study staff as desired.

Results

Of 1775 participants approached for the study, 775 (58% of those eligible) consented and completed baseline assessment. Mean age was 51.7 (SD=12.5, age range 18–79), with 56% male. 57% required staff contact one or more times; a majority were for minor technical issues or delays in completion of enrollment or baseline assessment.

Discussions/Conclusions

This study demonstrated the potential for providing internet-based survivorship care to long-term survivors of HSCT. Although building a survivorship internet site requires a team with diverse expertise, once built, these resources can be implemented rapidly with large numbers of survivors.

Implications for Cancer Survivors

While internet-based services will not meet all the needs of cancer survivors, this methodology represents an important modality for augmenting onsite clinical services as a method for meeting psycho-educational, information and resource needs of cancer survivors.

INTRODUCTION

The internet provides an obvious opportunity for addressing long-term side effects and needs for surveillance associated with elevated risks for multiple diseases in cancer survivors, particularly for those who live at a distance from major cancer resources. Among adult long-term cancer survivors, those who received hematopoietic stem cell transplantation (HSCT) live with some of the highest risks for life-threatening chronic health conditions including cardiovascular disease (hyperlipidemia and hypertension in particular), infection, diabetes, osteoporosis, and recurrence or second cancers, among other known risks [1–12]. Even more prevalent in HSCT survivors are symptoms that inhibit quality of life such as cancer-related distress, depression, and chronic fatigue [13–16]. Emotional distress, medical problems and specific functional deficits are more prevalent in HSCT survivors 2 to 10 years after treatment than in similar-aged normative adults or survivors of other cancer treatments [3,14,17–27]. Recent data indicate that mortality rates remain four to nine-fold higher than the expected population rate even 15 years post-HSCT, with about a 30% reduction in life expectancy [9]. These elevated risks underscore a need for ongoing preventive and surveillance care to reduce transplant-related mortality and health complications.

Survivorship care plans that include surveillance guidelines and symptom treatment or functional suggestions are recommended for all cancer survivors [1]. Despite availability of surveillance guidelines for HSCT survivors [28], research indicates that these survivors do not fully adhere to guidelines [29–31]. Furthermore, by 2 to 5 years after transplant, most survivors are no longer under the care of transplant physicians and have transitioned back to their primary care providers (PCP). However, PCPs responsible for these patients’ care are unlikely to be familiar with specific health needs of HSCT survivors [32]. These access and adherence barriers can be readily addressed through tailored internet-based resources, education and tools targeted to survivors who can then become advocates for their own care and providers of information to their PCPs. The internet provides a more flexible and tailored alternative to print materials as a modality for providing information to survivors at a location and at ‘teachable moments’ convenient to meet survivors’ needs.

Numerous examples of randomized controlled trials of internet interventions have been successful outside the field of cancer survivorship [33,34]. However, there is a wide range of enrollment rates among internet based interventions (12%–54%) and high attrition [35–39]. The extent to which internet interventions are effective may in part depend on their ‘ingredients:’ components, delivery methodology, design, and tailoring, among other implementation specifics [40]. In order to increase enrollment and efficacy, internet based interventions can improve website utilization using regularly updated content, “push” reminders (such as emails), personal contacts by study staff, and tailored messaging.

Internet-based research is especially efficient when fully automated systems allow participants to register, consent and provide assessments online, in addition to receiving the intervention content. Numerous studies have shown that online surveys can be less expensive, reach larger and more widely dispersed study populations, and can increase the accuracy, completeness and consistency of data collection compared to paper versions [41,42]. Programming online surveys to allow participants to skip questions either by choice or to tailor to user characteristics (e.g., males do not see questions pertinent only to females; those who indicate no symptom present are not asked further questions about that symptom) has also been shown to increase response rates [42].

The study described here provided an online secure patient portal and internet site, called INSPIRE, for survivors of hematologic malignancy who are 3 to 18 years after HSCT. While the initial approach occurred with a letter briefly describing the study and providing the URL link for the study internet site, all further study participation could be done online, including study registration, login and password setting, consent, baseline assessment, internet site content access and outcome assessment. Automated email messages provided links at each stage of participation and “pushes” of specific internet site content were delivered throughout the intervention period.

The purpose of this paper is to describe the components of the INSPIRE internet site and details of the study implementation to assist others in utilizing or adapting these methods for future internet studies for cancer survivors. This paper includes the design of the study, internet site content description, security and design of the internet site data collection, and enrollment procedures. We address the extent to which an internet-based intervention can be automated, and the amount of human interface required to enroll and assess participants through a largely online, automated system for meeting survivorship care needs of cancer survivors.

INSPIRE INTERNET SITE DESIGN AND DEVELOPMENT

Team and Timeframe

The design and development of the INSPIRE project was a collaboration of efforts of three psychologists, three HSCT physicians, a nutritionist, two internet design specialists, an internet writer, three programmers, an administrator to build and test the administration and tracking of the online system, an HSCT survivor and advocate, three non-specialists who provided a non-medical perspective, and innumerable HSCT survivors who have provided input on the content of long-term survivor text-based education interventions over many years that culminated in this online product. We aimed to build a nearly entirely automated research website that was easy to use, had strong security features for the assessment component, and had extensive testing to prevent technical problems. The automated elements and other requirements were greater than initially conceived, which increased the time required to finalize and launch the website.

The processes of conceptualizing the website, developing the content, implementation, and final launch are described below. In total, development and testing of the website was a two-year-plus effort, a major portion of which included institutional construction of a secure patient portal and the design, building and testing of a secure site for online study registration and assessment. The start-up effort included pilot testing of the registration and assessment procedures with 144 participants in the assessment component of the procedures [43]. The institution also needed to establish new definitions of security standards as well as develop Institutional Review Board standards for reviewing internet studies. Content selection, design, drafting, web writing and building required about one year.

Needs Targeted by the Intervention

In selecting the intervention targets, we reviewed the extensive research, including our own, that has defined long-term deficits after HSCT. Medical problems, lack of stamina, depression and emotional distress are more prevalent in HSCT survivors than in normative controls [14,17–19]. This is also the case when comparing HSCT survivors to survivors of other cancer treatments [3,14,20–27,44,45]. Rates of depression after HSCT are as much as twice those reported in the general population, although symptoms often do not reach clinical levels [13,15], and the rate of recovery flattens in survivors by one year after HSCT [13,46]. When depression does occur among HSCT survivors, it is consistently a risk factor for poorer long-term physical and mental health as well as survival [13,47]. Distress in relation to cancer is described as worry or stress specific to the disease and its treatment, and is higher than rates for clinical depression or anxiety in HSCT survivors [13,44,48,49]. Living with uncertainty and fear of recurrence are the most severe sources of distress after cancer and HSCT [44,46,50–52]. Physical limitations are often called fatigue by survivors, but after HSCT these symptoms most commonly include lack of stamina and muscle weakness more than tiredness [43]. These related symptoms are some of the most persistent beyond the first year of recovery, and thus were the focus for the internet content [14,15,53].

Internet Site Design and Content

The theoretical model on which the content was based was broadly psycho-educational [54,55], with a cognitive-behavioral approach used for the Restoring Energy and Renewing Outlook sections [56,57]. Problem-solving strategies were described and tools provided for individual use of the problem-solving methodology [58,59]. The identification of the content approach was designated before the grant submission. Based on the targets for intervention, the interdisciplinary research team developed a site map for content (Figure 1). The site map allowed us to combine the broad and varied topics within the body of HSCT survivor research described above into a logical and cohesive set of sections that were informative, approachable, and encouraging of behavior change. Content for these sections was then written by the team members, discussed at weekly meetings and edited by all members of the group to ensure that the intervention targets were successfully met in a user-friendly and engaging way.

Figure 1.

INSPIRE Site Map

The three main intervention sections were determined by the literature review of the needs of HSCT survivors: Boosting Health (cardiovascular, bone and second cancer risks and recommendations), Restoring Energy (fatigue, muscle weakness and inactivity) and Renewing Outlook (depression, distress and social isolation). Two additional sections provided resources and opportunities for ‘Getting Connected’ and ‘Tips and Tools.’ The ‘Tips and Tools’ section provided downloadable and printable symptom and goal-monitoring resources for participant use. The ‘Getting Connected’ section provided a managed ‘Our Space’ page on which participants could write their own comments to each other or to the study staff for individual response by return email or phone and/or for posting on the site. A ‘News You Can Use’ page had links to current research or news articles relevant to HSCT survivors.

For the look and feel of the internet site, we worked with the internet design team that developed the Hutchinson Center’s internet interface. The site utilized rotating images to maintain engagement with the site [60]. With each page change or refreshed page, a new image appeared on the internet site’s banner to engage the viewer and increase interest in the internet site content. Psycho-educational material with a cognitive-behavioral theoretical base formed the foundation of the content approach. To build a sense of familiarity and meet expectations, each main section had the same format and subheadings for content organization (Take Charge, Join Forces, Best Choices). Images selected were nearly all stock photos, diverse in age, race/ethnicity, body shapes, sizes, and activities they portrayed. Links were added throughout the internet site to connect to outside resources as supplements to the INSPIRE internet site content and to reduce duplication with readily available materials on other widely used sites.

Design of the assessments was based on results from previous internet studies determining that one question per computer screen makes it less likely that respondents will skip a question [42]. Each question is clearly visible and respondents do not get off track in responding to questions and their response options. While this helps to reduce missing responses and measurement errors, it also adds some effort for the user as he or she must click a ‘next’ box or the ‘enter’ key between each answer and wait for the next question to appear. In addition, each page provides the response instructions, which can add reading time if a participant re-reads the instructions on each page.

Naïve testers provided feedback on the usability and feel of the internet site. This included navigation, font size, color, consistency of the “style sheet” to maintain cohesiveness of the internet site. Changes made after testing included: 1) redesign of the INSPIRE logo to make it feel more active and energetic; 2) use of brighter colors to give a greater sense of freshness and liveliness; 3) added links within each internet page so that participants could readily skip to lower sections within the page and return to the top of the page. This latter redesign facilitated the flow of the internet site with the goal of encouraging participants to explore the site and content more thoroughly. Stagnancy of the internet site content was addressed during user testing. Testers indicated that internet content, specifically the ‘News You Can Use’ and ‘Our Space’ pages, were the most pertinent sections to be regularly updated.

Tailoring of the internet site was provided through three mechanisms. First, the welcome page always included the participant’s first name. This was based upon research suggesting that personalizing online content increases participation rates in web-based interventions [61]. Second, on first and future logins, the welcome page suggested content specific to deficits identified for a particular participant in their baseline assessment. For this purpose, deficits were defined as follows. 1) Participants were first directed to Renewing Outlook if the HADS Depression score was greater than or equal to 5 or the HADS Anxiety score was greater than or equal to 7. 2) Participants were directed to Restoring Energy if the SF36 Vitality T-score was less than 45 and the HADS Depression score was less than 5 and the HADS Anxiety score was less than 7. 3) If the SF36 Vitality T-score was less than 45 and the HADS Depression score was greater than 5 or the HADS Anxiety score was greater than or equal to 7 participants were directed to Restoring Outlook and Restoring Energy. If participants had no deficits in fatigue, depression, or anxiety they were directed to Boosting Health. Upon next logins sections not yet suggested at previous logins were recommended, until each of the three main sections had been suggested. The final method of tailoring was through email ‘pushes’ sent eight times over the course of the first 12 weeks of the intervention to encourage viewing all the content sections of the site. All participants cycled through an email push directing them to each of the 8 sections of the internet site, beginning with the initial effort focused on areas of need for them.

Internet Patient Portal: Development and Security

The Hutchinson Center’s Clinical Research Division IT group known as Clinical Research Data Systems (CRDS) developed a specific, secured ‘patient portal’ for online research studies to both secure the data entered by participants on the registration and online assessments, as well as to secure other Hutchinson Center data from access through the patient portal. Procedures were closely screened and approved by the Hutchinson Center’s Institutional Review Board. Programming for study registration, online agreement to consent, assessments, scoring of assessments for cut-off determinations, and internet site content access, with individual tailoring of the home page welcome were designed to be entirely automated.

Security

Participants registered for the study on the INSPIRE study secure internet site. Participants’ email addresses were used as their login name which allowed future contacts with study staff and verifiable unique ID in case of duplicate first and/or last names. Issues developed only when participants incorrectly typed their email address, forgot which email address they had used, or changed their email address over the duration of the study.

Balancing security and access in setting password protection parameters was a particular challenge. On the registration page, the participant was required to select a password, which needed to meet the Hutchinson Center password criteria: at least 6 characters with a combination of three of the following four parameters: uppercase, lowercase, number, or symbol. Other required registration information included a security question which could be used if a password was forgotten. We designated a ‘password age’ of 15 months after which a change in password was required. This permitted participants to complete the study without needing to change their password prior to completing the 6-month outcome assessment, and to have time after the study for return to the INSPIRE content pages. This level of security, while not risk free, was judged by the study investigators and Hutchinson Center Information Security Office to be of acceptably low risk while reducing barriers to access to a manageable level.

After confirming all registration information, the participant was directed to a printable page which contained all the information entered during registration. Next, the participant received an email generated by the system and sent to the participant's registered email address. This email contained a link which the user could click or copy and paste into their browser. This link contained a unique key of letters and numbers which identified the participant and activated his or her account once the link was clicked. This step enabled the study internet site administrator to verify that participants each had a valid email address, and that they would receive future correspondence with study staff including survey reminders and content “push” reminder emails. Participants could not proceed until they activated their account by clicking or pasting in the link from the system generated email.

We set two levels of security to meet the competing needs for Protected Health Information (PHI) security and to provide ready study access during the time when no data was collected from the participant other than site pages viewed. The first security level required participants to enter their login and password at each access to the baseline and 6-month assessments, which requested potentially revealing PHI. To protect the privacy of this PHI, no data was stored on the participants’ personal computers. Once participants submitted their assessment responses or a month went by without additional activity, assessments were permanently secured behind a firewall and could not be accessed through the internet even with a password. We initially set this time frame at one week, which proved to be too brief for participants to complete the lengthy baseline assessment. The second level of security was for INSPIRE intervention content access. No participant information was collected from these logins except the pages viewed and time on the internet site. Therefore, this access was deemed to be of little to no security risk, and the priority was to make site connection as easy as possible while restricting access for the control participants. Thus we provided participants with the option to click a check box to ‘remember me on this computer.’ As indicated in the ‘privacy policy’ notice, this placed a ‘cookie’ on their computer and future access to the INSPIRE intervention content from that computer did not require login and password. If users did not place a cookie on their computer or used a new computer, the login name and password were required.

Participants who experienced trouble with navigating the internet site or otherwise had questions could use the “contact us” button available on every page of the internet site to send a message to study staff. They could also call the study toll free line for assistance.

Site beta testing: “break the system”

All steps of the system were tested by at least four staff with a goal of trying every option during registration and each variant related to skip patterns during the assessment. Every complication or incorrect/incomplete response or inconsistency that we could envision of was tried to assure that the procedures ran smoothly and that the automated responses were appropriate to the situation. All links were tested and confirmed, including that internet site links outside of the INSPIRE site were still accurate. In addition, the steps of the system were tested on both PC’s and Mac’s, and on multiple browsers, within and outside the Hutchinson Center domain. This process was lengthy and required numerous programming corrections given the many opportunities for failures or glitches in the system.

Site maintenance

The programming staff in CRDS worked with the researchers to make modifications to the internet site and the application as needed. Furthermore, CRDS insured that all the data, content, and code was backed up in its entirety each evening ensuring minimal loss of data should a catastrophic event occur. The host platform was maintained by CRDS to adhere to all security policies of FHCRC and in accordance with the regulations and standards that all platforms holding patient data must meet.

Data downloading procedures

When surveys were accessed or submitted, data were immediately downloaded to the secure institutional server. The server linked to an ACCESS software platform at its front-end, for study personnel viewing and utilization, leaving the underlying raw data in SQL. In surveys at baseline and 6-month time points, missing data fields could occur as a result of skip patterns based on program specifications (e.g., male questions if a participant was female). Other skips occurred from voluntarily skipped items, or skip programs based on a participant’s previous answer (e.g., a response of ‘none’ to a question about prescribed medications led to skipping a request to list specific medications). Each type of missing data was programmed to be automatically filled with a placeholder value that alerted the data manager to the type of data missing. A randomization form was programmed to be filled by running an ACCESS query using automatically downloaded data from the online survey.

Internet site data usage was captured in the SQL database. Logins captured time and date of login, but did not track use for those who placed a ‘cookie’ on their login page, thus these data were not usable to calculate number of internet site logins. However, each page that was viewed was recorded with user identification including time and date, and provided utilization data.

Pilot Testing

We designated the first 20 randomized participants as pilots. These pilot participants allowed us to work through any glitches in the entire system for registration, enrollment, baseline assessment, randomization, and intervention procedures. This pilot group also verified the accuracy of the automated registration and randomization systems and other procedures prior to official study start-up. Only minor technical problems were found in this initial implementation. Pilot testing did demonstrate the need for email pushes to encourage continued viewing of the INSPIRE site, therefore, this component was added after the pilot period.

METHODS

Participants

Using the transplant center’s research database, all hematologic malignancy survivors between 3 to18 years after their first HSCT at a single transplant center were identified as potentially eligible for this study if they were at least 18 years old and lived in the United States or Canada. These survivors were contacted by sending a letter of approach. Participants were required to have access to the internet and email, either within their own home, at work, at a friend or neighbor’s, or at the library. When possible, those without a personal internet connection or email, who were interested in participating, worked with study staff to find access to the internet or set up an email account. In addition, participants were required to have no known recurrence or second cancer that was actively treated in the previous two years (other than cancer treated only with surgical excision such as basal or squamous cell skin cancer or breast ductal carcinoma in situ), and be able to read and understand English adequate to complete the assessments. Otherwise eligible survivors who reported thoughts of suicide or had major clinical depression as indicated on their baseline assessment received a phone interview from a study psychologist and were excluded from the study randomization but given INSPIRE site access.

Procedure

All materials and procedures were reviewed and approved by the Institutional Review Board at the Fred Hutchinson Cancer Research Center in Seattle. Approach letters briefly explained the study purpose and procedures, and provided recipients with a return envelope and form to complete. On this form they could indicate that they were interested, would like further information, or were not interested. In addition, for those not interested, they could specify whether this was because they did not use a computer or email, or whether they refused for other reasons.

The study provided personalized technical and other support as needed, within one business day of contact, with most service needs met the same day. To meet this aim, survivors were able to submit internet-linked email questions, comments or requests at any time to study staff. Alternatively they could call a toll free line at any time with response to voicemail messages on weekdays within 24 hours and on weekends on the next business morning. The toll free line was staffed during extended business hours on all weekdays to accommodate the variety of time zones and work schedules of participants in the study.

Registration and consent

All survivors interested in signing up for the study registered directly on the INSPIRE internet site. After setting up a study account by choosing a username and password, all interested participants agreed to consent online. During the registration process, participants were prompted to open the consent form and read it before clicking “Yes, I agree to participate”. Participants could also decline by clicking “No, I disagree.” If participants chose that option, they were thanked for their time and no longer contacted for the study. The consent opened in a print-friendly format for participants to print for their personal records. The time and date of participants clicking “I agree” on the consent page was captured in the study tracking database. Those that clicked “I agree” were automatically taken to the start of the baseline survey. The print-friendly consent form was available for study participants at all times via a link at the bottom of each internet page.

Assessments

At baseline and 6-month outcome assessments, participants could skip any questions that were not used for randomization and still remain in the study. The baseline survey took participants an average of one hour but up to three hours for dial-up or slow internet connections or slow responders. Participants were able to log out of the survey and resume where they left off upon the next login. Reminder emails were sent 3 and 6 days after starting the survey if it was not yet complete. The emails were programmed to contain a completion requirement date of one week after the start of the survey. Those that did not finish the survey within this one week time period were sent up to three additional weekly reminder emails and given reminder phone calls. If a participant still did not log on, the Study Coordinator called to find out whether any issues could be addressed to facilitate completion of the assessment.

Randomization and intervention

The design of the study was a randomized controlled trial (RCT). Participants were randomized to one of three groups: internet site access, internet site access plus six to eight phone calls with a psychologist trained in cognitive behavioral therapy and certified in problem-solving treatment, or delayed INSPIRE access (control). Once participants completed the baseline assessment, they received an email indicating randomization status within 24 hours. For those in the control group, they could only access the assessment page of the website to complete the baseline and follow-up assessments, but they did not receive other study content or web-based intervention during the 6 months before completing the follow-up assessment. They were notified both with their randomization email and with a 6-month assessment reminder email that they would have access to the full INSPIRE site once they finished their 6-month assessment. Those with INSPIRE website access were given immediate access to the web content for a period of 6 months, during which reminder “push” emails were sent as described above. Access to the web content was suspended during the 6-month assessment window to encourage participants to complete the assessment and be granted access to the website once again. Before the 6-month assessment window opened, those with website access were given reminders that the assessment was due and access would be terminated until it was completed. Once completed, all groups and arms had access to the INSPIRE website for the duration of the study. Since the focus of this paper is on the internet-specific site design and implementation, we will not focus on the phone-based treatment component of this study in the further description and results for this paper.

RESULTS

Of the 1775 HSCT survivors approached for the study, 1311 were eligible and 755 (58%) enrolled in the study. Characteristics for all survivors approached, according to eligibility and enrollment status are displayed in Table I. Participants ranged in age from 18 to 82 years. Younger survivors were less likely to enroll in the study (P =.006) as were non-white racial groups (P <.001). In addition, those who were longer term survivors (≥10 years after HSCT) were less likely to enroll (P <.001).

Table I.

Characteristics of all survivors approached according to eligibility and enrollment status.

	All survivors approached N=1775	Ineligible survivors (identified before or after baseline assessment) N=464	Eligible based on available data, but declined, did not respond, or withdrew N=556	Eligible survivors consenting and completing baseline assessment N=755	P valuea
Age, Mean(SD)     Range	51.7(13.5) 18.0–82.0	54.1(14.1) 18.2–82.0	49.6(14.1) 18.0–78.1	51.7(12.5) 18.0–78.7	.006
Gender: Male, N (%)	1014(57)	267(58)	328(59)	419(56)	.206
Race, N (%)					<.001
African American	27(1.5)	13(2.8)	9(1.6)	5(0.7)
Asian	60(3.4)	19(4.1)	29(5.2)	12(1.6)
Native American/Alaska Native	22(1.2)	11(2.4)	8(1.4)	3(0.4)
Pacific Islander	1(0.1)	1(0.2)	0	0
White/Caucasian	1517(85.5)	368(79.3)	470(84.5)	679(89.9)
Unknown	148(7.4)	52(11.2)	40(7.2)	56(7.4)
Ethnicity: Hispanic or Latino, N (%)	56(3.2)	26(5.6)	18(3.2)	12(1.6)	.084
Diagnosis, N (%)					.003
Chronic myeloid leukemia	541(31)	120(26)	201(36)	220(29)
Acute leukemia	494(28)	118(25)	149(27)	227(30)
Non-Hodgkin lymphoma	295(17)	84(18)	87(16)	124(16)
Multiple Myeloma	167(9)	68(15)	41(7)	58(8)
Myelodysplasias	144(8)	34(7)	31(6)	79(11)
Hodgkin lymphoma	87(5)	28(6)	33(6)	26(3)
Other	47(3)	12(3)	14(3)	21(3)
Type of transplant:					.590
Autologousb N(%)	508(29)	176(38)	145(26)	187(25)
Years post-transplant, N(%)					<.001
3.0 – 9.9 years	1054(59)	279(60)	291(52)	484(64)
10.0 – 19.0 years	721(41)	185(40)	265(48)	271(36)

^aResult of t-test or chi square comparing those eligible and completing baseline assessment versus those eligible from data known and declining participation or not responding.

^bAutologous transplant recipients receive stem cells from their own blood or marrow stored before treatment; allogeneic transplant recipients receive stem cells from a related or unrelated donor.

Table II displays the characteristics of enrolled patients. The average time from first approach to randomization was 68 days with a wide range from 2 to 379 days. Males were slower to respond and enroll than females (P <.001), as were younger adults compared to both middle age and older adults (P <.02). While non-white survivors were less likely to enroll in the study, those who did took no longer than white survivors to register for the study and complete their assessments.

Table II.

Number of days from first contact to randomization.

	Mean(SD)	Range	P valuea
All	68.0(69.2)	2–379	–
Gender			<.001
Male	75.9(75.2)	5–379
Female	58.1(59.6)	2–323
Age			.001
18–39	86.8(76.3)	5–323	(<.02b)
40–64	66.5(68.6)	2–379
65-oldest	52.4(58.3)	2–282
Race			.83
White, non-Hispanic	67.6(69.7)	2–379
African American	65.0(84.9)	7–209
Asian	67.1(74.0)	6–310
Native American	42.5(46.0)	10–75
Other non-White, Non-Hispanic	89.4(68.7)	7–181
Ethnicity			.006
Hispanic/Latino	109.9(89.1)	7–303
Non-Hispanic/Latino	66.9(68.4)	2–379

^aResult of t-test or one-way ANOVAs comparing groups within each category.

^bP values in parentheses represent significant post-hoc test results after significant ANOVA testing. For age, 18–32 year olds were slower to complete consent and assessment than either other age group.

Although all participation in the study could be done online, most participants (57%) required at least one phone call as part of their enrollment in the study. Twenty-nine percent required 2 or more calls, with the number of calls during which the participant was reached ranging from 0 to 7. Over two thousand phone calls were made successfully, but this represents only a fraction of the total number of phone call attempts since many calls did not connect with a survivor. Study staff completed phone contacts with participants 2154 times prior to randomization (Table III). Contacts by email were even more common, with 3247 emails sent to participants prior to their randomization to the study. Among the 2154 phone calls completed, 1685 (78.2%) and most of the emails were to remind potential participants to visit the study internet site and register for the study.

Table III.

Number of phone calls and emails initiated by study staff during the pre-randomization phase of the study(N=1775).

	Emails	Phone Calls	Total
Pre-Randomization, No.(mean per participant)
Registration	1972(1.1)	1685(0.9)	3657(2.1)
Activation	454(0.3)	84(0.0)	538(0.3)
Start Baseline	381(0.2)	91(0.0)	472(0.3)
Finish Baseline	440(0.2)	294(0.2)	734(0.4)
TOTAL	3247(1.8)	2154(1.2)	5401(3.0)

Few participants initiated calls to the toll free line of the study and these were not tracked for content. Most participants initiated contact through the study email, the ‘Contact Us’ or ‘Comment’ links on the INSPIRE site (Table IV). Technical support questions accounted for 117 of the 193 (60.1%) emails received from study participants. Most of technical problems pertained to login difficulty and forgetting the complex password the participant set up during the baseline assessment period (85.5%). Five participants encountered an internet site timeout while leaving the computer inactive during the process of completing the survey. Despite this timeout, all answers were saved, so logins returned the participants to where they had left off in the survey. All technical issues were fixed within an hour of study staff contact.

Table IV.

Types of emails, ‘contact us’ or ‘comments’ page responses initiated by participants during the pre-randomization phase of the study.^a

		INSPIRE Email	Comments or Questions Submitted on the Website	Total
Technical Support
	Contact participant to correct email address when unreadable	64		64
	Activation link or consent login problem	20		20
	Name and password problem	16		16
	Survey problem	3	3	6
	Email address changed	3		3
	Email confirming consent	3		3
	Request to resend site link	2		2
	General technical questions	1	2	3
Technical Support Total		112	5	117
General Study questions
	Eligibility questions	10	4	14
	Explain reason for doing study	2		2
	What participants have to do	2		2
	General questions	2	5	7
General Study Questions Total		16	9	25
Clinical Questions Total		0	8	8
Specific Comments
	Refuse/withdrawal emails	23		23
	Relapse/withdrawal emails	3		3
	Survey too long complaints	5	5	10
	Survey wording complaints	3	1	4
	General complaints, Opinions	2	1	3
Comments Total		36	7	43
TOTAL		164	29	193

^aPhone calls initiated by potential participants were few and not recorded.

Due to differing personal email security settings and a variety of email providers, for some participants, emails went into their “junk” mailbox. This caused some participants to not see the activation code email or reminder emails, which prolonged the time taken to complete the registration and/or baseline survey process. If study staff received no response to these emails, they would call after three emails to ensure the participant had received them.

DISCUSSION

This description of the development and implementation of an online survivorship care study for long-term HSCT survivors demonstrates the components needed and potential for internet-based programs for survivors. The INSPIRE study demonstrated that study design, content writing, programming, security and implementation of the intervention could all be accomplished successfully with contributions from diverse disciplines and strong internet technology collaborations. With extensive preliminary work, implementation was smooth and a large number of participants were enrolled within a relatively short time frame (18 months).

This study had a higher rate of enrollment (58%) compared to many health targeted internet clinical trials previously reported [35–39]. However, to reach these high enrollment completion rates, extensive effort was required for up to a year to enroll all participants who expressed interest but did not get through the consenting and baseline assessment process despite indicating interest and intent to do so. The large number of emails and phone calls that were necessary to engage and remind potential study subjects highlights the need to plan adequately for staff to follow up with participants to achieve optimal enrollment rates. In this study, 57% of participants required at least one contact, and nearly a third required multiple contacts. Although 60% of contacts initiated by survivors were related to technical issues such as forgetting passwords, most contacts initiated by the study staff were not related to technical issues, but instead were required to encourage enrollment and assessment completion. Unfortunately, despite the attractiveness of email alone as a method of participant communication due to its relative speed and efficiency over phone calls, we found that phone calls were also required to encourage enrollment and completion of assessments.

Once designed and thoroughly tested, technical issues were relatively infrequent. However, rapid response to emails requesting assistance required dedicated study staff to work through any technical issues with study participants as well as to communicate with internet technology specialists to fix any issues.

This study has both strengths and limitations. In strengths, all survivors were screened for eligibility and approached who were between 3 and 18 years after an HSCT at the single transplant center participating in the study. Enrollment rates were high, reflecting persistent, concerted efforts to enroll as many survivors as possible. Limitations of the study included enrollment by survivors from only a single transplant center, with one of the most robust long term follow up programs worldwide. Survivors from this transplant center have access to a long-term follow-up program for the rest of their lives, although no psycho-educational efforts directly target distress, depression or fatigue and no internet-based materials are offered. However, survivors are sent annual guidelines for health care surveillance. It is possible, given the ongoing resources available to the cohort enrolled in this study, that needs could be greater and enrollment rates could differ at sites that maintain less regular contact with their survivors. As in all internet based interventions, participation was limited to those who have internet access. Further, the INSPIRE study was offered in English only, which excluded non-English speaking survivors.

The INSPIRE study demonstrated the potential for providing internet-based survivorship care to long-term survivors of HSCT, and may be scalable to survivors of other cancer treatments. Although creating a survivorship internet site requires a team with diverse expertise to build, and a technical and clinical research team to maintain ongoing problem-solving for both technical and clinical questions, the study can be implemented rapidly with a large number of survivors, with a broad geographic reach. Security for personal health information during assessments can be balanced with the provision of ready, unrestricted access to the internet site content. The INSPIRE project used similar techniques as other behavior change websites to engage participants and keep the website interesting and fresh, including tailoring, push emails, updated ‘News You Can Use’ and ‘Our Space’ comments’ [35–41]. Our procedures for the development of the website were similar to those used by Mayer and colleagues [62]. However that group conducted a needs assessment with parents of HSCT patients to develop content. The next research with the INSPIRE site will include usability testing with survivors. The feedback from participants provided throughout the current study, also will be an invaluable asset to the further development of the INSPIRE website.

The INSPIRE website is now being tested in a dissemination project for a nationwide access through transplant survivor websites. We also plan an effectiveness clinical trial in which access is coordinated by specific transplant centers, with content specific to the transplant center resources for their survivors, and with enhanced tailoring, video features and social networking. A strength of using the internet as an information based communication modality in this population is that content can be easily updated as new evidence and recommendations become available. While internet-based services will not meet all the needs of cancer survivors, this methodology represents an important tool for augmenting onsite clinical services as a method for meeting educational and resource needs of cancer survivors.

Figure 2.

Sample screen shots of the INSPIRE website.