Challenges and Solutions for Using Informatics in Research

Abstract

Computer technology provides innovations for research but not without concomitant challenges. Herein, we present our experiences with technology challenges and solutions across 16 nursing research studies. Issues included intervention integrity, software updates and compatibility, Web accessibility and implementation, hardware and equipment, computer literacy of participants, and programming. Our researchers found solutions related to best practices for computer-screen design and usability testing, especially as they relate to the target populations' computer literacy levels and use patterns; changes in software; availability and limitations of operating systems and Web-browsers; resources for on-site technology help for participants; and creative facilitators to access participants and implement study procedures. Researchers may find this information helpful as they consider successful ways to integrate informatics in the design and implementation of future studies with technology that maximizes research productivity.

Introduction

The growth of computer technology includes its use as a research tool. Optimal application of computer technology as a research tool requires nurse researchers with informatics expertise (Gugerty, 2006). Web-based studies have proven feasible and acceptable for allowing study access by diverse populations (Hamel, Robbins, & Wilbur, 2011; Moloney et al., 2009). Some challenges and/or lessons from informatics-assisted studies have been reported by nurse researchers (Bond, 2006; Conn, 2010; Duffy, 2002; Loescher, Hibler, Hiscox, Hla, & Harris, 2011; Moloney et al., 2009) as well as researchers in other fields (Adaji, Schattner, & Piterman, 2011; Chen & Goodson, 2010; Danaher & Seeley, 2009; Hamel, Robbins, & Wilbur, 2011; Suarez-Balcazar, Balcazar, & Taylor-Ritzler, 2009). This manuscript presents real-life challenges and solutions encountered in informatics-assisted studies by one research–intensive college of nursing. At the University of Illinois at Chicago College of Nursing (UIC CON), our researchers use a variety of informatics strategies to deliver interventions and aid in data collection. The Nursing and Health Informatics Core of the Center of Excellence for End-of-Life Transition Research facilitates CON investigators’ use of informatics in their studies. As part of the Core’s work, CON faculty were polled to identify recent informatics challenges and solutions in their research.

Challenges and Solutions

Table 1 presents a synopsis of the studies in which investigators encountered informatics-related issues for which they and their teams devised solutions. In this article, problems and solutions are outlined related to the following issues: intervention integrity, software updates and compatibility, Web accessibility challenges, hardware and equipment challenges, computer literacy of participants, and programming challenges.

Table 1.

Studies in which Investigators encountered Informatics-Related Issues

Study	Purpose	Design	Sample	Informatics Measure or Methods	Issues
CHOICES: Tailored Education for Informed Reproductive Decisions by People with Sickle Cell Disease or Sickle Cell Trait (1U54 HL090513; 1R01 HL114404-01) **	Compare usual care (e-Book) and CHOICES for effects on changes in reproductive health knowledge, and intention and behaviors to implement an informed parenting plan	Randomized clinical trial with 24-month follow-up	234 Chicago-area adults 18–35 years of age with sickle cell disease or sickle cell trait, able and desiring to have children	Internet-based application delivered as local host application on Pentablet for participant to complete pre- and posttest data collection and engage the multimedia intervention or e-Book	Ineligible volunteers from all over the US identified from a Craigslist posting
FPS - Fertility Preservation Study	To describe the decision making process among young women newly diagnosed with cancer regarding their decision to use novel fertility preservation techniques (oocyte and embryo cryopreservation) for use post cancer therapy	Grounded theory, qualitative approach	27 young adult women diagnosed with cancer of which 74% were recruited from Web strategies and 26% were recruited from two large academic centers	Development of a Web site and identification of key conduits for announcing the study and liking to the Web site. In-depth semi-structured interviews were completed with each participant by phone (n = 21) or e-mail (n = 6)	Establishing relationships and obtaining access from Web gatekeepers, stakeholders, and Web site managers took a significant amount of time and was challenging
HANDS: Deployment and Testing of a Careplanning Software that Included the Standardized Nursing Terminologies of NANDA-I, NOC, and NIC *	To test the hypothesis that HANDS can be implemented uniformly in different hospitals and result in positive RN attitudes about the system and produce interoperable data.	Longitudinal	From a representative sample of 4 different hospitals in one Midwestern state a sample of 8 medical surgical units who met specified criteria and agreed to implement HANDS for 12 or 24 months	Informatics applied at three levels: (a) The intervention involves the integration of standardized terminologies into a software application and its database; (b) it is deployed via the Web from offsite secure servers; (c) accessed by users through a Web browser link that is made available through a hospital portal.	Need for hospitals’ support’, unique electronic hospital infrastructures and version of software (e.g., browsers); lack of feasibility of tailoring intervention to all hospital specifications
HFSx - Symptom Clusters in Patients with Severe Heart Failure+	To determine symptom clusters that occur in persons with severe heart failure and determine if these clusters prognosticate end of life	Longitudinal cohort study	Persons receiving treatment for heart failure at two different clinics in a major Midwestern city were included if their health care provider stated that they would not be surprised if the patient died in the next two years (N = 194)	Patients completed a Web-based comprehensive symptoms questionnaire and performed a Q-sort of symptoms using WebQ software at baseline and every 3 months for 1 year	Newer versions of Flash Player Automatic delivered via automatic updates were not compatible with WebQ software. Programming, data retrieval.
KHASS - Knowledge of Heart Attack and Stroke Symptoms	To deliver a tailored intervention to a group of at-risk males to assess their knowledge of Heart attack and stroke symptoms	Cross-sectional pretest-posttest	Low-income males at risk for heart attack and stroke (N = ?).	Pretest, tailored intervention based on responses, posttest	Programming, data retrieval.
MAS - Effects and Costs of Massage in Hospice Cancer Patients (1R01 NR009092)*	Compare usual hospice care and usual hospice care plus five daily massage therapy sessions for effects on patient symptom outcomes and costs in a diverse sample of hospice patients with cancer	Randomized clinical trial	193 patients with cancer receiving hospice or palliative care from one of three home hospice programs, having daily pain or taking daily analgesics and Palliative Performance Scale score of ≥ 40	Pentablet-based pain and symptom assessment by patient daily in their homes, massage therapy documentation, and blinded pre and post massage pain reports	Equipment theft. Subject fear of monitoring. Computer literacy.
(Mzanga) - Behavioral Change in the Mzanga Samala Moyo Wako Intervention**	To collect sensitive sexual behavior information from young rural women before and after an intervention to increase reproductive health	Cross-sectional pretest-posttest	Young women in rural Malawi (N = ?)	Audio-assisted intervention presented	Physical stability of computers, lack of tech support, recharging challenges
OC - Oral Conditions+	To examine the presence, severity, and functional impact of oral health conditions in terminally ill patients with advanced cancer receiving home-level hospice care	Cohort study
PASCC - Physical Activity in Survivors of Childhood Cancer	To examine correlates of participation in regular physical activities among young adult survivors of childhood cancers	Cross-sectional Web-based survey	117 long-term survivors of various types of childhood cancers recruited from throughout the United States through advertisements posted on cancer survivor-related Web sites, newsletters, listservs, a study Web site, word of mouth, specialty cancer survivor clinics, and cancer camps.	Self-developed Web-based survey that included four rating scales (physical activity stages of change, autonomous motivation, physical activity pros and cons, self-efficacy, and self-reported worries), a stages of change measure, and background questions	The survey was programmed to work only in one Web browser (Internet Explorer).
PAT2D - Physical Activity in Type 2 Diabetes	To use real-time technology continuously over six days to measure physical activity, fatigue, blood glucose, and self-care behaviors along with retrospective measures associated with physical activity	Pilot cohort study with data collection over six days	Eight men and women aged ≥ 45 years with T2DM treated at a VA hospital in a large, Midwestern Metropolitan area.	Continuous glucose monitoring of interstitial fluid collected with CGM iPro and real-time physical activity monitoring with Respironics Actiwatch	Software for the CGM iPro is currently only compatible with Windows XP® or earlier operating systems.
PGD - Preimplantation Genetic Diagnosis	To describe the decision-making process among genetically at-risk couples regarding their decision to use Preimplantation Genetic Diagnosis (PGD) to prevent the transmission of known genetic disorders to their future child(ren)	Grounded theory, qualitative approach	22 genetically at-risk couples (44 individual partners) of which ~82% of couples were recruited from Web strategies and ~19% of couples were recruited from traditional strategies (i.e., large PGD center and a patient-focused newsletter)	Development of a Web site and identification of key conduits for announcing the study and linking to the Web site. Private, in-depth semi-structured interviews were completed with each individual partner within the couple dyad, separate from their respective partner, by phone (n = 28 individual partners) or e-mail (n = 16 individual partners).	Establishing relationships and obtaining access from Web gatekeepers, stakeholders, and Web site managers took a significant amount of time and was challenging
PR-CA - Computerized PAINRelieveIt Protocol for Cancer Pain (2 R01 CA081918) *	To compare usual care and PAINRelieveIt for effects on patient cancer pain outcomes and clinician outcomes	Randomized clinical trial with pre- and 4-week posttest measures	168 patients with advanced-stage cancers receiving radiation or chemotherapy or ongoing cancer care, pain ≥ 3 on 0–10 scale	Pentablet-based pain assessment completed by patient, tailored multimedia patient education, and just-in-time printed decision support for providers	Wireless connection issues for a non-Web-based application
PR-SC - Computerized PAINRelieveIt for Adult Sickle Cell Disease (1R01 HL078536) *	To compare usual care and the PAINReportIt/PAINUCope or PAINConsultN for (1) short-term effects on patient pain outcomes and (2) long-term effects on (a) pain episodes, (b) ED and inpatient provider pain documentation and appropriateness of prescribed analgesics, and (c) number of ED visits and hospitalizations for painful SC crisis)	Two randomized clinical trials to test (1) education at 3 months and (2) decision support for 24 months	279 adults with sickle cell disease, SCD pain =>3 on 0–10 scale within the last 12 months, had an ED visit or hospitalization within the previous two years	Pentablet-based pain assessment completed by patient every 3 months in the outpatient clinic, each ED visit, each Day Hospital visit, and daily while inpatient, tailored multimedia patient education, and just-in-time printed decision support for providers in acute care settings	Amount of data collected exceed capacity of the MS Access database application. Equipment theft.
RCP - Relieve Children’s Pain **	To provide a pain management educational intervention, and to collect information before and after the intervention to evaluate the feasibility of the intervention and study procedures	Quasi-experimental one-group pretest-posttest design	Registered nurses caring for pediatric patients who experience pain	Potential participants contacted investigator via e-mail and Web page used for consent, pretest, intervention, and posttest	Portions of Web-based intervention not compatible with Macintosh browser
ROLU - Reaching Out, Lifting Up: The Truth about Depression and Suicide+	To compare suitability ratings, determine cultural effects, and explore suggestions about changes or additions for a well-accepted suicide awareness video by Asian American and non-Hispanic White college students	Cross-sectional pretest-posttest	Convenience sample of 736 college students at one Midwestern University who responded to a call for participation via announcements on university announcement page; request to enroll sent to student organizations, or in-class announcements	Pre-assessment screening, registration, consent, baseline assessment streaming video intervention debriefing and post assessment--all study procedures completed via Web site	Participants signed in and registered but did not watch video prior to post assessment
					University streaming server deactivated as application was being tested
STHSCT - Strength Training following Hematopoietic Stem Cell Transplantation.	To test the effects of strength training compared to usual activity on physical activity, muscle strength, fatigue, health status perceptions, and quality of life following hematopoietic stem cell transplantation To further evaluate the feasibility of the revised strength training intervention by determining the week-to-week strength training frequency of study participants.	Two-group, randomized block design	Nineteen subjects randomly assigned to strength training (n = 9) or usual activity (n = 10)	Physical activity was objectively measured using a wrist-worn accelerometer, the Actiwatch Score® (Phillips Respironics, Bend, OR). The subjective event marker of the Actiwatch Score® was programmed to be used as a single-item, global, self-report scale to measure real-time fatigue intensity.	Subjective event marker redesigned with newer model of Actiwatch Score. Older version of Actiwatch-Score not compatible with upgrades in Actiware software.

Note.

⁺P30 Pilot Project.

^*P30 Research Base.

^**Used P30 Nursing and Health Informatics Core Services.

Intervention Integrity

Compliance with surveys and interventions may be an issue due to the anonymous nature of Web intervention and data collection. In the ROLU study (Table 1), key challenges were (1) participants completing the pre and post video assessments but not watching the entire video and (2) changes in the University streaming server. Solutions to these problems were to:

• stress, when promoting the study to prospective participants, that the entire video needed to be watched to earn the incentive;

• add a pop-up message at the top of video screen that stressed the importance of watching the entire video, informed the participant that the page would be monitored, and asked for the participant’s help by their watching and listening to entire video. This message was repeated if the participant tried to complete the post assessment without watching the video;

• add a new function: if the participant did not watch the entire video and clicked the next page to advance to the post assessments, a pop-up informed them that they had the choice of withdrawing from the study and not receiving an incentive or going back to complete viewing the video and be eligible for the incentive; and

• analyze the manipulation check questions to ascertain whether participants seemed to have watched and paid attention to the video.

The function that documented the amount of time that the participant remained on each page was another issue. Researchers captured the time from the participant’s computer (client) rather than from the server and found that some participants spent an unreasonable amount of time completing the study (several days). Researchers learned that some college students use an application that changes the time on their computers so that they can get extra time to play computer games. To overcome this problem in future studies, our researchers recommend that the time be captured from the server rather than from the client.

A final issue was the deactivation of the university-supported streaming server just as the study was in the final testing phase. Without a streaming server, the participants would not be able to view the Web-based video. Researchers solved the problem by dividing the video into three parts, each 10 minutes or less in length, and posting them to a private channel on YouTube. This solution prevented non-participants from gaining access to the video and allowed the participants to watch each of the three parts without leaving the single video screen.

Software Updates and Compatibility

The HFSx study (Table 1) investigators used a self-designed symptoms inventory tool and a symptoms Q-sort instrument administered via FlashQ, a Web-based Q-sorting program. The FlashQ program was built based on a specific version of RealPlayer media player, and automatic updates delivered to the study pentablet computers after the study was initiated made the program nonfunctional and inaccessible to the study because FlashQ could not run the updated version. To solve this problem, the study pentablets were configured to return to the version of RealPlayer that was compatible with the FlashQ program and to block future automatic updates.

The PAT2D study (Table 1) used software for the CGM iPro for continuous interstitial glucose monitoring that was only compatible with Windows XP^® or earlier operating systems, and the agency where data collection was taking place upgraded its entire computer network to newer operating systems. An update was not available, and an alternate instrument was not identified. Therefore, an older computer with Windows XP^® software was retained to process the study data.

In the STHSCT studies (Table 1), upgrades to the Actiwatch Score (an accelerometer with a subjective event marker) and Actiware (the software that analyzes the data from the Actiwatch Score) presented challenges. During the course of the study, additional Actiwatch Scores were needed to accommodate the growing enrollment. The subjective event marker, however, was upgraded, and the differences in the visual aspect of the subjective event marker had the potential to influence real-time data collection. A software upgrade to the Actiware further complicated matters because the older model Actiwatch Scores were not compatible with the new software. Instead of mixing devices with two different subjective event markers in the study, additional older model Actiwatch Scores were borrowed from a colleague to maintain consistent real-time data collection across all subjects. The older software was used to program the Actiwatch Scores and download data was stored on the device. The data were then imported into the upgraded Actiware software, which allowed use of the enhanced data analysis components.

The PR-SC (Table 1) study team encountered a software storage capacity problem with Access™ (Microsoft, Redmond, WA). The study included 27 months of follow-up data collection. Given the massive amount of data collected, the database exceeded the software's maximum size (2 GB) after four years. To solve the problem, the team exported the image files to an archive database and restored capacity to collect data. For future longitudinal, repeated-measures studies, the investigators plan to use a SQL (Structured Query Language) database instead of Access™ to avoid such space issues.

Open source versions of SQL are available

Two studies encountered issues related to use and understanding of software packages by participants. The RCP study (Table 1) researchers noted that participants using non-Windows-based computers could not access parts of the Web-based intervention. In the pilot study, participants were required to find a Windows-based computer to participate. In the subsequent study, two Windows-based computers were purchased for participants who did not have access to one.

Researchers in the PASCC study (Table 1) used a self-developed, Web-based survey instrument administered to a nationwide sample of young adults. Although the instructions specified what Web browser (Internet Explorer) participants should use, some participants were unclear what a Web browser was. Researchers communicated by e-mail with participants who had problems, and also had phone conversations with some participants to clarify this issue.

The HANDS study (Table 1) researchers learned that addressing all possible technical contingencies is not feasible when deploying an electronic application via the Web to multiple hospitals because each health care organization typically has its own unique electronic infrastructure. Because it is essential that informatics interventions be studied under real-time conditions and beyond a single site to demonstrate generalizability, the researchers approached individuals in institutions who had in-depth knowledge of the health care organization’s technical infrastructures to help determine a common set of constraints that could be implemented simply and at little cost. As a result, those researchers were able to deploy HANDS in each study institution without adapting the software. The main lesson learned was that it is important to talk with parties within institutions who have expertise and are in a position to help design a feasible solution before approaching administrators.

Web Accessibility Challenges

The HFSx study (Table 1) researchers took pentablet computers to two hospital-based sites to have subjects complete Web-based instruments. At one clinic, the firewall blocked transmission of the data to the server located three blocks away. Negotiation with the Information Technology Department allowed the hospital to grant a firewall exception so that data could be transmitted to the server. At the second data collection site, wireless access was not available. The researchers initially tried an external wireless access device that operated like a cell phone, but interference related to the building and hospital equipment made wireless access with the device unreliable. The solution to this issue involved connecting the laptops to the Web via a cable located in the exam rooms where data collection occurred.

The OC study (Table 1) team collected data at patients’ homes, where Web access was often not available. The researchers anticipated this challenge and advised the programmers to save data onto the computer’s hard drive rather than entering data through a Web-based data capture system. After the study visit, the data were downloaded onto a secure data storage system.

The PR-CA (Table 1) investigators planned to avoid daily synchronization of multiple pentablets by writing data to a secure local area network using the wireless capacity. Unfortunately, wireless connections do not provide instantaneous access. There can be millisecond periods of time that the wireless connection is unavailable and automatically restored, and if the software attempts to write data at that precise moment, the software does not work properly. The financially feasible solution was to abandon the wireless connection and use a cable to connect to the secure local area network. In the PR-SC (Table 1) study, however, cable connections were not available at the data collection site, which meant that frequent synchronization was required. Now, the software is available as a Web application with data-writing requirements accounting for the frequent break in signal access that is a function of wireless connections.

The CHOICES (Table 1) study investigators posted the study advertisement on the Chicago craigslist Web site, but volunteers responded from across the United States. Because the IRB approved the study only for recruitment from the Chicago area, these volunteers were ineligible. The investigators, however, recognized that the study procedures could be adapted to a Web-only recruitment process. They applied for additional funding to conduct a multi-city pilot study using only Web-based recruitment and retention strategies that would potentially reduce costs.

The PGD and FPS studies (Table 1) developed and implemented informational Web sites to augment Web recruitment (Hershberger et al., 2011). One challenge that researchers in these studies recognized was the need to identify and work with moderators or gatekeepers of Web communities to gain access to these communities for recruitment purposes. Our researchers recommend the following actions to prevent or alleviate issues related to the relationship among gatekeepers, Web site managers, and investigators:

• Use clear and respectful language in all e-mail and phone communications.

• Establish a communication log to track dates, times, and text in e-mails, and (if appropriate) phone conversations. Include a calendar in the log to facilitate re-contacting individuals at appropriate intervals if e-mails or calls are unanswered.

• Provide gatekeepers and stakeholders with information about the high quality of the study and the credibility of the investigators early in the process. Provide examples of published research articles or other examples of the investigator’s activities to improve the health of the target population and also demonstrate sensitivity toward the population.

• Consider adding a “virtual” community partner to the research team to screen the language of announcements on Web sites, discussion boards, or other Web media for sensitivity to the target population and also to enhance engagement with the research study.

Hardware and Equipment Challenges

The Mzanga (Table 1) study was conducted in rural Malawi, which presented challenges of hardware stability and power. The investigators purchased sturdy Panasonic Toughbook model computers and purchased a back-up battery for each computer.

The MAS (Table 1) investigators encountered two equipment-related issues because computers remained in the homes of participants (Wilkie et al., 2009). Some subjects were suspicious of the green power-indicator light, stating that they did not want “Big Brother” watching them. The researchers solved the problem by placing the pentablet in a drawer so that the light was not visible and the participant removed it to complete the daily questionnaires. Although the investigators insured the pentablets against theft, one was stolen (by a non-participant). In anticipation of such an issue, the investigators protected the database with a strong password and encrypted all HIPAA-protected data as they were written to the database.

Computer Literacy of Participants

The issue of computer literacy presented challenges to several of the study investigators in this paper. In both the HFSx study and the OC study (Table1), many participants were older and unfamiliar with computer usage. Consequently, many participants did not choose to use the computer-based data entry system. Some patients had limited dexterity and were physically unable to use the mouse or mouse pad or touch screens on the laptop computer to enter data, and some had low literacy that prevented them from being able to read the questions on the screen. As a solution to these issues, a family member who was present or a trained research assistant entered the patient’s responses into the computer-based data collection instruments.

Programming Challenges

Researchers in the KHASS study (Table 1) envisioned an intervention tailored to subjects’ responses to previous questions that progressed through specific educational content. These researchers found it painstaking to anticipate all possible scenarios and to work with the programmer to establish this progression. Researchers in the PR-CA and PR-SC studies worked with a system analyst to develop tailored interventions prior to giving specifications to the programmers.

Researchers in the HFSx study (Table 1) carefully diagrammed the screens that they wanted for their instruments, but programmers independently inserted color. Unfortunately, the programmers elected to use red for the answer that would be most desirable, thus cueing subjects to choose that response. Colors were subsequently changed to be equally appealing for all possible responses, thus avoiding undue influence on the responses.

Two studies illustrated that assuring that data were appropriately written to the database prior to starting data collection was essential. Researchers in the KHASS study (Table 1) began to analyze the data and discovered that the data to three key questions had not been written to the database. Unfortunately, the data could not be retrieved, and more subjects needed to be recruited into the study to collect the necessary data. Researchers in the HFSx study (Table 1) found that data from the Q-sort portion of the data collection instrument were written to the database, but in a format that could not be imported into the data analysis program. The researchers contacted the creator of the program and were told that there was no solution and that data would need to be entered manually. Fortunately, a programmer with the Informatics Core wrote another program that successfully transferred the data. These researchers stressed that thorough pilot testing including analysis of pilot data is necessary to assure that all programming functions correctly.

Discussion

A variety of challenges to informatics-assisted research have been reported previously, and our experiences support these and suggest additional challenges. Obtaining a representative sample, correct contact information, and successful delivery of study materials have been reported as key elements to prevent sampling errors when recruiting potential respondents over the Web (Chen & Goodson, 2010; Duffy, 2001). Limiting recruitment to a specific geographic area may be challenging because the Web is widely used. This issue was demonstrated in the CHOICES study when volunteers responded from across the country even though the recruitment materials were posted on the Chicago craigslist Web site. If the sample frame does not match the target population, results are not meaningful due to unrepresentative sampling issues (Chen & Goodson, 2010; Duffy, 2002); therefore, the researchers in this study expanded the recruitment to include a wider geographic area.

While Loescher et al. (2011) reported that lost log-in passwords to access Web-administered questionnaires/programs could decrease the response rate, we did not have direct experience with this challenge. Another challenge identified in the literature was that study e-mails could be identified as spam or junk mail (Chen & Goodson, 2010) and attached e-mail files could not be opened by the intended user (Loescher et al., 2011). We also did not experience this challenge because our study materials resided on separate Web sites or servers.

Contacting some subjects (e.g., members of professional organizations/groups) by e-mail has been identified as more difficult than most researchers expected because it took time to collect their e-mail addresses (Chen & Goodson, 2010). While we did not experience this problem, our researchers have noted that working with “gatekeepers” may be challenging but necessary to gain access to sensitive populations. Tracking how many e-mails are opened and read by participants (cited as a problem in a previous study) (Loescher et al., 2011) is now possible through common software.

Environmental factors such as technical compatibility and users’ computer/Web literacy affect validity of Web-based studies. User literacy was particularly prevalent in several studies, especially studies recruiting older subjects or economically disadvantaged subjects. Compatibility of operating systems (PC vs. Apple) presented a technical challenge that was solved by providing specific equipment. We also noted that transmission of data to servers was a challenge due to firewall issues, signal interference, and how data were transmitted to servers.

Computer errors and inconsistent Web access can decrease willingness among participants to complete questionnaires or a study (Conn, 2010). Our researchers have provided some solutions to this issue. In Chen and Goodson’s study (2010), those researchers had to change their software package to personally guide some participants on how to link to or fill out the Web-based questionnaires or both, and to assure participants about the anonymity of their responses. Although our researchers also encountered this issue, our researchers found that family members or trained research assistants could provide on-site assistance. In another study, some participants had problems viewing a video because the Web service was not fast enough to keep them watching the video (Loescher et al., 2011). Our researchers experienced challenges with the Web, but they were able to transfer our video interventions to YouTube, which is widely available, reliable, and free. Limited familiarity with Web technology, especially among low-income populations, is recognized as contributing to participants’ experiencing frustration as they search for information about Web-based studies, and such frustration can decrease the response rate (Suarez-Balcazar et al., 2009). As the Web becomes increasingly familiar to more people, Web-based studies are not only increasingly possible but in some cases superior to traditional methods.

Our biggest category of challenges related to software updates. We found that it is critical to be aware of the software necessary to run various data collection and data analysis programs and be able to revert back to systems available before updates in order to continue to collect and analyze data.

It is also crucial to consider acceptability and usability of Web technologies among target populations (Sandars & Lafferty, 2010). Our experiences indicate that pilot testing increases the usability of Web sites by identifying and resolving possible difficulties in advance. Large fonts and icons and designs tailored to older adults with chronic illness make Web-based health care interventions more feasible and acceptable among the elderly (Bond, 2006). Chen and Goodson (Chen & Goodson, 2010) suggested that use of such tailored formatting to increase relevance to participants in Web studies requires proper pre-testing and cautious application. Our researchers concur and strongly recommend that researchers work closely with programmers and informatics experts, conduct usability testing (Wilkie et al., 2003) with cognitive interviews and think-aloud approaches (Jha et al., 2010), pilot test interventions, and test data retrieval prior to launching any studies that use Web technology. Well-designed screens with sufficient usability testing with the target audience will reduce the impact of Web literacy issues, and newer technologies can accommodate for reading problems such as eyesight issues or inability to read.

Application of Web technologies requires adequate financial support and qualified and resourceful human resources. When financial support is limited, one strategy is to seek computer equipment from a university surplus division, library system, or local charitable and non-profit organizations or to use a rebuilt product (Adaji et al., 2011; Bond, 2006).

Other financial costs include the incentives to motivate participants to initiate, continue, and complete Web-based studies (Gajic, Cameron, & Hurley, 2011; Suarez-Balcazar et al., 2009). Our researchers found, however, that, when using financial incentives, it is necessary to closely monitor participation and responses to assure that subjects complete the required intervention prior to applying for the incentive. Use of technology opens doors for nursing researchers in areas such as access to subjects, delivery of interventions, direct methods of data entry, and expeditious data analysis. Using new technologies can also create some unexpected challenges. Researchers need to be aware of best practices for computer-screen design and usability testing, especially as they relate to the target populations' computer literacy levels and use patterns; changes in software; availability and limitations of operating systems and Web-browsers; resources for on-site technology help for participants; and creative facilitators to access participants and implement study procedures. Researchers can use the information in this article as they consider successful ways to integrate informatics in the design and implementation of future studies with technology that maximizes research productivity.

Table 2.

Challenges and Solutions of Incorporating Informatics into Studies

Challenge	Solution
Intervention Integrity	Stress importance of completing intervention both during recruiting and during intervention Analyze data to assess if participants completed the intervention Capture participation time from server not subject Consider using YouTube for video content
Software Updates and Compatibility	Configure computers to block automatic updates Keep compatible equipment throughout the entire study Upload data from older programs to newer programs for enhanced functionality Consider storage capacity; open-source SQL is available Be prepared for incompatibility of PC vs. Apple operating systems Develop mechanism to coach participants with questions Engage persons who are experts in technical capability of participating organizations
Web Accessibility Challenges	Request firewall exceptions if needed Hardwire computers if wireless is not available Save data to local hard drive Avoid wireless connections for data transfer Consider Web recruitment Use software specifically designed for wireless connections Use clear and respectful language in all e-mail and phone communications to gatekeepers Establish a communication log to track details of all communication Provide gatekeepers and stakeholders with information early in the process Consider adding a “virtual” community partner to the research team
Hardware and Equipment Challenges	Have back-up battery when power source is inconsistent Keep computers out of sight Use strong passwords and encryption
Computer Literacy of Participants	Have a family member or trained research assistant enter the patient’s responses into the computer-based data collection instruments Work closely with programmer to create instruments or computer screens Check data written to database as part of pilot testing
Programming Challenges	Include system analyst, informatics researcher as consultant or investigator as member of the research team Conduct standardized testing, pilot testing with analysis of data to assure all programming functions