Abstract
Clinical trials have been slow to incorporate e-technology (digital and electronic technology that utilizes mobile devices or the Internet) into the design and execution of studies. In the meantime, individuals and corporations are relying more on electronic platforms and most have incorporated such technology into their daily lives. This paper provides a general overview of the use of e-technologies in clinical trials research, specifically within the last decade, marked by rapid growth of mobile and Internet-based tools. Benefits of and challenges to the use of e-technologies in data collection, recruitment and retention, delivery of interventions, and dissemination are provided, as well as a description of the current status of regulatory oversight of e-technologies in clinical trials research. As an example of ways in which e-technologies can be used for intervention delivery, a summary of e-technologies for treatment of substance use disorders is presented. Using e-technologies to design and implement clinical trials has the potential to reach a wide audience, making trials more efficient while also reducing costs; however, researchers should be cautious when adopting these tools given the many challenges in using new technologies, as well as threats to participant privacy/confidentiality. Challenges of using e-technologies can be overcome with careful planning, useful partnerships, and forethought. The role of web- and smartphone-based applications is expanding, and the increasing use of those platforms by scientists and the public alike make them tools that cannot be ignored.
Keywords: clinical trials, e-technology, social media, apps, smartphones, internet
1. INTRODUCTION
Historically, clinical trial activities, including recruitment, retention, delivery of interventions, and data collection, have been conducted using conventional “face-to-face” approaches. For example, newspaper or radio advertisements are used to recruit participants, mail or telephone calls are used to conduct follow-up assessments, interventions are delivered in person, and data are collected using paper-and-pencil instruments. Clinical trials have been slow to incorporate e-technology (i.e., digital and electronic technology that utilizes mobile devices or the Internet) into the design and execution of studies (Riley et al., 2013; Baker et al., 2014) and are challenged to keep pace with fast-moving developments in technology. For example, in the time it takes to design, implement, and publish findings from a research study (approximately 6 years), the world went from playing interactive video games (Wii) to using voice-activated personal assistants (Siri) (Riley et al., 2013). Also, during this timeframe the Apple “App” store added approximately 1 million apps for the iPhone. Additional reasons for the sluggish adoption of e-technology include the limited empirical evidence on whether e-technologies improve or enhance the design of clinical trials and the paucity of regulatory guidance and policies, particularly when FDA-approval is required.
Study teams are, however, incorporating more e-technology into their study designs, perhaps most importantly because mobile technologies and Internet-based communication are becoming the new norm for patients. The culture of communication has changed as potential research participants integrate the Internet, smartphones, and social media into all aspects of their lives. According to recent surveys conducted by the Pew Research Center and summarized in Table 1, approximately 90% of all adults in the United States use the Internet and own a cell phone, and 74% are on a social network site, such as Facebook, Twitter, or Instagram (Duggan, Ellison, Lampe, Lenhard, & Madden, 2015). While younger people are more likely to be using e-technology, older adults are increasingly adopting it into their lives as well. For example, the percentage of adults 65 years and older going online has increased from 14% in 2000 to 59% in 2013; 77% reported having a cell phone in 2013 (Pew Research Center, April 2014). Most people agree that the growth of the Internet, smartphones, and tablet devices could have widespread and beneficial health effects.
Table 1.
Internet Use | Social Media Use | Own Cell//Smartphone | |
---|---|---|---|
|
|||
Teens (12-17) | 95% | 81% | 88%//73% |
|
|||
Adults - all | 87% | 74% | 90%//58% |
Adults <30 | 97% | 89% | 98%//83% |
Adults >65 | 59% | 56% | 77%//18% |
SOURCE: Pew Research Center, surveys conducted 2012-2014 http://www.pewresearch.org/
The purpose of this paper is to provide a general overview of the use of e-technologies in clinical trials research, specifically within the last decade. The objectives are to (1) present a summary of how e-technology is currently being integrated into clinical trial design, execution and dissemination; (2) illustrate how e-technology-based interventions are being used in the treatment of alcohol and drug abuse, as an example; (3) present the current status of regulatory guidelines for e-technology use, challenges and limitations; (4) summarize the advantages and limitations of e-technologies in clinical trials research; and (5) highlight future directions for the use of e-technologies in clinical research.
2.0 INTEGRATING E-TECHNOLOGIES INTO CLINICAL TRIAL DESIGN, IMPLEMENTATION, AND DISSEMINATION
Some clinical trial researchers have been early adopters of e-technology, using the Internet to recruit study participants (Etter & Perneger, 2001) and creating computer- or Internet-based interventions (Gustafson et al., 1994; Cawthorpe et al., 2001). Researchers have been using electronic tools for the last two decades to develop protocols, communicate with study personnel, randomize participants, collect data, and analyze results (Paul et al., 2005; Marks et al., 2001; Santoro et al., 1999). Early on, communication with participants was limited to directing individuals to a website for information about the study and providing specific contact information for recruitment and retention purposes (Scholle et al., 2000). Later, websites were used to distribute and collect data from online questionnaires for eligibility and consent purposes. More recently, clinical trials have started using social media (e.g. Facebook, Twitter), text messaging, and blogs to recruit and enhance retention of participants (Gurol-Urganci et al., 2013) and to meet regulatory requirements of community consultation (Stephens et al., 2013; Galbraith, 2014), as well as mobile technologies (tablets, smartphones) to collect data (surveys, patient reported outcomes) and assess or monitor study compliance (Wagner et al., 2014). Investigators are also starting to integrate other innovative data collection approaches, such as using apps (Buller, Borland, Bettinghaus, Shane, & Zimmerman, 2014), GPS (global positioning system) (Epstein et al., 2014), and wearable devices (Hossain et al., 2014).
Electronic systems have been used for clinical trial implementation procedures, such as randomization and data entry, for the last two decades (Pepine et al., 1998; Santoro et al., 1999). However, overall technological advances since then have been astronomical (Baker, 2014). On March 9, 2015, Apple introduced its ResearchKit software, designed for medical and health research. As of March 30, 2015, several iPhone apps have been developed for use in large-scale studies on asthma, breast cancer, cardiovascular disease, diabetes, and Parkinson’s Disease (http://www.apple.com/pr/library/2015/03/09Apple-Introduces-ResearchKit-Giving-Medical-Researchers-the-Tools-to-Revolutionize-Medical-Studies.html). Also, Google, Inc.’s has developed a tool intended as a medical device that could be used for clinical trials in the near future (http://hitconsultant.net/2015/06/24/google-developing-wearable-for-clinical-trial-research/). Below is an overview of how researchers are using technology to assist with various aspects of clinical trials, such as recruitment, retention, data collection, and dissemination of results.
2.1 Recruitment
Recruitment of participants into clinical trials is crucial to ensure the generalizability and validity of the study; however this is often one of the more challenging aspects of clinical research. Many clinical trials fail to meet initial participant recruitment goals as outlined in the study protocol (Bernardez-Pereira et al., 2014; Davis et al., 2014; Butler et al., 2015). In the last few years, Internet-based approaches have been increasingly used to supplement traditional recruitment strategies, and such approaches appear to be effective (Frandsen, Walters, & Ferguson, 2014; A. Leonard, et al.(2014); Gorman et al., 2014; Yuan et al., 2014). For example, Yuan and colleagues (2014) used various web-based and social media strategies (Facebook, Twitter, LinkedIn, Craigslist, and Tumblr) to recruit individuals living with HIV, a challenging population to recruit in clinical trials, in part due to the stigma associated with HIV. The study successfully recruited, primarily through Facebook, 1,404 adults who had diverse socio-demographic characteristics and represented a broad range of ages. Another study compared participant recruitment for a smoking cessation program using Facebook advertisements versus traditional strategies (flyers, newspaper advertisements) (Frandsen et al., 2014). The investigators found that approximately half (51.9%, n=138) of the total sample (n=266) were recruited via Facebook, with the only statistically significant difference between the two recruitment groups being age; subjects recruited using Facebook were approximately seven years younger compared to subjects recruited using print media. No between-group differences were observed for education, ethnicity, income, gender, or smoking characteristics.
Patients’ preferences for e-technologies may influence the extent to which these tools can be effective strategies. Internet-based personal registry tools are used for screening and recruitment (“23andMe”, “PatientsLikeMe”) (Hafen et al., 2014; Carere et al., 2014). Also numerous apps are being developed by sponsors and academic institutions to help patients find relevant clinical trials, such as the National Library of Medicine Pharmaceutical Product Development’s Clinical Trials app (http://www.clinicaltrials.com/industry/clinicaltrials_mobile.htm). Additional research is needed to elucidate how social media and other e-technologies are effective for recruitment based on specific content, specific disease types, when and how to target specific demographic subgroups, and other participant variables.
2.2 Engagement and retention
Another area that commonly challenges researchers is the retention of participants in studies, especially during long follow-up periods after the conclusion of active intervention. Prior to the advancement of mobile and Internet technologies, maintaining contact with participants over months or years of research took extensive staff effort and often resulted in less than ideal retention rates. The ability to maintain contact with subjects using mobile phones (e.g., calls, voicemail, texting), social media (e.g., Facebook), and websites has altered traditional retention strategies. The results of recent studies, however, again suggest that participant preference for e-technologies may influence how successful different engagement and retention strategies might be. For example, Rooke and colleagues (2014) compared two studies that used telephone versus Internet-based delivery of a treatment for cannabis use disorder and found that the telephone intervention had a significantly lower dropout rate compared with the web-based study (38% vs. 46% respectively, p<0.01). Age may be a factor in participant preference for retention strategies and modality of intervention deployment, although other factors such as type and length of intervention and burden of disease cannot be ignored.
A study of African American breast cancer survivors’ preferences for physical activity interventions found that there were no age differences in whether the physical activity intervention was administered via email/Internet, an in-person group, or the telephone (Paxton et al., 2014). Duncan and colleagues (2014) studied a physical activity/healthy eating intervention using IT- and print-based delivery modes and reported no significant differences in outcomes among middle-aged males. Two additional studies with younger participants also found preferences for e-technologies. In a study of youth with Type 2 diabetes, Nguyen and colleagues (2014) found that monetary incentives and technological approaches (mobile phones and websites) were the most effective strategies to engage and retain participants. In contrast, patients with rheumatoid arthritis, with a mean age of 61 years, preferred to complete research surveys via regular mail, followed by telephone, with the Internet identified as their least preferred method (Garcia et al., 2014).
2.3 Data collection
Most of the published research on data collection using e-technologies is about using Electronic Data Capture (EDC) data entry systems (Babre, 2011) and Internet-based strategies for administering surveys and questionnaires for behavioral or health promotion studies (Stone et al., 2003; Geyer, et al 2011; Anton et al., 2012; Muzafkar et al., 2014; Schwinn et al., 2014). A novel approach for data collection that is facilitated by e-technologies is the ecological momentary assessment (EMA), which is a method designed to collect data in real time, recently extended to also deliver interventions for individuals with substance use disorders as well (Shiffman, 2009; Runyan et al., 2013; Epstein et al., 2014). More recently, investigators are collecting biological data through wearable devices that can sync with smartphone apps. For example, Hossain and colleagues (2014) developed a model to detect drug (cocaine) use events from physiological measurements using a wearable electrocardiogram (ECG) sensor and accelerometers. Over 11,000 hours of data from 13 participants was collected and used to develop a model using physiological measures to track an individual’s health status. This demonstrates the way new technologies can facilitate the compiling and analyzing of substantial quantities of data that would have been nearly impossible to collect and manage several years ago.
In 2014, Orri and colleagues reported on the REMOTE trial (2014). In this study, the investigators used e-technologies to recruit (social media, Craigslist, health websites, online community forums) and manage (Internet-based consents and surveys) participants with Overactive Bladder (OAB). Laboratory test results used for consenting, screening, and determining eligibility were also conducted online, with participant identification verified using a confidential third-party vendor. Data collection occurred in real time via study-provided smartphones pre-loaded with study apps, with study medications mailed to participants. While this study was not able to reach its recruitment goal, it nevertheless demonstrates the feasibility of integrating e-technology into clinical trials while still achieving FDA regulatory approval.
Resources are now available to help investigators use e-technologies to collect data. For example, Coons and colleagues (2014) offered recommendations for obtaining Patient-Reported Outcomes (PROs) data using the participants’ own mobile devices, via an app or web-based portal.
2.4 Using electronic platforms for recruitment, retention and data collection
2.4.1 Registries
Researchers are conducting registry-based randomized trials, using existing registries to screen, recruit, randomize, and collect data. For example, Frobert et al. (2013) and Lagerqvist et al. (2014) reported a trial with patients with myocardial infarction (TASTE study) who were recruited using the Internet-based Swedish Coronary Angiography and Angioplasty Registry. The investigators used the registry to recruit, randomize and collect outcome data and reported no participants were lost to follow-up. Furthermore, Lauer & D’Agostino (2013) commented that this approach could lower costs of clinical trials as well as improve trial efficiency.
2.4.2 Electronic Health Records
Another tool for recruitment and data collection increasingly becoming essential to clinical trials is electronic health records (EHRs). With the recent emphasis on comparative effectiveness research (CER) and precision medicine, studies are using or planning to use EHR to facilitate recruitment and consent and collect clinical data (Kaelber, Foster, Gilder, Lore, & Jain, 2012; Baer et al., 2013; Fleurence et al., 2014; Fleurence et al., 2014; Lu et al., 2014; Collins & Varmus, 2015). EHRs can provide an automated electronic approach to (1) identify or flag potential subjects at the point of clinical care (Ruffin & Nease, 2011; Brieger & Aliprandi-Costa, 2013; Amin et al., 2014; Smith et al., 2007); (2) differentiate between clinical and research procedures/costs; (3) extract clinical data for import into study databases (Waitman et al., 2014) (Hunkeler et al., 2012; Sargious & Lee, 2014); and (4) collect study clinical outcomes directly (Otsuka et al., 2013; Basch, 2014; Stillman et al., 2014). Health systems with comprehensive integrated EHRs, like the Department of Veterans Affairs (VA), are starting to use this approach to recruit, screen, randomize, obtain consent, and collect data. A recent study by the VA (Fiore et al., 2011; D’Avolio et al., 2012) compared two standard clinical practices of insulin management for in-patients with diabetes on length of hospital stay. This “learning healthcare system” approach allows investigators to embed a clinical trial directly into clinical care settings to allow for pragmatic studies with immediate real-world impact.
2.4.3 Health Apps
The development of new e-technology and mobile device applications is fast-paced, and consumers have expectations regarding usability and efficiency (see Figure 1for a typical app development timeline). For example, as of July 2014, more than 3 million apps were available for download into mobile devices (http://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/). The Apple “App” store has 1.3 million apps, the majority of which were added in the last 6 years (http://www.statista.com/statistics/263795/number-of-available-apps-in-the-apple-app-store/), and the number continues to grow, with approximately 60,000 new apps launched per month (https://www.adjust.com/assets/downloads/AppleAppStore_Report2014.pdf). Of these, there are an estimated 100,000 health and fitness apps available, the fastest growing category. It is estimated that by 2018, 1.7 billion mobile device users will have downloaded a health app (http://mhealtheconomics.com/mhealth-developer-economics-report/). Health-related apps include reminders, diagnostic “helpers,” devices connected to hospital instruments, and tools for medication adherence and chronic care management (Bucklen & Abbott, 2014). Although these health apps have the potential to improve patient outcomes, very few enjoy any empirical support of their efficacy for the stated benefits.
The Apple software, ResearchKit (see Figure 2), is the most recent example of a national (soon to be international) tool developed to assist the research community in the conduct of research studies (http://www.apple.com/pr/library/2015/03/09Apple-Introduces-ResearchKit-Giving-Medical-Researchers-the-Tools-to-Revolutionize-Medical-Studies.html). In one of the first tests of recruitment effectiveness, the ResearchKit software helped to find 11,000 participants for a cardiovascular study in 24 hours, a task which would have taken a year using conventional methods in 50 medical centers (Bort, 2015). However, critics question the validity of the data acquired using this approach. They point out that potential issues, such as patient misrepresentation (both intentional and accidental) and demographic differences may yield misleading data (Cortez & Chen, 2015). In the near future, the Google X Wristband will also be in the market seeking regulatory approval as a device (http://hitconsultant.net/2015/06/24/google-developing-wearable-for-clinical-trial-research/).
2.5 Dissemination of study results
A critical component of any research study is the presentation and dissemination of findings. Traditionally, this has been the role of peer-reviewed journals or other print publications, typically accessible only to subscribers, and conference presentations and posters, often seen only by fellow conference attendees. Considering that the impact of a study’s results is largely dependent on how effectively those results are disseminated, using faster, cheaper, and more efficient means for dissemination, such as Twitter, Facebook, and blogs (regularly updated, often interactive web sites that feature essay-style posts providing information or opinion), are promising tools (Allen, Stanton, Di Pietro, & Moseley, 2013). These platforms can be used to disseminate study results not just to scientists and practitioners, but also to the individuals who participated in the study and to the general public. Not only have journal publishers begun using blogs and social media to disseminate their current contents, but research institutions and individual researchers are increasingly promoting their work via Twitter and other social media outlets as well. A growing number of platforms are available for researchers to more widely disseminate findings and share with colleagues, including PubPeer, ResearchGate, and Academia.edu, among others.
A recent study examined the impact of some of these newer Internet-based dissemination tools (Allen et al., 2013). Sixteen PLoS ONE articles from the clinical pain sciences were released on two randomly selected dates, with only the first release accompanied by blogging and simultaneous postings on Facebook, Twitter, LinkedIn, and ResearchBlogging.org. Investigators found that the date accompanied by social media increased both viewing and downloads of the article by a daily average of 12 views of the HTML page of the article and 3 downloads of the article’s PDF. Another study reported on an analysis of the impact of social media posts about scholarly articles, examining 4,208 tweets that cited 286 Journal of Medical Internet Research papers (Eysenbach, 2012). Frequently tweeted articles were 11 times more likely to be highly cited than less-tweeted articles. Top-cited articles could be predicted from top-tweeted articles with 93% specificity and 75% sensitivity (Eysenbach, 2012).
Blogging has also played a significant role in the dissemination of new research, allowing both for the speedy translation of potentially complex results into formats suitable for a wider range of audiences and as a way to stimulate discussion. Aggregators like ResearchBlogging.org have helped increase access to findings both in the scientific community and to members of the public, by collecting and organizing the wide range of blog posts on the Internet citing peer-reviewed research and providing a single point of access to those posts (Fausto et al., 2012). Many of the bloggers whose posts are represented on ResearchBlogging.org also use Twitter to further expand dissemination of their content (Shema, Bar-Ilan, & Thelwall, 2012), suggesting that multiple routes for sharing information are beneficial in expanding its overall reach.
In addition to simply disseminating study results, the Internet is increasingly being used for post-print peer-review, as more scientists use social media to offer critical commentary on a study’s methods or application to clinical practice. Twitter is a frequent choice for post-print debates (Choo et al., 2014), and a newer platform called “PubPeer” (http://pubpeer.com) has also been developed with a focus specifically on this role. Users of PubPeer must be published authors themselves, and, once granted an account, they are able to post under their own names or anonymously to discuss specific papers, often launching debates over a study’s quality or outcomes. The ability to remain anonymous has caused some strife in the science community, with one researcher even suing PubPeer requesting to expose the names of people who had criticized his work on the site (Sarkar v. John and/or Jane Does, Wayne County Circuit Court; the verdict was in favor of PubPeer). Nevertheless, this site appears to be growing in use among researchers globally and it provides a useful forum that could advance the design and quality of clinical trials.
Numerous professional groups have begun to launch Twitter-based journal clubs using a hashtag to tag tweets relevant to the club (e.g., #cpjc for clinical psychology, #ebnjc for evidence-based nursing, and #twitjc for microbiology). Selected articles are shared with the journal club members, typically via a blog, and then during a specified time, members log into Twitter to discuss the article together online. A search of the hashtag in Twitter will retrieve the complete exchange at any time, and the journal clubs are generally open to anyone interested from around the globe, facilitating international conversations and science dissemination. Twitter use is also growing among conference attendees, allowing for real-time sharing of the latest study results prior to their publication in journals (Wilkinson et al., 2014; Choo et al., 2014). Conference organizers typically prepare a hashtag for the event in advance (e.g., #APA2014 for the American Psychological Association’s 2014 annual convention) and distribute it to attendees, encouraging a virtual discussion forum accessible to the world at large.
Another expanding platform in the sciences is ResearchGate, a social networking site specifically for researchers and scientists. Unlike PubPeer, researchers on ResearchGate (http://researchgate.net) create profiles using their real names and affiliations, filling in detailed information about their field of specialty, and posting links to or full-text copies of their published papers. Discussion forums allow for conversation among participants. In a survey of 3,000 scientists and engineers published in 2014 (Van Noorden, 2014), more than half the participants reported using Research Gate “regularly,” beating out every other social media and research-profiling site except for Google Scholar. Academia.edu is similar to ResearchGate, though in 2013, it was sued by Elsevier for allowing users to share post-print copies of articles from copyrighted Elsevier titles, highlighting some of the complexities generated by new e-technology platforms.
A major caveat to the promise of dissemination through e-technologies is the informal nature of these various platforms and the challenges for end users in seeking and verifying reliable information. Many sites may not be monitored for accuracy or quality. Misinformation is rampant on the Internet, and it can be particularly challenging to figure out what information is legitimate when users rely on anonymous accounts for sharing. The onus of determining what information is valid and what is not is primarily on the end user. With so much information available at a user’s fingertips, however, the consumer’s ability to make quick, accurate determinations about the quality of information presented becomes crucial in successful dissemination of science-supported information. Researchers and institutions can take some simple steps to assist the user in making these determinations, however. A few suggestions include:
Publish in open access peer-reviewed journals whenever possible – this makes research readily available to the public, via sites the user can quickly determine are legitimate.
Include your name, credentials, and institution in your profile when using social media accounts.
Facilitate confirmation of sources by including a link in your tweet or post to your original paper or another reliable source of information.
Moderate and facilitate online discussions of your work; don’t let misinterpretations or misinformation about your research stand uncorrected.
Create an ORCID iD (orcid.org) and include this in your user information so that users can quickly and reliably identify you and your other works. ORCID iDs are permanent identifiers for researchers that help protect your unique scholarly identity and unite all your work under a single profile.
2.6 TECHNOLOGY AS A PLATFORM FOR INTERVENTIONS
The integration of technologies into clinical interventions varies significantly across disciplines and by disease or health condition. There are many published reports of Internet and mobile device interventions (both prevention and treatment) for a great variety of conditions, including obesity/weight loss, cancer, diabetes, depression, stroke, arthritis and so on. Most of these interventions do not focus on pharmacotherapies, but rather behavioral, disease management, and/or health promotion interventions. It is beyond the scope of this article to comprehensively review how technology has been used to deliver interventions in clinical trials, so this paper will focus on technology-based interventions for substance use disorders (SUD) as an illustrative example.
2.6.1 SUD Background
E-technologies, such as computer-assisted interventions or mobile phone applications, for SUDs have made exciting gains over the last decade. Of particular importance, as the behavioral health treatment landscape has changed with the passing and implementation of the Patient Protection and Affordable Care Act (ACA) (US GPO, 2010) and the Mental Health Parity and Addiction Equity Act (US GPO, 2008), is the renewed attention on the availability of SUD treatment options and the integration of behavioral health into non-specialty settings, such as primary care. Integration of care aims to improve the accessibility and acceptability of behavioral health care, while managing the high costs associated with mental health and addiction problems. Thus, the role of e-technologies in this changing landscape has the potential to make a powerful impact, especially given that only about 10% of those who need addiction treatment services actually receive it (McLellan & Woodworth, 2014).
Although the need for e-technologies is growing, the research base is also still accumulating and, as noted earlier, there is a gap between advances in technology and the ability to conduct rigorous research trials testing different e-technology interventions. The need for high quality, methodologically rigorous research trials to expand the evidence base for e-technology tools is essential, especially with the influx of tools available on the Internet for which there is little or no efficacy research to support their use. There is also a need to better understand ways to scale up research-based e-technologies that have a science base. For example, greater collaboration between relevant stakeholders (e.g., researchers, clinicians and service providers, governmental institutions, businesses) is needed to develop models for treatment development that take into account how to make an intervention available via the Internet in a sustainable way should it be shown to be efficacious. This may take the form of innovative research/business partnerships that will move e-technology tools into the real world and better keep up with the pace of technological advances.
There is a vast array of tools already under study that cross the care continuum, including e-technologies to prevent, assess, treat, and support populations affected by SUD. Although certainly electronic intervention tools for assessment of substance abuse (e.g., ASI-MV, Butler et al., 2001; CHAT-Comprehensive Health Assessment Tool for Teens, Lord et al., 2011) and primary prevention of SUD are available (e.g., Web-based Substance Use Prevention for Adolescent Girls, Schinke, 2009), this overview will focus on secondary prevention, treatment, and recovery management tools for SUD.
2.6.2 SUD Prevention
E-technologies for secondary and tertiary prevention have primarily targeted alcohol use among at-risk college-age populations. The majority of these studies have examined feasibility questions and been comprised of convenience samples (i.e., offered to large groups of individuals who can opt in or out). Recent research reviews and studies of e-technologies for these types of prevention interventions demonstrate that brief interventions for alcohol are superior to waitlist or no treatment controls and roughly equivalent to alternative interventions for problem drinking (Carey, Scott-Sheldon, Elliott, Bolles, & Carey, 2009; Bewick et al., 2008; Elliott, Carey, & Bolles, 2008; White, 2008; Lewis & Neighbors, 2007). There is evidence of diminishing returns for electronic-based brief interventions over time, and positive shorter term, but not-longer term, outcomes (Khadjesari, Murray, Hewitt, Hartley, & Godfrey, 2011; Kypri et al., 2004; Kypri, Langley, Saunders, Cashell-Smith, & Herbison, 2008). Direct comparisons with face-to-face interventions have been completed less frequently, but have tended to favor face-to-face interventions on outcome measures of quantity and consequences, especially among individuals with higher problem severity. However, several individual studies found e-technology prevention interventions to be as effective as brief therapies for problem drinking in college students (Barnett, Murphy, Colby, & Monti, 2007; Butler et al., 2009).
A number of secondary or tertiary prevention interventions have been developed with the goal of raising awareness of hazardous substance use and encouraging change in use through a combination of assessment, normative feedback, and increasing motivation for modifying use or seeking more intensive addiction treatment services (e.g., Miller, Sovereign, & Krege, 1988; Miller, Zweben, DiClemente, & Rychtarik, 1992). In this way, science-based interventions that have been traditionally delivered by a clinical provider can now be deployed via a computer and/or through the Internet. Obvious advantages of an electronic version of a brief assessment/feedback intervention include the ability to deliver it to vastly more individuals (increasing accessibility) and to individuals who are ambivalent about seeking treatment but may be open to exploring a potential problem by completing an intervention online (addressing acceptability of treatment options and stigma related to treatment attendance). Brief interventions to increase awareness of problem substance use and enhance motivation to seek additional services include: “E-Check-up” to Go or “E-Chug” (e.g., Hustad, Barnett, Borsari, & Jackson, 2010; Walters, Vader, & Harris, 2007; Doumas, Kane, Navarro, & Roman, 2011); Drinker’s Check-up (Hester, Delaney, & Campbell et al., 2012); and Check Your Drinking (Cunningham, Wild, Cordingley, Van Mierlo, & Humphreys, 2009). Ondersma and colleagues (2007; 2014) have completed two randomized controlled trials of the Motivational Enhancement System, a brief 20-minute computer-assisted intervention targeting motivation to reduce drug use, designed specifically for postpartum women.
2.6.3 SUD Treatment and Recovery Interventions
Fewer e-technology treatments have been tested among illicit substance users (Bewick et al., 2008; Portnoy, Scott-Sheldon, Johnson, & Carey, 2008; Copeland & Martin, 2004; Carroll & Rounsaville, 2010), although the research base is growing. Systematic and meta-analytic reviews of e-technology treatments for mental health and SUDs report promising findings, but methodological weaknesses have tempered enthusiasm (Bewick et al., 2008; Bickel, Christensen, & Marsch, 2011; Kiluk et al., 2011; Payne, Lister, West, & Bernhardt, 2015; Portnoy et al., 2008; Cuijpers et al., 2009; Spek et al., 2007). Important questions also remain about the best methods of integrating e-technologies into usual substance use disorder care. Litvin, Abrantes, and Brown (2013) report on the successful adaptation of empirically supported face-to-face treatments (e.g., cognitive behavioral therapy and community reinforcement approach) in computer-assisted modalities, but suggest that these types of interventions do not necessarily take full advantage of what technology has to offer (an exception to this may be emerging in technologies related to GPS and individualized, portable hand-held applications). The current crop of electronic alcohol and drug treatment tools can provide valuable assistance for moderate levels of problem use (e.g., for those that do not require detoxification or addiction medications) and for those who, for various reasons, may not seek face-to-face treatment (Litvin, Abrantes, & Brown, 2013).
For treatment seekers, e-technologies can be integrated into care in the following ways: (1) as an adjunct to enhance or improve standard addiction treatment (i.e., in addition to or as a supplement once a patient leaves treatment – continuing care); (2) as a clinical extender (i.e., as a substitution for a portion of standard treatment); or (3) as a replacement for standard care (although most of these studies have included some brief face-to-face clinician time). Substitution and replacement models may be particularly beneficial because they have the capacity to reduce the amount of time providers spend with clients, freeing them to see additional clients or focus additional time on patients who might have more complicated needs. Adjunct models may enhance usual care and produce better outcomes. The following section focuses on two key targets of e-technology research for SUD treatment: treatment and recovery management.
The number of available products with a science base has increased substantially in the last five years. There are now several efficacious interventions for illicit drug use, including CBT4CBT (Carroll et al., 2008; Carroll et al., 2014), Therapeutic Education System (Bickel et al., 2008; Campbell et al., 2014; Marsch et al., 2014), SHADE (Kay-Lambkin et al., 2012), and Reduce Your Use (Rooke, Copeland, Norberg, Hine, & McCambridge, 2013). Most of the e-technology treatment interventions are Internet- or app-versions of manualized, face-to-face, evidence-based treatments. For example, cognitive behavioral therapy techniques are available in several web-supported treatments (e.g., CBT4CBT and Therapeutic Education System). These types of interventions have been examined as adjuncts to standard substance abuse treatment (Carroll et al., 2008; Carroll et al., 2014), as replacement for a portion of standard care (Campbell et al., 2014; Marsch et al., 2014; Chaple et al., 2014), and as stand-alone alternatives to specialty addiction care (Kay-Lambkin et al., 2012). Some have also integrated other co-occurring disorders into the intervention including depression (Kay-Lambkin et al., 2012) and posttraumatic stress disorder (Brief et al., 2013).
Several e-technology treatment interventions have also been tested with specific subgroups of individuals with SUDs. For example, HealthCall (Hasin et al., 2013) is a brief intervention (Motivational Interviewing plus interactive voice response [IVR] technology to assist with daily self-monitoring) developed specifically to reduce problem alcohol use among HIV-infected primary care patients. HealthCall was designed to enhance motivational interviewing without additional burden to busy primary care staff. It was later updated to replace IVR with smartphone technology (Hasin, Aharaonovich, & Greenstein, 2014) with pilot results showing enhanced patient engagement. As mentioned earlier, Ondersma and colleagues also designed a brief computer-assisted intervention targeting polysubstance use among postpartum women (Ondersma, Svikis, & Schuster, 2007; Ondersma, Svikis, Thacker, Beatty, & Lockhart, 2014).
Several e-technologies have recently been developed to target problem cannabis use specifically. Tossman and colleagues (2011) tested “Quit the Shit” (QTS), a German-language, 50-day, web-based counseling program based on the tenets of solution-focused therapy and the principles of self-regulation and self-control. The intervention is comprised of online questionnaires, an entry and exit “chat” with a psychotherapist, and a weekly diary of cannabis use combined with feedback. Reflecting a common problem with web-based interventions, this study had significant attrition and only 16% of enrolled participants took part in the 3-month post-test. Those in QTS who were retained in treatment versus wait-list did demonstrate greater reductions in cannabis use, although additional research is necessary to determine efficacy. Rooke and colleagues (2013) developed and tested Reduce Your Use, a 6-week, web-based, self-guided educational program for cannabis use disorder focused on cognitive, motivational, and behavioral principles. In a randomized controlled trial, two-thirds of participants completed the 6-week post-intervention assessments, with participants in Reduce your Use completing an average of 3.5 of 6 modules. At the end of the 6-week intervention period, Reduce Your Use participants reported fewer days and quantity of cannabis use. Although the intervention group reported fewer cannabis dependence symptoms at the 3-month assessment as well, there were no longer differences in quantity. These interventions suggest the promise of e-technologies for cannabis use reduction but also demonstrate common challenges including attrition and shorter-term outcomes.
Another expanding area of research is focused on telehealth technologies, the delivery of addiction and other mental health care remotely using video conferencing and including assessment, treatment, counseling, medication-management, education, monitoring, and collaboration (Thomas & Capistrant, 2014). This is particularly promising for rural populations or individuals who live in areas with limited services or where the distance to a treatment program is a significant time and transportation barrier. Importantly, reimbursement for telehealth services is becoming much more common, expanding the potential of this tool (Thomas & Capistrant, 2014). The National Frontier and Rural Addiction Technology Transfer Center (see: http://attcnetwork.org/national-focus-areas/?rc=frontierrural), funded by the Substance Abuse and Mental Health Services Administration (SAMHSA), has been leading the national effort to promote awareness and implementation of telehealth technologies to deliver addiction treatment, prepare providers to adopt and implement telehealth technologies, and support telehealth competencies and policy recommendations (e.g., national license portability).
E-technologies are also being developed that focus on post-treatment recovery. These recovery management tools include the smartphone delivered Addiction-Comprehensive Health Enhancement Support System (A-CHESS; Gustafson et al., 2014) and the web-based My Ongoing Recovery Experience (MORE; Klein, Slaymaker, Dugosh, & McKay, 2012). A-CHESS provides monitoring and information, as well as opportunities for real-time communication and other support services to assist patients leaving residential treatment. In an efficacy study of A-CHESS (Gustafson et al., 2014), patients with alcohol use disorder (N=349) received treatment-as-usual or A-CHESS. Those in the A-CHESS group reported fewer risky drinking days. A-CHESS is now being tested in combination with the Therapeutic Education System (the integrated intervention is known as SEVA) among individuals with SUDs in three federally qualified health centers (estimated completion in 2017; ClinicalTrials.gov #NCT01963234) to explore implementation strategies and other organizational and patient outcomes (Quanbeck et al., 2014).
The web-based MORE intervention is based on motivational interviewing, cognitive behavioral therapy, and 12-step facilitation and was tested with 1,124 patients leaving residential treatment. At one year post discharge, 84% of patients had completed one MORE module, but only 5% had completed all 7. However, those that were more adherent reported longer periods of continuous abstinence, and the likelihood of abstinence increased as patients completed more modules (controlling for motivation, self-efficacy, and baseline severity). These types of e-technologies offer a way to extend treatment and provide additional support to patients often at high-risk of relapse.
Text-messaging-based interventions for addiction treatment have also received some attention. Muench and colleagues (2013) recently published a study examining acceptability, messaging preferences, and behavior change strategies for an interactive mobile text messaging continuing care intervention. Findings demonstrated a preference for benefit-driven over consequence-driven messages, although perceived benefits of behavior change predicted message preference. The authors call for additional intervention development based on user preference and further study of the effect of different types of messages, some of which are underway (e.g., Muench, van Stolk-Cooke, Morgenstern, Kuerbis, & Markle, 2014).
Several resources are now available that provide information and summarize findings of e-technology research for substance use disorders. SAMHSA and the National Institute on Drug Abuse (NIDA) collaborated on a web resource for technology-assisted care for substance use disorders (http://www.SUDTECH.org) that includes training and implementation materials. SAMHSA is also in the process of publishing a Treatment Improvement Protocol (TIP) for Technology and Behavioral Health slated to be published in 2015. Finally, the Center for Technology and Behavioral Health (http://www.c4tbh.org/), partially funded by NIDA and located at Dartmouth College, brings together interdisciplinary teams to harness existing and emerging technologies to develop and deliver evidence-based interventions for substance use and co-occurring disorders. Center staff edited a recently published book (Marsch, Lord, & Dallery, 2014), Behavioral Health Care and Technology Using Science-based Innovations to Transform Practice, which includes a chapter on technology-based interventions for substance use problems (Campbell, Muench, & Nunes, 2014).
2.6.4 Implementation of E-technology Interventions for SUD
Of particular importance, and in great need of empirical data, is the study of implementation strategies for moving e-technologies into addiction care services. Although e-technologies remove a number of potential barriers to implementation (e.g., limited staff training, increased accessibility, and potential for patient tailoring), there are still outstanding questions about how best to link these interventions with populations who could benefit. Research has demonstrated that, in general, patients rate the acceptability of e-technologies quite highly. Additional research should focus on which types of patients, and at what point in their recovery, e-technologies might be most helpful. For example, in a recent effectiveness trial of the Therapeutic Education System, a secondary outcome analysis revealed that women with active substance use at study entry rated the intervention lower in acceptability compared to women who were abstinent at study entry and to men, regardless of their substance use status at study entry (Campbell et al., 2014).
As described earlier, there are numerous ways to marry e-technologies with traditional clinical care (e.g., as an adjunct to treatment, as a replacement for a component of treatment, or as a stand-alone intervention). Testing the ways in which clinical providers interact with or integrate components of e-technologies into their treatment curriculums to promote patient outcomes should be explored. At least one study (Quanbeck et al., 2014) is currently doing this in primary care health centers. Additionally, the types of infrastructure and resource support needed when introducing addiction treatment into non-specialty settings is also of importance. Thus, even if a stand-alone technology-based intervention is provided to primary care patients, primary care clinicians must be trained to screen patients, introduce the treatment and monitor progress (McLellan & Woodworth, 2014). Finally, reimbursement strategies and other policy-related changes to addiction treatment are needed to incentivize the use of e-technologies in diverse clinical settings. Reimbursement and billing shifts, from fee-for-service to achievement of specific performance outcomes, may help in this regard.
3. REGULATORY AND ETHICAL CONSIDERATIONS
Despite the ubiquity of social media and technology use in the general population and increasing efforts to integrate technology into research practices, little regulatory guidance exists on using technology in clinical research (Moreno, Goniu, Moreno, & Diekema., 2013; Swirsky, Hoop, & Labott, 2014; Taylor, Kuwana, & Wilfond., 2014). In 2012, the Office for Human Research Protections (OHRP) held a series of meetings and presentations on the topic promising forthcoming guidance, but it has yet to be published. OHRP has one training webinar for institutional review boards (IRBs) on this subject, providing some light regarding privacy/confidentiality and implications for internet research in human subject protections (http://videocast.nih.gov/summary.asp?Live=13932&bhcp=1).
3.1 IRB Review
In the absence of explicit guidance, IRBs have had to determine their own policies that are in line with 45 CFR 46 and FDA Title 21 regulations. In the past five years, IRBs have progressed from offering virtually no guidance on the issue to recognizing the increasing number of research protocols being submitted that integrate e-technology and hence the need to develop policies and standards. The two primary issues researchers must consider when using technology in clinical research are privacy and informed consent (Bushel, Mehdi, Cammilleri, Marzouki, & Elger, 2014).
The more technology is used, the more vulnerable individuals are to having their privacy violated. While people are worried about personal autonomy and privacy, they also voluntarily disclose a great deal of personal information on social networks and in day-to-day activities conducted online, like banking and shopping. Using apps, wearable devices, or any program that automatically “pushes” data to another online source makes people increasingly vulnerable to data leaks and threats of fraud. From a legal standpoint, online privacy policies consider whether or not a person has a “reasonable expectation of privacy.” Federal and state courts have determined that people who publicly post to Facebook do not a have a reasonable expectation of privacy for that information (Moreno et al., 2013). The very nature of social media sites is to share; once a person knowingly makes a public post to social media, that person loses any expectation of privacy. Researchers, then, can use that information like other public records without IRB review. Similarly, observational research that collects information from public YouTube videos or Twitter feeds would have little expectation of privacy.
Interactive research activities on social media may require contacting a potential participant directly, which would require IRB approval. There are differing opinions about the appropriateness of “following” or “friending” potential participants on social media. One study of social networks found that social media is a loose social network, so “friending” someone does not imply a relationship that oversteps the bounds of researcher and participant (Ellison, Steinfield, & Lampe, 2007). Others might question the appropriateness of this approach and consider any research-related outreach to be an interaction that requires IRB review.
3.2 Consent
Most online surveys or interviews require consent. The online consent process typically comprises reading through a series of questions and indicating comprehension of procedures, risks and benefits, and voluntary participation by checking a box. Researchers and IRBs may be more comfortable with online consent prior to online surveys or interviews, which can be seen as ensuring greater confidentiality than face-to-face interviews or paper surveys, since names and location data (phone number, address) may not be collected.
One drawback of online consent is the lack of face-to-face contact during the consent process. As a result, the interviewer may not be able to ensure participant understanding of the consent form in the same way they can in a face-to-face interaction. This is not unique to online consent, however, as it is also true for mailed surveys. In an argument in favor of obtaining online consent, Moreno and colleagues (2013) suggest that people who are accustomed to online interactions might be even more likely to reach out to an investigator with questions than those in a face-to-face situation. While this has not been addressed systematically, it is possible that greater use of e-technology can improve interactions between researchers and participants.
Another complication in the consent process is obtaining parental consent for minors. Online, there is no way to know if a minor is providing consent on behalf of a parent without the parent’s knowledge. Similarly, it is difficult to verify a person’s identity online, where anonymity is often valued and “trolls” may pose as someone else as a way to incriminate or discredit that person.
In March 2015, the FDA issued a draft guidance on “Use of Electronic Informed Consent in Clinical Investigations,” providing recommendations for clinical investigators, sponsors, and IRBs on the use of electronic media and informed consent for FDA-regulated clinical investigations of medical products. The plan is for OHRP to consider the language and for the two agencies to issue a joint FDA-OHRP final document (http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM436811.pdf).
3.3 Other Recent Guidance
3.3.1 Health apps
Covered entities using medical health apps that intend to process or store protected health information (PHI) are subject to the federal Health Insurance Portability Accountability Act (HIPAA) and the Heath Information Technology for Economic and Clinical Health (HITECH) Act. Under these regulations, security breaches require notification and could result in penalties up to $50,000 per incident. There are also concerns that hackers could use these apps to obtain PHI and/or steal the devices to commit crimes. However, Terry (2014) argue that HIPAA rules may not apply to health data collected thru modern electronic technology.
Additionally, the FDA has provided guidance on using electronic source data (i.e., EHR) in clinical trials. (http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM328691.pdf).
3.3.2 Device Regulations
In February 2015, the Food and Drug Administration (FDA) issued an updated “Mobile Medical Applications Guidance” (http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM263366.pdf) which indicates the agency’s intent to regulate certain medical apps as medical devices. Is recommended that investigators contact the FDA for questions on how this guidance applies to clinical research.
3.3.3 Drug Labeling
FDA guidance provides recommendations for interactive post-marketing promotional materials (http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM381352.pdf).
3.4 Best Practices for Human Subjects Protection
As guidance continues to be developed, some best practices are coming to light in the field. Two of the coauthors (Campbell & Miele) of this paper were investigators in a randomized, multisite clinical trial studying the effectiveness of a web-based psychosocial intervention for SUDs. Given that e-technology was being used as an intervention, the researchers proposed using electronic communication tools (email, texting, and Facebook) for participant location and follow-up (see Campbell et al., 2012 and 2014 for methods and primary outcomes, respectively). At the time of the study (2010-2012), few IRBs had guidance about using social media and electronic communication in research, but use of social media and e-technology was becoming more common among participants. Three of 10 sites used electronic communication in this capacity. Research teams developed their own procedures and recommendations for review. The most important component of each policy was protection of human subjects, especially confidentiality. Based on these experiences and emerging developments in the field, the authors provide some recommendations for using e-technology in clinical trials while protecting participants (see Figure 3).
4. ADVANTAGES, CHALLENGES, AND LIMITATIONS OF E-TECHNOLOGY IN CLINICAL TRIALS
4.1 Advantages
Technology is dramatically changing the ways in which clinical trials are conducted, from study development through dissemination. Thanks to advances in communication, intervention delivery, and knowledge transfer, information and outreach are no longer confined to traditional silos, resulting in enormous potential for broadening scope and application. Potential advantages include:
4.1.1 Improving efficiency
Technology has the potential to impact the following areas to improve the efficiency of clinical trials: (1) Communication: Technology tools improve communications not just with study staff, but with patients and communities. This could expedite protocol develoment, regulatory approvals, and training, thus shortening the time from study concept to first randomization. (2) Recruitment: Using apps, registries, social media, and EHR could increase the number of participants that can be contacted, screened, and ultimately enrolled into a trial (e.g., Apple’s ResearchKit application). (3) Retention: Mobile phones/devices, apps, and social media offer the opportunity to connect with participants more often and potentially improve their engagement and retention into clinical trials. (4) Delivery of intervention: E-technology-based interventions can reduce resource requirements related to staff training and ongoing supervision, maintain consistent delivery of an intervention, increase accessibility across space and time, and potentially increase staff time to focus on more complex treatment delivery. (5) Data collection: Use of registries and EHRs can improve targeted recruitment and and make standard clinical data available in real-time for study outcome purposes. Smartphones, apps, and wearable body sensors can allow for large quantities of data to be collected automatically and not require face-to-face interactions with researchers.
4.1.2 Lowering cost
Using technologies for recruitment, delivery of intervention, and data collection has the potential to enroll large numbers of patients in a relative shorter time, or reduce the time it takes to reach target recruitment. It also may reduce the need for personnel/staff hours to perform these activities (delivery of intervention, data entry). Taken together, this means that e-technologies have the potential to allow clinical trials to be conducted faster and with less cost per patient.
4.1.3 Fostering research and development
There has been an explosion of health-care data available in the past decade (Szlezak, 2014). This offers a great opportunity not only for researchers but other stakeholders to make big advances in health care. This “Big Data” is becoming more available and accessible and can be used for research & development of pharmaceuticals, safety surveillance, marketing, and ultimately better patient outcomes.
4.1.4 Involving stakeholders
Technology could potentially serve as an effective and efficient platform for all individuals involved in the research enterprise, from participants in research trials to the practitioners who will ultimately implement evidence-based interventions in clinical practice or to obtain community consultation for certain types of planned research. Evidence-based e-technology treatments can be incorporated into the treatment process along a spectrum of involvement, from an adjunct or “clinical extender,” to a replacement for a component of treatment, or even as a stand-alone intervention.
4.1.5 Facilitate implementation and dissemination
The technology-based facilitation of information dissemination (e.g., to public websites in lay language) helps to expand both the recruitment of participants as well as the reach of scientific discoveries to those who may not otherwise have access to it, including the very people research interventions aim to assist: the patients themselves. It encourages more participation and collaboration in the scientific community, cuts back on duplication of efforts, and spurs new thinking and ideas. Technology may provide the flexibility that could help remove some of the barriers to intervention implementation, such as extensive staff training and fidelity monitoring.
4.2 Challenges and Limitations
Despite the ubiquity of technology and the numerous areas of promise, there are a variety of limitations and challenges in need of careful consideration and additional research. These include:
4.2.1 Privacy/confidentiality issues
E-technologies bring with them a tangled knot of privacy and confidentiality issues, frequently made more confusing by complex and under- or even un-developed regulatory guidelines from IRBs, funding agencies, and other stakeholder organizations. The potential violation of ethical standards and patient privacy and confidentiality, as well as professional codes of conduct, are major contributors to institutional hesitancy to allow and approve the use of new technologies in clinical research. Further, e-technologies, especially mobile apps, may have unknown or difficult to anticipate privacy and/or confidentiality problems that are further complicated by the risk of “hacking” or other cyber attacks. Researchers need to know the current federal, state, and local regulations and be cognizant that these are likely to evolve and expand over time.
4.2.2 Keeping up with technology
What was revolutionary when a trial started may be utterly obsolete by the time it is finished, leaving results, even when positive, of limited applicability. Trying to keep up with the changes during a study may result in unpredictable costs, both in staff time and budgets, as well as an imbalance in the type of technology used by participants, as some race to acquire the latest devices and others lag behind. It will be important to construct research questions that focus not just on a particular way of using technology, but on more general effectiveness questions that will continue to be meaningful with future technological advancements (e.g., components of Internet interventions that are effective but which could be deployed on different platforms). Baker and colleagues (2014) provide some strategies for addressing efficiency and quality of electronic technologies specifically for health research, given the speed with which electronic-based interventions are created, updated, or become obsolete.
4.2.3 Including a non-representative sample
A criticism of using technology in research is that participants recruited from social media and online sites are not a representative sample for most studies and that this may introduce bias. While there has been conflicting information about the “digital divide,” especially in people with substance use disorders (McClure, Acquavita, Harding, & Stitzer, 2013; Wolf-Branigan, 2009) or other disadvantaged populations, general research on Internet and mobile phone use indicates the divide is continuing to shrink by age, income, and race (Figure 4 represents demographics of adults using social network sites) (http://www.pewinternet.org/data-trend/internet-use/latest-stats/ http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/)
4.2.4 Adequate infrastructure
Web- or mobile-based interventions also require the use of different types of infrastructure and resource support, which can be difficult to navigate. For example, while app-based interventions may facilitate the movement of substance abuse treatments into primary care, lessening the burden on physicians to obtain training on intervention delivery, those physicians still need to learn how to screen patients, introduce the treatment, and monitor progress, all using a medium they may be unfamiliar with. Implementation science studies will be useful for developing best practices on how to integrate e-technologies into healthcare settings. In addition, research teams should include others, such as engineering, data science, and business experts who can assist with study development and implementation, as well as assist with minimizing some of the challenges and limitations of the technology tools.
4.2.5 Big data/Accuracy of data
Given the sheer quantity of data available, one of the biggest challenges is identify which data to collect, how to capture it, and how to analyze it. Once researchers figure this out, the other challenge is data quality and consistency, given the current absence of standard methodologies, such as unique identifiers for patients, physicians, and diseases for use across all data types (Szlezak et al., 2014) and varying expectations and time to collect fully complete data within EHRs.
4.2.6 User identification
More challenging may be the complexities that come when users online have the ability to remain anonymous or to invent characteristics for themselves. Though the ability to be mostly or completely anonymous can increase the comfort required for some to participate in unfettered dialogue, it can also make it difficult to verify identities or specified demographics in a study population (e.g., age, gender). The lack of any face-to-face contact may also make informed consent harder to ensure. Anonymous accounts also make it impossible to determine how much trust to put in the quality of information being distributed. The Internet is infamous for the propagation of inaccuracies, and if a person cannot verify the credentials of a poster on Twitter or a blog, for example, it can be a challenge to determine the reliability of the content.
4.2.7 Provider attitudes
Time limitations for researchers and stigma about “professionalism” can be barriers to the use of blogs and social media as wel. In one survey of faculty at Johns Hopkins School of Public Health (Keller, Labrique, Jain, Pekosz, & Levine, 2014), a majority of respondents were either not interested in or actively opposed to professional engagement in social media. Lack of time was frequently cited, and only 24% felt social media usage was useful for “career advancement.”
5. FUTURE DIRECTIONS
The advent of new technologies for clinical trials and availability of “Big Data” provides an enormous opportunity for researchers, but many challenges still remain. Investigational teams should keep in mind that e-technologies are simply additional types of research tools not intended to completely supplant traditional methods, particularly given the need for more empirical evidence. Newly released technologies require time to be tested and refined to overcome challenges. Future research can expand our knowledge of the impact of e-technologies on clinical trial design and interpretation, particularly in respect to validity of results, randomization and blinding. When determining whether to integrate e-technologies into clinical trials, one may want to consider 1) the target population and their access to and preferences for specific e-technologies, 2) how technology may change during the time it takes to conduct the trial, 3) whether methods will be compliant with regulations on the use of e-technologies, 4) the available infrastructure to process large volumes of data, and 5) whether the e-technologies introduce additional and/or unnecessary risks to subjects’ privacy or confidentiality. Investigators should partner with professionals from multidisciplinary knowledge areas (e.g., engineering, information/computer science, business) and patients to address the variety of challenges involved in integrating e-technologies in clinical trials. Careful planning will not just help with a successful implementation of the clinical trial, but will increase the chances of the usability and sustainability of the studied technology after the end of grant funding.
While theoretical and practical challenges definitely exist when it comes to new e-technologies in clinical research, many of these challenges can be overcome with careful planning, useful partnerships, and forethought. The role of web- and smartphone-based applications, including social media, is expanding in every field and the increasing use of those platforms by scientists and the public alike make them tools that cannot be ignored. These new mediums allow researchers to do things never before possible and have the potential to improve efficiency and lower the cost of clinical trials while radically expand both the availability and reach of clinical interventions to those who need them the most.
Highlights.
Clinical trials have been slow to incorporate e-technology into its design and execution, due to a range of challenges.
This article provides a general overview of the use of e-technologies in clinical trials implementation.
E-technologies are already integrated into patients’ lives and could improve clinical trials efficiency while reducing costs.
Many challenges exist, including a lack of regulatory guidance and threats to data validity and privacy.
Challenges of using e-technologies can be overcome; these tools can serve as a vital adjunct to clinical trial practices.
Acknowledgements
Funding
This work was supported by grants from the National Institute on Drug Abuse Clinical Trials Network: U10 DA013035 (PIs: Nunes, Rotrosen); U10 DA013714 (PIs: Donovan, Roll).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Declaration of conflicting interests
The authors do not have any conflicting interests.
Portions of this manuscript were presented at SAMHSA’s Technology Assistance Center Strategy Regional Summit, June 10, 2014, Reno, NV and the Society for Clinical Trials 35th Annual Meeting, May 18, 2014, Philadelphia, PA.
REFERENCES
- Allen HG, Stanton TR, Di Pietro F, Moseley GL. Social media release increases dissemination of original articles in the clinical pain sciences. PLoS ONE. 2013;8(7):e68914. doi: 10.1371/journal.pone.0068914. doi: 10.1371/journal.pone.0068914. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Amin W, et al. PaTH: towards a learning health system in the Mid-Atlantic region. Journal of the American Medical Information Association. 2014;21:633–636. doi: 10.1136/amiajnl-2014-002759. doi: 10.1136/amiajnl-2014-002759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Anton SD, et al. Use of a computerized tracking system to monitor and provide feedback on dietary goals for calorie-restricted diets: the POUNDS LOST study. Journal of Diabetes Science and Technology. 2012;6(5):1216–1225. doi: 10.1177/193229681200600527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Babre E. Electronic data capture - Narrowing the gap between clinical and data management. Perspectives in Clinical Research. 2011 Jan-Mar;2(1):1–3. doi: 10.4103/2229-3485.76282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baer HJ, et al. Use of a web-based risk appraisal tool for assessing family history and lifestyle factors in primary care. Journal of General Internal Medicine. 2013;28(6):817–824. doi: 10.1007/s11606-013-2338-z. doi: 10.1007/s11606-013-2338-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baker TB, et al. How can research keep up with eHealth? Ten strategies for increasing the timeliness and usefulness of eHealth research. Journal of Medical Internet Research. 2014;16:e36. doi: 10.2196/jmir.2925. doi: 10.2196/jmir.2925. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barnett NP, Murphy JG, Colby SM, Monti PM. Efficacy of counselor vs. computer-delivered intervention with mandated college students. Addictive Behaviors. 2007;32(11):2529–2548. doi: 10.1016/j.addbeh.2007.06.017. doi: 10.1016/j.addbeh.2007.06.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annual Review of Medicine. 2014;65:307–317. doi: 10.1146/annurev-med-010713-141500. doi: 10.1146/annurev-med-010713-141500. [DOI] [PubMed] [Google Scholar]
- Bernardez-Pereira S, et al. Prevalence, characteristics, and predictors of early termination of cardiovascular clinical trials due to low recruitment: insights from the ClinicalTrials.gov registry. American Heart Journal. 2014;168(2):213–219. doi: 10.1016/j.ahj.2014.04.013. doi: 10.1016/j.ahj.2014.04.013. [DOI] [PubMed] [Google Scholar]
- Bewick BM, et al. The effectiveness of web-based interventions designed to decrease alcohol consumption – A systematic review. Preventive Medicine. 2008;47:17–26. doi: 10.1016/j.ypmed.2008.01.005. doi: 10.1016/j.ypmed.2008.01.005. [DOI] [PubMed] [Google Scholar]
- Bickel WK, Marsch LA, Buchhalter A, Badger GA. Computerized behavior therapy for opioid-dependent outpatients: a randomized controlled trial. Experimental and Clinical Psychopharmacology. 2008;16(2):132–143. doi: 10.1037/1064-1297.16.2.132. doi: 10.1037/1064-1297.16.2.132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Christensen DR, Marsch LA. A review of computer-based interventions used in the assessment, treatment, and research of drug addiction. Substance Use & Misuse. 2011;46(1):4–9. doi: 10.3109/10826084.2011.521066. doi: 10.3109/10826084.2011.521066. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bort J. Thanks to Apple, 11,000 people signed up for a heart disease study in one day. Business Insider. 2015 Mar 12; Retrieved June 23, 2015, from http://www.businessinsider.com/apple-makes-heart-disease-study-go-nuts-2015-3.
- Brief DJ, et al. Web intervention for OEF/OIF veterans with problem drinking and PTSD symptoms: a randomized clinical trial. Journal of Consulting and Clinical Psychology. 2013;81(5):890–900. doi: 10.1037/a0033697. doi: 10.1037/a0033697. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brieger D, Aliprandi-Costa B. Developments in procedural and disease registries: a focus on coronary artery disease. Current Opinion in Cardiology. 2013;28(4):405–410. doi: 10.1097/HCO.0b013e328361e95a. doi: 10.1097/HCO.0b013e328361e95a. [DOI] [PubMed] [Google Scholar]
- Brooks A. Healthcare texting in a HIPAA – compliant environment. American Academy of Orthopaedic Surgeons. 2013 Retrieved from http://www.aaos.org/news/aaosnow/aug12/managing5.asp.
- Bucklen KW, Abbott BM. Promise and challenges with the use of mobile applications to support and improve patient care: an industry perspective. Clinical Pharmacology & Therapeutics. 2014;95(5):469–471. doi: 10.1038/clpt.2014.34. doi: 10.1038/clpt.2014.34. [DOI] [PubMed] [Google Scholar]
- Buller DB, Borland R, Bettinghaus EP, Shane JH, Zimmerman DE. Randomized trial of a smartphone mobile application compared to text messaging to support smoking cessation. Telemedicine Journal and E-Health. 2014;20(3):206–214. doi: 10.1089/tmj.2013.0169. doi: 10.1089/tmj.2013.0169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buschel I, Mehdi R, Cammilleri A, Marzouki Y, Elger B. Protecting human health and security in digital Europe: How to deal with the “Privacy Paradox”? Science and Engineering Ethics. 2014;20:639–658. doi: 10.1007/s11948-013-9511-y. doi: 10.1007/s11948-013-9511-y. [DOI] [PubMed] [Google Scholar]
- Butler J, et al. Improving cardiovascular clinical trials conduct in the United States: Recommendation from clinicians, researchers, sponsors, and regulators. American Heart Journal. 2015;169(3):305–314. doi: 10.1016/j.ahj.2014.12.001. doi: 10.1016/j.ahj.2014.12.001. [DOI] [PubMed] [Google Scholar]
- Butler SF, Budman SH, Goldman RJ, Newman FL, Beckley KE, Trottier D, Cacciola JS. Initial validation of a computer administered addiction severity index: The ASI–MV. Psychology of Addictive Behaviors. 2001;15:4–12. doi: 10.1037/0893-164x.15.1.4. [DOI] [PubMed] [Google Scholar]
- Butler LH, Correia CJ. Brief alcohol intervention with college student drinkers: Face-to-face versus computerized feedback. Psychology of Addictive Behaviors. 2009;23(1):163–7. doi: 10.1037/a0014892. doi: 10.1037/a0014892. [DOI] [PubMed] [Google Scholar]
- Campbell A, Muench F, Nunes EV. Technology-based behavioral interventions for alcohol and drug use problems. In: Marsch L, Lord S, Dallery J, editors. Behavioral Health Care and Technology: Using Science-Based Innovations to Transform Practice. Oxford University Press; New York: 2014. [Google Scholar]
- Campbell ANC, et al. Gender-based outcomes and acceptability of a web-delivered psychosocial intervention for substance use disorders. Journal of Substance Abuse Treatment. 2014. (in press). doi: 10.1016/j.jsat.2014.12.006. [DOI] [PMC free article] [PubMed]
- Campbell ANC, et al. Internet-delivered treatment for substance abuse: A multi-site randomized controlled clinical trial. American Journal of Psychiatry. 2014;171(6):683–90. doi: 10.1176/appi.ajp.2014.13081055. doi: 10.1176/appi.ajp.2014.13081055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell ANC, et al. Design and methodological considerations of an effectiveness trial of a computer-assisted intervention: An example from the NIDA Clinical Trials Network. Contemporary Clinical Trials. 2012;33(2):386–395. doi: 10.1016/j.cct.2011.11.001. doi: 10.1016/j.cct.2011.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carere DA, et al. Design, methods, and participant characteristics of the Impact of Personal Genomics (PGen) Study, a prospective cohort study of direct-to-consumer personal genomic testing customers. Genome Medicine. 2014;6(12):96. doi: 10.1186/s13073-014-0096-0. doi: 10.1186/s13073-014-0096-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carey KB, Scott-Sheldon LAJ, Elliott JC, Bolles JR, Carey MP. Computer-delivered interventions to reduce college student drinking: a meta-analysis. Addiction. 2009;104(11):807–1819. doi: 10.1111/j.1360-0443.2009.02691.x. doi: 10.1111/j.1360-0443.2009.02691.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carroll KM, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. American Journal of Psychiatry. 2008;165(7):881–888. doi: 10.1176/appi.ajp.2008.07111835. doi: 10.1176/appi.ajp.2008.07111835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carroll KM, et al. Computer-assisted delivery of cognitive-behavioral therapy: efficacy and durability of CBT4CBT among cocaine-dependent individuals maintained on methadone. American Journal of Psychiatry. 2014;171(4):436–444. doi: 10.1176/appi.ajp.2013.13070987. doi: 10.1176/appi.ajp.2013.13070987. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carroll KM, Rounsaville BJ. Computer-assisted therapy in psychiatry: Be brave – it’s a new world. Current Psychiatry Reports. 2010;12:426–432. doi: 10.1007/s11920-010-0146-2. doi: 10.1007/s11920-010-0146-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cawthorpe D. An evaluation of a computer-based psychiatric assessment: evidence for expanded use. Cyberpsychology & Behavior. 2001;4(4):503–510. doi: 10.1089/109493101750527060. [DOI] [PubMed] [Google Scholar]
- Chaple M, et al. Feasibility of a computerized intervention for offenders with substance use disorders: A research note. Journal of Experimental Criminology. 2014;10(1):105–127. doi: 10.1007/s11292-013-9187-y. doi: 10.1007/s11292-013-9187-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Choo EK, et al. Twitter as a tool for communication and knowledge exchange in academic medicine: A guide for skeptics and novices. Medical Teacher. 2014. in press. doi: 10.3109/0142159x.2014.993371. [DOI] [PubMed]
- Collins FS, Varmus H. A new initiative on precision medicine. New England Journal of Medicine. 2015;372(9):793–795. doi: 10.1056/NEJMp1500523. doi: 10.1056/NEJMp1500523. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Coons SJ, et al. Capturing Patient-Reported Outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. Patient. 2014. in press. [DOI] [PMC free article] [PubMed]
- Copeland J, Martin G. Web-based interventions for substance use disorders: A qualitative review. Journal of Substance Abuse Treatment. 2004;26:109–116. doi: 10.1016/S0740-5472(03)00165-X. [DOI] [PubMed] [Google Scholar]
- Cortez M, Chen C. Thousands have already signed up for Apple’s ResearchKit. BloombergBusiness. 2015 Mar 1; Retrieved June 23, 2015, from http://www.bloomberg.com/news/articles/2015-03-11/apple-researchkit-sees-thousands-sign-up-amid-bias-criticism.
- Cuijpers P, et al. Computer-aided psychotherapy for anxiety disorders: A meta-analytic review. Cognitive Behavioral Therapy. 2009;38:66–82. doi: 10.1080/16506070802694776. doi: 10.1080/16506070802694776. [DOI] [PubMed] [Google Scholar]
- Cunningham JA, Wild TC, Cordingley J, Van Mierlo T, Humphreys K. randomized controlled trial of an Internet-based intervention for alcohol abusers. Addiction. 2009;104(12):2023–2032. doi: 10.1111/j.1360-0443.2009.02726.x. doi: 10.1111/j.1360-0443.2009.02726.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- D’Avolio L, et al. Implementation of the Department of Veterans Affairs’ first point-of-care clinical trial. Journal of the American Medical Information Association. 2012;19(e1):e170–176. doi: 10.1136/amiajnl-2011-000623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Davis JM, et al. Optimizing clinical trial enrollment methods through "goal programming.”. Applied Clinical Trials. 2014;23(6-7):46–50. [PMC free article] [PubMed] [Google Scholar]
- Doumas DM, Kane CM, Navarro TB, Roman J. Decreasing heavy drinking in first-year students: Evaluation of a web-based personalized feedback program administered during orientation. Journal of College Counseling. 2011;14(1):5–20. [Google Scholar]
- Duggan M, Ellison NB, Lampe C, Lenhart A, Madden M. Social Media Update 2014. Pew Research Center; Jan. 2015. Retrieved from: http://www.pewInternet.org/2015/01/09/social-media-update-2014/ [Google Scholar]
- Duncan M, et al. Effectiveness of a web- and mobile phone-based intervention to promote physical activity and healthy eating in middle-aged males: randomized controlled trial of the ManUp study. Journal of Medical Internet Research. 2014;16(6):e136. doi: 10.2196/jmir.3107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elliott JC, Carey KB, Bolles JR. Computer-based interventions for college drinking: A qualitative review. Additive Behaviors. 2008;33:994–1005. doi: 10.1016/j.addbeh.2008.03.006. doi: 10.1016/j.addbeh.2008.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ellison NB, Steinfield C, Lampe C. The benefits of Facebook “Friends:” social capital and college students’ use of online social network sites. Journal of Computer-Mediated Communication. 2007;12:1143–1168. doi: 10.1111/j.1083-6101.2007.00367.x. [Google Scholar]
- Epstein DH, et al. Real-time tracking of neighborhood surroundings and mood in urban drug misusers: application of a new method to study behavior in its geographical context. Drug and Alcohol Dependence. 2014;134:22–29. doi: 10.1016/j.drugalcdep.2013.09.007. doi: 10.1016/j.drugalcdep.2013.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Etter JF, Perneger TV. A comparison of cigarette smokers recruited through the Internet or by mail. International Journal of Epidemiology. 2001;30(3):521–525. doi: 10.1093/ije/30.3.521. [DOI] [PubMed] [Google Scholar]
- Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research. 2011;13(4):e123. doi: 10.2196/jmir.2012. doi: 10.2196/jmir.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Moor G, et al. Using electronic health records for clinical research: The case of the EHR4CR project. Journal of Biomedical Informatics. 2014;53:162–73. doi: 10.1016/j.jbi.2014.10.006. doi: 10.1016/j.jbi.2014.10.006. [DOI] [PubMed] [Google Scholar]
- Fausto S, et al. Research blogging: Indexing and registering the change in science 2.0. PLoS ONE. 2012;7(12):e50109. doi: 10.1371/journal.pone.0050109. doi: 10.1371/journal.pone.0050109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fiore LD, et al. A point-of-care clinical trial comparing insulin administerd using a sliding scale versus a weight-based regimen. Clinical Trials. 2011;8(2):183–195. doi: 10.1177/1740774511398368. doi: 10.1177/1740774510395635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fleurence RL, et al. Patient-powered research networks aim to improve patient care and health research. Health Affairs. 2014;33(7):1212–1219. doi: 10.1377/hlthaff.2014.0113. doi: 10.1377/hlthaff.2014.0113. [DOI] [PubMed] [Google Scholar]
- Fleurence RL, et al. Launching PCORnet, a national patient-centered clinical research network. Journal of the American Medical Informatics Association. 2014;21(4):578–582. doi: 10.1136/amiajnl-2014-002747. doi: 10.1136/amiajnl-2014-002747. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frandsen M, Walters J, Ferguson SG. Exploring the viability of using online social media advertising as a recruitment method for smoking cessation clinical trials. Nicotine & Tobacco Research. 2014;16(2):247–51. doi: 10.1093/ntr/ntt157. doi: 10.1093/ntr/ntt157. [DOI] [PubMed] [Google Scholar]
- Frobert O, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. New England Journal of Medicine. 2013;369(17):1587–1597. doi: 10.1056/NEJMoa1308789. doi: 10.1056/NEJMoa1308789. [DOI] [PubMed] [Google Scholar]
- Galbraith KL. Practical and ethical considerations for using social media in community consultation and public disclosure activities. Academic Emergency Medicine. 2014;21(10):1151–7. doi: 10.1111/acem.12483. doi: 10.1111/acem.12483. [DOI] [PubMed] [Google Scholar]
- Garcia I, Portugal C, Chu LH, Kawatkar AA. Response rates of three modes of survey administration and survey preferences of rheumatoid arthritis patients. Arthritis Care Research. 2014;66(3):364–70. doi: 10.1002/acr.22125. doi: 10.1002/acr.22125. [DOI] [PubMed] [Google Scholar]
- Geyer J, et al. Implementing a low-cost web-based clinical trial management system for community studies: a case study. Clinial Trials. 2011;8(5):634–644. doi: 10.1177/1740774511416384. doi: 10.1177/1740774511416384. [DOI] [PubMed] [Google Scholar]
- Gorman JR, et al. A diversified recruitment approach incorporating social media leads to research participation among young adult-aged female cancer survivors. Journal of Adolescent and Young Adult Oncology. 2014;3(2):59–65. doi: 10.1089/jayao.2013.0031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gurol-Urganci I, et al. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database Systematic Review. 2013;12:CD007458. doi: 10.1002/14651858.CD007458.pub3. doi: 10.1002/14651858.CD007458.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gustafson DH, et al. A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry. 2014;71(5):566–72. doi: 10.1001/jamapsychiatry.2013.4642. doi: 10.1001/jamapsychiatry.2013.4642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gustafson D, et al. Development and pilot evaluation of a computer-based support system for women with breast cancer. Journal of Psychosocial Oncology. 1994;11(4):69–93. doi: 10.1300/J077V11N04_05. [Google Scholar]
- Hafen E, et al. Health data cooperatives – citizen empowerment. Methods of Information in Medicine. 2014;53(2):82–86. doi: 10.3414/ME13-02-0051. doi: 10.3414/ME13-02-0051. [DOI] [PubMed] [Google Scholar]
- Hasin DS, Aharaonovich E, Greenstein E. HealthCall for the smartphone: Technology enhancement of brief intervention HIV alcohol dependent patients. Addiction Science and Clinical Practice. 2014;9:5. doi: 10.1186/1940-0640-9-5. doi: 0.1186/1940-0640-9-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hasin DS, et al. Reducing heavy drinking in HIV primary care: A randomized trial of brief intervention, with and without technological enhancement. Addiction. 2013;108(7):1230–40. doi: 10.1111/add.12127. doi: 10.1111/add.12127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hester RK, Delaney HD, Campbell W. The college drinker’s check-up: Outcomes of two randomized clinical trials of a computer-delivered intervention. Psychology of Addictive Behaviors. 2012;26(1):1–12. doi: 10.1037/a0024753. doi: 10.1037/a0024753. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hossain SM, et al. Identifying drug (cocaine) intake events from acute physiological response in the presence of free-living physical activity. IPSN. 2014:71–82. [PMC free article] [PubMed] [Google Scholar]
- Hunkeler EM, et al. A web-delivered care management and patient self-management program for recurrent depression: a randomized trial. Psychiatric Services. 2012;63(11):1063–1071. doi: 10.1176/appi.ps.005332011. doi: 10.1176/appi.ps.005332011. [DOI] [PubMed] [Google Scholar]
- Hustad JTP, Barnett NP, Borsari B, Jackson KM. Web-based alcohol prevention for incoming students: A randomized controlled trial. Addictive Behaviors. 2010;35(3):183–189. doi: 10.1016/j.addbeh.2009.10.012. doi: 10.1016/j.addbeh.2009.10.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaelber DC, Foster W, Gilder J, Love TE, Jain AK. Patient characteristics associated with venous thromboembolic events: a cohort study using pooled electronic health record data. Journal of the American Medical Informatics Association. 2012;19(6):965–972. doi: 10.1136/amiajnl-2011-000782. doi: 10.1136/amiajnl-2001-00782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kay-Lambkin FJ, et al. Study protocol: a dissemination trial of computerized psychological treatment for depression and alcohol/other drug use comorbidity in an Australian clinical service. BMC Psychiatry. 2012;12:77. doi: 10.1186/1471-244X-12-77. doi: 10-1186-1471-244X-12-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Keller B, Labrique A, Jain KM, Pekosz A, Levine O. Mind the gap: Social media engagement by public health researchers. Journal of Medical Internet Research. 2014;16(1):38. doi: 10.2196/jmir.2982. doi: 10.2196/jmir.2982. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Khadjesari Z, Murray E, Hewitt C, Hartley S, Godfrey C. Can stand-alone computer-based interventions reduce alcohol consumption? A systematic review. Addiction. 2011;106:267–282. doi: 10.1111/j.1360-0443.2010.03214.x. doi: 10.1111/j.1360-0443.2010.03214.x. [DOI] [PubMed] [Google Scholar]
- Kiluk BD, et al. A methodological analysis of randomized clinical trials of computer-assisted therapies for psychiatric disorders: Towards improved standards for an emerging field. American Journal of Psychiatry. 2011;168(8):790–9. doi: 10.1176/appi.ajp.2011.10101443. doi: 10.1176/appi.ajp.2011.10101443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klein AA, Slaymaker VJ, Dugosh KL, McKay JR. Computerized continuing care support for alcohol and drug dependence: A preliminary analysis of usage and outcomes. Journal of Substance Abuse Treatment. 2012;42:25–34. doi: 10.1016/j.jsat.2011.07.002. doi: 10.1016/j.jsat.2011.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kypri K, et al. Web-based screening and brief intervention for hazardous drinking: A double-blind randomized controlled trial. Addiction. 2004;99(11):1410–1417. doi: 10.1111/j.1360-0443.2004.00847.x. doi: 10.1111/j.1360-0443.2012.04067.x. [DOI] [PubMed] [Google Scholar]
- Kypri K, Langley JD, Saunders JB, Cashell-Smith ML, Herbison P. Randomized controlled trial of web-based alcohol screening and brief intervention in primary care. Archives of Internal Medicine. 2008;168(5):530. doi: 10.1001/archinternmed.2007.109. doi: 10.1001/archinternmed.2007.109. [DOI] [PubMed] [Google Scholar]
- Lagerqvist B, et al. Outcomes 1 year after thrombus aspiration for myocardial infarction. New England Journal of Medicine. 2014;371(12):1111–1120. doi: 10.1056/NEJMoa1405707. doi: 10.1056/NEJMoa1405707. [DOI] [PubMed] [Google Scholar]
- Lauer MS, D’Agostino RB., Sr. The randomized registry trial--the next disruptive technology in clinical research? New England Journal of Medicine. 2013;369(17):1579–1581. doi: 10.1056/NEJMp1310102. doi: 10.1056/NEJMp1310102. [DOI] [PubMed] [Google Scholar]
- Leonard A, et al. Recruitment and retention of young women into nutrition research studies: practical considerations. Trials. 2014;15:23. doi: 10.1186/1745-6215-15-23. doi: 10.1186/1745-6215-15-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis MA, Neighbors C. Optimizing personalized normative feedback: The use of gender-specific referents. Journal of Studies on Alcohol and Drugs. 2007;68(2):228. doi: 10.15288/jsad.2007.68.228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Litvin EB, Abrantes AM, Brown RA. Computer and mobile technology-based interventions for substance use disorders: An organizing framework. Addictive Behaviors. 2013;38:1747–1756. doi: 10.1016/j.addbeh.2012.09.003. doi: 10.1016/j.addbeh.2012.09.003. [DOI] [PubMed] [Google Scholar]
- Lord S, Trudeau K, Black R, et al. Comprehensive Health Assessment Tool for Teens (CHAT): Development and validation of a computer-delivered, self-report substance use assessment for adolescents. Substance Use and Misuse. 2011;46(6):781–794. doi: 10.3109/10826084.2010.538119. [DOI] [PubMed] [Google Scholar]
- Lu M, et al. Improving efficiency and reducing costs: Design of an adaptive, seamless, and enriched pragmatic efficacy trial of an online asthma management program. Contemporary Clinical Trials. 2014;38(1):19–27. doi: 10.1016/j.cct.2014.02.008. doi: 10.1016/j.cct.2014.02.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marks RG, et al. Paradigm shifts in clinical trials enabled by information technology. Statistics in Medicine. 2001;20(17-18):2683–2696. doi: 10.1002/sim.736. [DOI] [PubMed] [Google Scholar]
- Marsch LA, Lord SE, Dallery J, editors. Behavioral health care and technology: using science-based innovations to transform practice. Oxford University Press; New York: 2014. [Google Scholar]
- Marsch LA, et al. Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. Journal of Substance Abuse Treatment. 2014;46:43–51. doi: 10.1016/j.jsat.2013.08.012. doi: 10.1016/j.jsat.2013.08.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McClure EA, Acquavita SP, Harding E, Stitzer ML. Utilization of communication technology by patients enrolled in substance abuse treatment. Drug and Alcohol Dependence. 2013;129(1-2):145–50. doi: 10.1016/j.drugalcdep.2012.10.003. doi: 10.1016/j.drugalcdep.2012.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McLellan AT, Woodworth AM. The Affordable Care Act and treatment for “Substance Use Disorders:” Implications of ending segregated behavioral healthcare. Journal of Substance Abuse Treatment. 2014;46(5):541–545. doi: 10.1016/j.jsat.2014.02.001. doi: 10.1016/j.jsat.2014.02.001. [DOI] [PubMed] [Google Scholar]
- Miller WR, Sovereign RG, Krege B. Motivational interviewing with problem drinkers: II. The Drinker’s Check-up as a preventive intervention. Behavioural Psychotherapy. 1988;16(4):251–268. doi: 10.1017/S0141347300014129. [Google Scholar]
- Miller WR, Zweben A, DiClemente CC, Rychtarik RG. Motivational Enhancement Therapy Manual: A Clinical Research Guide for Therapists Treating Individuals with Alcohol Abuse and Dependence. Vol. 2. National Institute on Alcohol Abuse and Alcoholism; Rockville, MD: 1992. Project MATCH Monograph Series. [Google Scholar]
- Moore BA, Fazzino T, Garnet B, Cutter CJ, Barry DT. Computer-based interventions for drug use disorders: A systematic review. Journal of Substance Abuse Treatment. 2011;40(3):215–223. doi: 10.1016/j.jsat.2010.11.002. doi: 10.1016/j.jsat.2010.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moreno MA, Goniu N, Moreno PS, Diekema D. Ethics and social media research: common concerns and practical considerations. Cyberpsychology, Behavior and Social Networking. 2013;16(9):708–713. doi: 10.1089/cyber.2012.0334. doi: 10.1089/cyber.2012.0334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Muench F, Weiss RA, Kuerbis A, Morgenstern J. Developing a theory driven text messaging intervention for addiction care with user driven content. Psychology of Addictive Behaviors. 2013;27(1):315–321. doi: 10.1037/a0029963. doi: 10.1037/a0029963. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Muench F, van Stolk-Cooke K, Morgenstern J, Kuerbis AN, Markle K. Understanding messaging preferences to inform development of mobile goal-directed behavioral interventions. Journal of Medical Internet Research. 2014;16(2):e14. doi: 10.2196/jmir.2945. doi: 10.2196/jmir.2945. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Muzaffar H, Chapman-Novakofski K, Castelli DM, Scherer JA, The HOT (Healthy Outcome for Teens) project Using a web-based medium to influence attitude, subjective norm, perceived behavioral control and intention for obesity and type 2 diabetes prevention. Appetite. 2014;72:82–89. doi: 10.1016/j.appet.2013.09.024. doi: 10.1016/j.appet.2013.09.024. [DOI] [PubMed] [Google Scholar]
- Nguyen TT, et al. Challenging recruitment of youth with type 2 diabetes into clinical trials. Journal of Adolescent Health. 2014;54(3):247–54. doi: 10.1016/j.jadohealth.2013.08.017. doi: 10.1016/j.jadohealth.2013.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ondersma SJ, Svikis DS, Schuster CR. Computer-based brief intervention: a randomized trial with postpartum women. American Journal of Preventive Medicine. 2007;32(3):231–238. doi: 10.1016/j.amepre.2006.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ondersma SJ, Svikis DS, Thacker LR, Beatty JR, Lockhart RN. Computer-delivered screening and brief intervention (e-SBI) for postpartum drug use: a randomized trial. Journal of Substance Abuse Treatment. 2014;46:52–59. doi: 10.1016/j.jsat.2013.07.013. doi: 10.1016/j.jsat.2013.07.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Orri M, Lipset CH, Jacobs BP, Costello AJ, Cummings SR. Web-based trial to evaluate the efficacy and safety of tolterodine ER 4 mg in participants with overactive bladder: REMOTE trial. Contemporary Clinical Trials. 2014;38(2):190–7. doi: 10.1016/j.cct.2014.04.009. doi: 10.1016/j.cct.2014.04.009. [DOI] [PubMed] [Google Scholar]
- Otsuka SH, et al. Improving herpes zoster vaccination rates through use of a clinical pharmacist and a personal health record. American Journal of Medicine. 2013;126(9):832.e831–836. doi: 10.1016/j.amjmed.2013.02.018. doi: 10.1016/j.amjmed.2013.02.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Panahaizar M, Taslimitehrani V, Jadhav A, Pathak J. Empowering personalized medicine with big data and semantic web technology: promises, challenges, and use cases; Proceedings of the IEEE International Conference on Big Data; 2014. pp. 790–795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paul J, et al. The Internet and clinical trials: background, online resources, examples and issues. Journal of Medical Internet Research. 2005;7(1):e5. doi: 10.2196/jmir.7.1.e5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paxton RJ, et al. African-American breast cancer survivors’ preferences for various types of physical activity interventions: A Sisters Network Inc. web-based survey. Journal of Cancer Survivorship. 2014;8(1):31–8. doi: 10.1007/s11764-013-0307-5. doi: 10.1007/s11764-013-0308-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Payne HE, Lister C, West JH, Bernhardt JM. Behavioral functionalilty of mobile apps in health interventions: A systematic review of the literature. Journal of Medical Internet Research. 2015;3(1):e20. doi: 10.2196/mhealth.3335. doi:10.2196/mhealth.3335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pepine CJ, et al. Rationale and design of the International Verapamil SR/Trandolapril Study (INVEST): an Internet-based randomized trial in coronary artery disease patients with hypertension. Journal of the American College of Cardiology. 1998;32(5):1228–37. doi: 10.1016/s0735-1097(98)00423-9. [DOI] [PubMed] [Google Scholar]
- Pew Research Center Older Adults and Technology Use. 2014 Apr. Retrieved from: http://www.pewinternet.org/2014/04/03/older-adults-and-technology-use/
- Portnoy DB, Scott-Sheldon LAJ, Johnson BT, Carey MP. Computer-delivered interventions for health promotion and behavioral risk reduction: A meta-analysis of 75 randomized controlled trials, 1988-2007. Preventive Medicine. 2008;47:3–16. doi: 10.1016/j.ypmed.2008.02.014. doi: 10.1016/j.ypmed.2008.02.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Quanbeck AR, et al. Integrating addiction treatment into primary care using mobile health technology: protocol for an implementation research study. Implementation Science. 2014;9:65. doi: 10.1186/1748-5908-9-65. doi:10.1186/1748-5908-9-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riley WT, et al. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clinical Translational Medicine. 2013;2(1):10. doi: 10.1186/2001-1326-2-10. doi: 10.1186/2001-1326-2-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rooke SE, et al. Applying technology to the treatment of cannabis use disorder: comparing telephone versus Internet delivery using data from two completed trials. Journal of Substance Abuse Treatment. 2014;46(1):78–84. doi: 10.1016/j.jsat.2013.08.007. doi: 10.1016/j.jsat.2013.08.007. [DOI] [PubMed] [Google Scholar]
- Rooke S, Copeland J, Norberg M, Hine D, McCambridge J. Effectiveness of a self-guided web-based cannabis treatment program. Journal of Medical Internet Research. 2013;15(2):e26. doi: 10.2196/jmir.2256. doi: 10.2196/jmir.2256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ruffin M.T.t., Nease DE., Jr. Using patient monetary incentives and electronically derived patient lists to recruit patients to a clinical trial. Journal of the American Board of Family Medicine. 2011;24(5):569–575. doi: 10.3122/jabfm.2011.05.100169. doi: 10.3122/jabfm.2011.05.100169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Runyan JD, Steenbergh TA, Bainbridge C, Daugherty DA, Oke L, et al. A Smartphone Ecological Momentary Assessment/Intervention “App” for Collecting Real-Time Data and Promoting Self-Awareness. PLoS ONE. 2013;8(8):e71325. doi: 10.1371/journal.pone.0071325. doi:10.1371/journal.pone.0071325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Santoro E, et al. Internet for clinical trials: past, present, and future. Controlled Clinical Trials. 1999;20(2):194–201. doi: 10.1016/s0197-2456(98)00060-9. [DOI] [PubMed] [Google Scholar]
- Sargious A, Lee SJ. Remote collection of questionnaires. Clinical & Experimental Rheumatology. 2014;32(5 Suppl 85):S-168–172. [PubMed] [Google Scholar]
- Schick-Makaroff K, Molzahn A. Strategies to use tablet computers for collection of electronic patient-reported outcomes. Health and Quality of Life Outcomes. 2015;13:2. doi: 10.1186/s12955-014-0205-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schinke SP, Fang L, Cole KC. Preventing substance use among adolescent girls: 1-year outcomes of a computerized, mother-daughter program. Addictive Behaviors. 2009;34(12):1060–4. doi: 10.1016/j.addbeh.2009.06.007. doi: 10.1016/j.addbeh.2009.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scholle SH, et al. Effect of different recruitment sources on the composition of a bipolar disorder case registry. Social Psychiatry and Psychiatric Epidemiology. 2000;35(5):220–7. doi: 10.1007/s001270050231. doi: [DOI] [PubMed] [Google Scholar]
- Schwinn TM, Schinke S, Fang L, Kandasamy S. A web-based, health promotion program for adolescent girls and their mothers who reside in public housing. Addictive Behaviors. 2014;39(4):757–60. doi: 10.1016/j.addbeh.2013.11.029. doi: 10.1016/j.addbeh.2013.11.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shema H, Bar-Ilan J, Thelwall M. Research blogs and the discussion of scholarly information. PLoS ONE. 2012;7(5):e35869. doi: 10.1371/journal.pone.0035869. doi: 10.1371/journal.pone.0035869. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Psychological Assessment. 2009;21(4):486–497. doi: 10.1037/a0017074. doi: 10.1037/a0017074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith KS, et al. Using web-based screening to enhance efficiency of HMO clinical trial recruitment in women aged forty and older. Clinical Trials. 2007;4(1):102–105. doi: 10.1177/1740774506075863. [DOI] [PubMed] [Google Scholar]
- Sorenson C, et al. The politics of comparative effectiveness research: lessons from recent history. Journal of Health Politics, Policy, and Law. 2014;39(1):139–170. doi: 10.1215/03616878-2395199. doi: 10.1215/03616878-2395199. [DOI] [PubMed] [Google Scholar]
- Spek V, et al. Internet-based cognitive-behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychological Medicine. 2007;37:319–338. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
- Stephens SW, et al. Preliminary experience with social media for community consultation and public disclosure in exception from informed consent trials. Circulation. 2013;3:267–270. doi: 10.1161/CIRCULATIONAHA.113.002390. doi: 10.1161/CIRCULATIONAHA.113.002.390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stone AA, et al. Patient compliance with paper and electronic diaries. Control Clin Trials. 2003;24(2):182–199. doi: 10.1016/s0197-2456(02)00320-3. [DOI] [PubMed] [Google Scholar]
- Stilman MD, et al. Health care utilization and barriers experienced by individuals with spinal cord injury. Archives of Physical Medicine and Rehabilitation. 2014;95(6):1114–1126. doi: 10.1016/j.apmr.2014.02.005. doi: 10.1016/j.apmr.2014.02.005. [DOI] [PubMed] [Google Scholar]
- Swirsky ES, Hoop JG, Labott S. Using social media in research: new ethics for a new meme? American Journal of Bioethics. 2014;14(10):60–61. doi: 10.1080/15265161.2014.948302. doi: 10.1080/15265161.2014.948302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Szlezak N, Evers M, Wang J, Perez L. The role of big data and advanced analytics in drug discovery, development, and commercialization. Clinical Pharmacologicay & Therapeutics. 2014;95(5):492–5. doi: 10.1038/clpt.2014.29. doi: 10.1038/clpt.2014.29. [DOI] [PubMed] [Google Scholar]
- Taylor HA, Kuwana E, Wilfond BS. Ethical implications of social media in health care research. American Journal of Bioethics. 2014;14(10):58–59. doi: 10.1080/15265161.2014.947820. doi: 10.1080/15265161.2014.947820. [DOI] [PubMed] [Google Scholar]
- Terry N. Health Privacy is Difficult but Not Impossible in a Post-HIPAA Data-Driven World. Chest. 2014;146(3):835–840. doi: 10.1378/chest.13-2909. doi: 10.1378/chest.13-2909. [DOI] [PubMed] [Google Scholar]
- Thomas L, Capistrant G. 50 state telemedicine gaps analysis: Coverage and reimbursement. American Telemedicine Association; 2014. Retrieved from: http://www.americantelemed.org/docs/default-source/policy/50-state-telemedicine-gaps-analysis…coverage-and-reimbursement.pdf?sfvrsn=6. [Google Scholar]
- Tossman HP, Jonas B, Tensil MD, Lang P, Struber E. A controlled trial of an Internet-based intervention program for cannabis users. Cyberpsychology, Behavior, and Social Networking. 2011;14(11):e26. doi: 10.1089/cyber.2010.0506. doi: 10.1089/cyber.2010.0506. [DOI] [PubMed] [Google Scholar]
- Van Noorden R. Scientists and the social network: Giant academic social networks have taken off to a degree that no one expected even a few years ago. A Nature survey explains why. Nature. 2014;512(7513):126–129. doi: 10.1038/512126a. doi: 10.1038/512126a. [DOI] [PubMed] [Google Scholar]
- Wagner B, et al. Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. Journal of Affective Disorder. 2014;152-154:113–121. doi: 10.1016/j.jad.2013.06.032. doi: 10.1016/j.jad.2013.06.032. [DOI] [PubMed] [Google Scholar]
- Waitman LR, et al. The Greater Plains Collaborative: a PCORnet clinical research data network. Journal of the American Medical Informatics Association. 2014;21(4):637–641. doi: 10.1136/amiajnl-2014-002756. doi: 10.1136/amiajnl-2014-002756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Walters ST, Vader AM, Harris TR. A controlled trial of web-based feedback for heavy drinking college students. Prevention Science. 2007;8(2):83–88. doi: 10.1007/s11121-006-0059-9. [DOI] [PubMed] [Google Scholar]
- White RE. Health information technology will shift the medical care paradigm. Journal of General Internal Medicine. 2008;23(4):495–499. doi: 10.1007/s11606-007-0394-y. doi: 10.1007/s11606-007-0394-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilkinson SE, Basto MY, Perovic G, Lawrentschuk N, Murphy DG. The social media revolution is changing the conference experience: Analytics and trends from eight international meetings. BJU International. 2014. in press. doi: 10.1111/bju.12910. [DOI] [PubMed]
- Wolf-Branigan M. New media and social networks: considerations from clients in addictions treatment. Journal of Technology in Human Services. 2009;27(4):339–45. [Google Scholar]
- Yuan P, et al. Using online social media for recruitment of human immunodeficiency virus-positive participants: a cross-sectional survey. Journal of Medical Internet Research. 2014;16(5):e117. doi: 10.2196/jmir.3229. doi: 10.2196/jmir.3229. [DOI] [PMC free article] [PubMed] [Google Scholar]