Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Dec 1.
Published in final edited form as: J Fam Psychol. 2017 Dec;31(8):983–993. doi: 10.1037/fam0000349

Using Technology to Enhance and Expand Interventions for Couples and Families: Conceptual and Methodological Considerations

Brian D Doss 1, Leah K Feinberg 2, Karen Rothman 3, McKenzie K Roddy 4, Jonathan S Comer 5
PMCID: PMC5761076  NIHMSID: NIHMS927253  PMID: 29309184

Abstract

Technological advances provide tremendous opportunities for couple and family interventions to overcome logistical, financial, and stigma-related barriers to treatment access. Given technology’s ability to facilitate, augment, or at times even substitute face-to-face interventions, it is important to consider the appropriate role of different technologies in treatment and how that may vary across specific instances of technology use. To that end, this article reviews the potential contributions of telemental health (aka telehealth; e.g., videoconferencing to remotely deliver real-time services) and asynchronous behavioral intervention technologies (BITs; e.g., apps, web-based programs) for couple and family interventions. Design considerations – such as software and hardware requirements and recommendations, characteristics of intended users, ways to maximize engagement, and tips for integrating therapists/coaches – are included for both types of technology-based intervention. We also present suggestions for the most effective recruitment and evaluation strategies for technology-based couple and family interventions. Finally, we present legal and ethical issues that are especially pertinent when integrating technology into couple and family interventions.

Keywords: telemental health, telehealth, behavioral intervention technologies, internet, methodology


Couples and families experience relationship and family distress at an alarming rate, with up to one third of couples reporting distress at a given time, and nearly 40% of marriages in the US ending in divorce (Whisman, Beach, & Snyder, 2008; CDC 2015). Relationship distress is associated with increased risk of mental and physical health problems, including depression, anxiety, substance use, and cardiovascular and immune diseases (Robles, Slatcher, Trombello, & McGinn, 2014). Despite the prevalence and negative impact of relationship distress, few couples seek professional help for relationship problems. In lieu of couple therapy, the majority of couples turn to self-help books or relationship workshops (Stewart, Bradford, Higginbotham, & Skogrand, 2016), and those who do seek couple therapy most frequently do so with a religious leader or organization (Doss, Rhoades, Stanley, & Markman, 2009). Moreover, nearly one in five children and adolescents experience mental illness, with the most common diagnoses being disruptive behavior disorders, mood disorders, anxiety disorders, and developmental disorders (Olfson et al., 2014). Despite the prevalence of childhood problems, geographic barriers to care, mental health stigma, and lack of financial resources prevent many from seeking and/or accessing help (Mukolo, Heflinger, & Wallston, 2010).

Technology can be Leveraged to Increase the Reach of Services for Couples and Families

Technological advances provide tremendous opportunities for couple and family services to overcome common barriers to treatment seeking and improve cost effectiveness.

Reduced barriers

First, technology can be harnessed to overcome logistical barriers and provide unfettered access to services at the pace, schedule, and preference of the client. The most commonly discussed benefit of technology use in psychological interventions is its potential to transcend the confines of geography (Nelson & Bui, 2010) and extend the reach of psychological services (Comer & Barlow, 2014). The majority of mental health services are concentrated in major metropolitan regions and academic hubs. As a result, a sizable proportion of U.S. counties – especially rural and remote communities – do not have a mental health provider specializing in youth, family, or relationship interventions (National Organization of State Offices of Rural Health, 2011). In contrast, 77% of adults in the United States own a smartphone and 73% have broadband access at home; in rural areas of the United States, the comparable numbers are 67% and 63% (Smith, 2017).

Improved cost effectiveness

Financial burdens associated with mental healthcare often reduce or prevent access to mental health treatment (Sareen et al., 2007). Financial barriers are especially acute for treatments of relationship or family distress because insurance companies often do not reimburse for couple therapy or family interventions not focused on a child’s diagnosis. Technology can seemingly be used to overcome these financial barriers, although the issue has received scant empirical attention. For clients, technology can cut down on transportation, parking, and childcare costs that are often associated with clinic-based treatment (Smit et al., 2006). Technology may also reduce costs for providers, as brick and mortar therapy rooms are not required. Moreover, most of the costs of automated or asynchronous applications are incurred up-front, with little added costs associated with each additional couple or family served (e.g., Georgia, Rothman, Roddy, & Doss, 2017).

Importance of Technology for Family and Couple Interventions

Integration of technology into interventions may be particularly useful for the unique needs and circumstances of couples and families. First, couple and family interventions come with unique barriers to access to care. Treatment involving multiple participants requires coordination of multiple schedules and costs associated with multiple people missing work or school. Couple and family treatment also requires engagement of more clients with different priorities or goals. Unfortunately, unwillingness of family members to attend therapy sessions contributes to non-engagement in therapy (Wang et al., 2006).

Second, technology can reduce the stigma of face-to-face family and couple interventions – especially amongst men and children. Stigma significantly predicts likelihood of seeking mental health services, with higher self-stigma associated with fewer services sought (Lannin et al., 2016). Men are more likely than women to be reluctant about entering in-person couple therapy (Doss, Atkins, & Christensen, 2003). Moreover, children who are less willing to participate in in-person treatments show poorer treatment outcomes (Karver, Hendelsman, Fields, & Bickman, 2006).

Types of Technologically-Supported Interventions

In broad strokes, the application of technology to enhance, supplement, or replace in-person mental health interventions can be grouped into two categories – telemental health interventions and behavioral intervention technologies (BITs). Telemental health interventions refer to services that leverage videoconferencing media platforms to facilitate real-time interactions for the provision of mental health care that is traditionally delivered in person (American Telemedicine Association, 2017). BITs include applications of technology that are used to augment in-person treatments (e.g., an app to remind clients of homework assignments or track thoughts/behaviors) or provide self-help interventions without formal in-person treatment (e.g., an online program, with or without coach support). As these applications of technology differ in important ways, we cover each separately in this paper. For both telemental health and BITs, we will present recommendations for design and implementation, recruitment, and evaluation of these technologies. Throughout, we will draw from an extensive body of research that has been conducted on applications of technology for individual disorders (e.g., Marsch, Lord, & Dallery, 2014) and, where available, from the couple and family literature.

Telemental Health Interventions

The key distinguishing feature of telemental health is the use of synchronous interactive telecommunication technologies to enable providers to remotely deliver real-time services. However, telemental health interventions can additionally incorporate asynchronous store-and-forward communications between clients and providers (as well as related eHealth, mHealth, or sensor-based components). Across healthcare disciplines, mental health care may be particularly amenable to videoconferencing-based delivery, given that mental health care relies heavily on both verbal and non-verbal communication. Videoconferencing platforms now afford very high-quality, simultaneous, two-way audio and video transmissions. Accordingly, clinical interest in telemental health interventions for children, couples, and families, as well as focused research evaluating the potential of such programs, have sharply increased in recent years (e.g., Comer et al., 2017; Comer et al., in press; Myers et al., 2015; Vander Stoep et al., 2017). For example, Internet-delivered Parent-Child Interaction Therapy (I-PCIT; Comer et al., 2015) uses videoconferencing to deliver parent training to families in their own homes; the family uses a webcam to stream family interactions to a therapist who remotely coaches parents through a parent-worn Bluetooth earpiece. Delivering treatment in consumers’ own homes or natural settings increases the ecological validity of interventions for couples and families. For example, in parent training, having the child in his/her typical setting may increase the chances that the child responds in typical ways, providing more representative challenges for the parent and therapist to work through, and facilitating direct opportunities for generalization of treatment gains. Indeed, a recent randomized controlled evaluation found a significantly higher rate of excellent responders among families treated with I-PCIT relative to families treated with clinic-based PCIT (Comer et al., in press).

How Does One Design and Implement a Telemental Health Intervention?

Matters of security, confidentiality, comfort, engagement, and reimbursement are critical to consider when designing and implementing telemental health couple and family interventions.

Videoconferencing platforms

Videoconference platforms can be lumped into two main categories: standards-based or consumer-grade applications. Standards-based videoconferencing systems involve point-to-point transmission, using an application that is installed on the computer of both the provider and the client. Standards-based point-to-point applications allow for firewalls, encrypted communication, and in general, a highly secure interface. In general, nothing offers higher quality and security than T-1 and T-3 line connections, which provide point-to-point transmissions that do not go over the public Internet. However, T-1 and T-3 line connections, and related standards-based systems, are very expensive and can require considerable maintenance. As such standards-based systems can be cost-prohibitive for many providers and healthcare systems, particularly when telemental health interventions are delivered from one site to multiple locations (e.g., individual client homes) rather than to a single site (e.g., one primary care office in a rural community). More affordable and practical consumer-grade alternatives transmit communications over the public Internet from one Internet Protocol (IP) address to another, albeit with relatively slower speeds and somewhat reduced security. Many consumer-grade videoconferencing platforms (e.g., VSee) also offer screen sharing features that allow providers and clients to easily exchange images, progress graphs, or questionnaires, or to remotely collaborate on documents, in real time.

When using a consumer-grade videoconferencing platform, it is critical that the provider confirm that any videoconferencing platform selected: (a) does not store or retain any session information on its network after sessions are complete, (b) allows users to host meetings that are unlisted and requires the host to invite all attendees, (c) requires identification of all meeting attendants, (d) requires all meeting attendants use a strong password to log in, (e) requires approval of “Forgot Password” requests, and (f) uses the Advanced Encryption Standards (AES) or comparable algorithm and a secure encryption tunnel (see Chou, Comer, Turvey, Karr, & Spargo, 2016 for further discussion). While several such consumer-grade platforms are available, VSee is a leading provider this is relatively affordable and easy to use.

When selecting appropriate videoconferencing software, the most affordable options are free services, such as Skype, but such platforms offer somewhat poorer audio and video quality than other options (e.g., more disrupted or dropped calls), and importantly they often store and retain meeting data, which raises concerns about privacy and confidentiality. As such, we do not recommend free videoconferencing services, such as Skype, for delivery of telemental health services. It is recommended that providers and clients use password-protected computing devices (rather than mobile devices) and utilize Ethernet cords (not wireless connections).

Hardware

At a minimum, couple and family telemental health interventions typically require a computing device with online connectivity, webcam, and microphone at both the provider site and at the client site. Most modern-day computing devices include built-in cameras and microphones. Some telemental health programs, such as I-PCIT, entail remote bug-in-the-ear coaching and thus additionally require Bluetooth earpiece devices. Telemental health treatments that entail naturalistic observations of couples or families who may not always be facing the camera throughout the session require an omnidirectional room microphone to capture the full audio of the room, regardless of client position.

Delivery settings

Couple and family telemental health sessions are often grouped into “supervised” and “unsupervised” settings. A supervised setting is one in which clinical staff (albeit not the telemental health provider) is present at the client site. An example of telemental health delivered to a supervised setting is when one is remotely providing mental health care via videoconferencing to a child while that child sits in a pediatrician’s office with a nurse, physician, and/or other healthcare provider on site. Studies to date of telemental health for children and families delivered to supervised settings have demonstrated strong feasibility, acceptability, efficacy, and client satisfaction (e.g., Myers et al., 2015). Telemental health with certain high-risk client populations may always need to be delivered to a supervised setting. For example, it may be inappropriate to conduct remote sessions with families with maltreatment histories in the absence of a clinical provider at the client site.

It is not always feasible to deliver telemental health in supervised settings, and for some forms of couple or family telemental health it may even be preferable to deliver care to an unsupervised natural setting. Indeed, many telemental health interventions are intentionally designed to evaluate couples and families in their own homes rather than in clinic settings (e.g., I-PCIT; Comer, Furr et al., 2015; Comer et al., in press). Telemental health interventions delivered to unsupervised settings are an increasingly popular telemental health strategy (Luxton, Sirotin, & Mishkind, 2010) and have demonstrated initial feasibility, safety, and efficacy in a growing number of controlled trials (e.g., Comer, et al., 2017; Comer, et al., in press). However, safety concerns render telemental health treatment in unsupervised settings inappropriate for some clinical populations, a point to which we now turn.

Safety planning and crisis management

Using videoconferencing to provide remote care in unsupervised settings may be inappropriate for certain high-risk client populations (e.g., maltreating families, suicidal clients). However, clinical emergencies occur even after screening out particularly high-risk clients from telemental health practices. Prior to initiating treatment, the telemental health provider and client must develop a detailed risk management plan in the event that a crisis should occur. Telemental health providers should assess potential safety risks in the client site environment (e.g., firearms and/or household hazards). Therapists should also have a secondary way to contact the participant if technology fails in the middle of a session (e.g., a plan that the therapist will call the client’s cell phone). When telemental health is delivered to clinically supervised settings (e.g., local pediatrician’s office, community outpatient clinic, school site), local professionals must be familiar with emergency procedures. Recent telemental health guidelines (American Telemedicine Association, 2017) advise providers to always: (a) identify local emergency resources and phone numbers, (b) familiarize themselves with the location of the nearest emergency resources capable of managing mental health emergencies, and (c) store contact information for clients’ families and designated supports. During the consent process and throughout treatment, clients should be informed and reminded that in the case of an emergency or if they are unable to reach the therapist due to technology failure and require immediate assistance, they should call 911.

Technological literacy and comfort

Telemental health treatment progress is substantially hindered when participants do not have the knowledge to set up sessions and navigate the features of videoconferencing software. Prior to treatment, it is useful to conduct an informational session over the phone about the technological components of treatment and/or to send clients instructional manuals and troubleshooting resources. Usability is critical for the success of telemental health; interventions that are too complex for clients or for providers will not see widespread adoption and implementation. When resources allow, it is useful to have a dedicated staff member available to provide remote tech support as needed to clients during telemental health sessions. Telemental health providers should also check in frequently with clients to discuss any frustrations with technology and with treatment in general.

Therapeutic alliance and engagement

Clinical trials evaluating couple and family telemental health find high treatment retention, client engagement, client-therapist agreement on therapeutic tasks, client-therapist agreement on therapeutic goals, and client satisfaction, as well as positive affective bonds between clients and therapists (Comer, et al., 2017; Comer, et al., in press). Indeed, research consistently demonstrates alliance to be comparable across telemental health and office-based treatment modalities (Comer et al., 2017). These results are promising, but supplementary steps should nonetheless be taken to maintain engagement and the therapeutic alliance in videoconference-based treatment. Real-time screen sharing activities (and interactive games when children are involved in treatment) can meaningfully increase client engagement during telemental health and enhance client understanding of session content. For example, in a study of videoconferencing-delivered, family-based treatment for early-onset obsessive-compulsive disorder (OCD), Comer and colleagues (2017) created interactive computer games using Google Drawing and Google Documents that were collaboratively played in session by the child and therapist. One activity included an interactive “build a worry monster” game in which therapists and families selected cartoon monster features from an image bank to build a cartoon “Worry Monster” in order to help children externalize their OCD (see Comer, Furr et al., 2014).

Reimbursement

Telemental health payment issues vary widely by provider, services offered, population treated, and insurance provider. Medicaid reimbursement covers telemental health in most states (Center for Telehealth and eHealth Law, 2011) and even includes an additional “telehealth originating site facility fee.” States vary, however, as to whether they will reimburse for telemental health outside of delivery to rural or otherwise underserved regions. Unfortunately, many private payers do not currently reimburse for telemental health services; however, trends point to increasing coverage for telemental health by private insurers (Center for Telehealth and eHealth Law, 2011).

How Does One Recruit for a Telemental Health Intervention?

Many couple and family telemental health providers offer both clinic-based and videoconferencing-based treatment options, and their recruitment for telemental health services does not extend beyond recruitment efforts in place for clinic-based services. Accordingly, clients treated via videoconferencing are often those who initially reach out for services at a provider’s brick-and-mortar office, and then elect to participate in remote treatment options to improve the convenience of care. Broader recruitment efforts for telemental health are needed.

Successful recruitment methods for telemental health have included collaboration and engagement with pediatric and primary care practices (e.g., Myers et al., 2015), school settings (Stephan et al., 2016), and juvenile justice systems (Batastini, 2016). Outreach to rural and other underserved and remote communities is particularly needed (Nelson & Bui, 2010). Given the increasing role of the Internet as the first point of contact for couples and families seeking healthcare resources, telemental health providers must maintain a strong web presence.

Although telemental health increases opportunities to treat clients near and far, licensure and jurisdiction matters merit comment. At this time, most states require that providers be licensed in the state in which the client is receiving services (regardless of where the therapist is situated at the time of care; see Kramer et al., 2015 for a review of legal and regulatory issues). Although specific licensure regulations vary across states, for the most part a telemental health provider can only deliver telemental health services to clients in states in which the provider is licensed; thus, they are important considerations in recruitment.

How Does One Test a Telemental Health Intervention?

To continue to move the field of couples and family telemental health forward, researchers must systematically rely on research strategies that achieve favorable balances between scientific rigor and clinical relevance.

Control conditions

When designing a randomized clinical trial to evaluate a telemental health family or couple intervention, the first task is to select an appropriate control condition against which to compare the outcomes of the telemental health intervention. Here, we consider three primary options. First, “no-treatment” and “waitlist” controls assess the overall effect of the intervention but cannot meaningfully isolate the specific effects of the intervention or the effects of delivering the intervention via a videoconferencing portal.

A second control condition is one in which participants receive the office-based format of the same treatment. Such a comparison benchmarks outcomes against the same intervention delivered in person, with a common hypothesis being that there will be no difference between the two groups (i.e., a non-inferiority trial). In this design, an a priori decision is first made about the minimum difference between the conditions that would be meaningful enough to deem the groups nonequivalent (e.g., a 0.2 SD difference), and then tests are conducted to reject the null hypothesis that the observed difference between the two conditions is equal to the a priori minimum difference deemed meaningful (see Rogers et al., 1993). The only component manipulated is the delivery format (Internet vs. face-to-face), as treatment content is held constant, and condition differences can be attributed exclusively to the modality of care. Other trials might hypothesize that the telemental health format will yield a significant improvement over a standard clinic-based format, and indeed some recent trials have found telemental health formats to outperform parallel clinic-based care for family problems (e.g., Comer et al., in press).

Despite the advantages and rigor inherent in comparing a telemental health intervention to a parallel office-based format of that same intervention, such a comparison requires that participants live close enough to the investigative team so that they can feasibly participate in office-based treatment if that is the condition to which they are assigned. Findings from such a trial may not generalize to the more geographically remote and underserved populations for whom telemental health formats are most needed. As Chou, Bry, and Comer (in press) note, in order to position technology-based treatments for optimal uptake and reach, research needs to examine these treatments in the very populations that they are envisioned to serve.

Drawing on a “treatment as usual” (TAU) comparison—in which control participants are provided the standard of care that is routinely given to that population in their own community—increases the portability of a trial and allows the investigator to examine a telemental health couple or family intervention in more remote and underserved populations. Unlike trials comparing telemental health to the same treatment delivered in an office-based format, TAU comparisons do not require that participants live close enough to the investigative team so that they can feasibly participate in either treatment condition. Despite the benefits, however, what exactly constitutes “treatment as usual” is difficult to operationalize as it varies widely across regions and settings, making it difficult to integrate findings across studies. Further, differences between a telemental health intervention and a TAU condition might be attributed to differences in provider training, quality, or supervision, or to differences in the frequency, duration, or intensity of care, rather than to differences in the treatment delivery format.

Outcomes

Testing telemental health couple and family interventions requires one to decide what dependent variables to assess and how to assess them. In addition to standard clinical trial outcomes (e.g., symptom responses, diagnostic responses, improvements in functioning and quality of life), investigators of telemental health couple and family interventions are particularly interested in assessing the acceptability and feasibility of the novel intervention format. Matters of treatment satisfaction and therapeutic engagement and alliance are critical. Moreover, investigators of telemental health are wise to assess technological barriers to care, including technology-related disruptions to treatment delivery as well as negative technology-related attitudes and poor technological literacy that can interfere with treatment engagement or moderate treatment responses.

Finally, to facilitate subsequent cost-effectiveness analyses, one should record costs and savings associated with treatment delivery. These costs include, but are not limited to, technology to deliver the intervention (both client and therapist expenses), therapist salaries for initial training and ongoing treatment delivery, and supervisor salaries for training and ongoing supervision. Savings that will need to be recorded are beneficial effects of the program (on key outcome variables) as well as savings resulting from reductions in costs typically associated with in-person treatment (e.g., transportation, office space, child care, missed work or school).

Funding

The research needed to evaluate the promise and potential of telemental health will be a very costly endeavor. As Chou, Bry, and Comer (in press) note, current federal funding priorities in mental health are focused on an experimental therapeutics agenda—in which conceptual mechanisms underlying intervention effectiveness and neurobiological treatment targets are often prioritized over examinations of treatment outcomes and innovative delivery formats. Researchers working to develop and evaluate telemental health interventions will need to be increasingly creative in order to secure needed funding. Industry partnerships and newer funding sources (e.g., the Patient-Centered Outcomes Research Institute, or PCORI) may be critical for supporting the costly work needed.

Telemental Health Interventions or BITs?

Telemental health interventions hold great promise to transform the accessibility and scope of real-time mental health care with a provider by overcoming traditional geographic barriers and treating families in their natural settings. However, telemental health interventions for couples and families are not without limitations. Most notably, telemental health does not increase the overall number of individuals, couples, and families who can be treated at a given time. There is an enormous discrepancy between the large numbers in need of care and the relatively small numbers of providers. The discrepancy is so large that Kazdin and Blase (2011) estimated that even doubling the mental health workforce would yield only a minor public health impact. Using synchronous communication technologies to provide real-time services still requires a live provider for every session. As such, BITs that rely largely on self-administration and/or asynchronous communications with providers are better equipped than telemental health interventions to increase the number of individuals, couples, and families that can be treated.

Behavioral Intervention Technologies (BITs)

As described earlier, BITs either augment in-person treatment or provide self-help interventions without in-person treatment. Not surprisingly, BITs can take on many forms (Muñoz, 2016). Deciding which type of BIT suits a particular target population and intervention goals requires an understanding of the different practices.

Automated BITs

Automated couple and family BITs omit therapist, coach, or staff support and rely solely on technology to provide the intervention to consumers. This category of Automated BITs includes apps, websites, and/or phone lines that use voice or text recognition software to respond to client needs. Without the need of human support, Automated BITs have the potential to reach limitless number of clients continuously as there is no need to coordinate schedules or limitation of staff hours. Additionally, the potential for translation of these programs into different languages can further increase their reach. The quality of the Automated BIT can range from standardized information, such as learning about a particular disorder, to personalized prompts and feedback based on an array of previously entered or acquired data.

Guided BITs

In Guided BITs, like Automated BITs, technology provides the majority of the couple and family intervention. However, Guided BITs also include personal support in the form of a therapist or coach. There is great variety within Guided BITs as to the quantity and intensity of human support provided, ranging from the high end of using therapists throughout the intervention to the low end of using trained staff to aid in technological issues on an as-needed basis. Compared to in-person delivery of services and telemental health, Guided BITs have the potential to significantly decrease delivery costs, improve the reach of interventions across geographic boundaries, and increase the flexibility of services (e.g., because online activities can be completed on the user’s schedule, not on the provider’s schedule). However, Guided BITs still involve some staff contact and associated salary and training costs.

Compared to Automated BITs, Guided BITs provide users with a greater sense of accountability, encouragement, and motivation to both complete and engage in program content. Users in Guided BITs, compared to Automated BITs, tend to report higher levels of satisfaction with the intervention (Aardoom et al., 2016). Additionally, within couple and family interventions, guided BITs show superior completion rates and clinical outcomes compared to Automated BITs (Roddy, Nowlan, & Doss, 2016). However, with advancements in automated feedback (e.g., including, and tailored to, participants’ responses in the program), there is some emerging evidence that Automated BITs are getting closer to Guided BITs in their completion rates and clinical outcomes (e.g., Dear et al., 2015; Kelders et al., 2015; Titov et al., 2015). Our sense is that, with improvements in technology, automated BITs may soon be able to provide contingent and personalized feedback rivaling that found in Guided BITs. What remains to be seen is how important the personal connection with a coach, absent in even the best automated BITs, is in encouraging program completion.

Augmentation of traditional in-person services

BITs can also be used to augment traditional in-person couple and family services rather than substitute for them. Many therapists are already utilizing BITs in this manner (often without labeling it as such) by encouraging clients to use cell phone apps to track behaviors, moods, or automatic thoughts between in-person sessions or to monitor sleep and activity with wearable technology (Muñoz, 2016). Augmented BITs are easy to incorporate into clinicians’ current practice and, for clients who are accustomed to using their phones or computers daily, may be more attractive than diaries or worksheets for completing homework assignments. Additionally, the supplemental reminders and practice have been shown to improve treatment outcomes (e.g., Jones et al., 2014). However, Augmented BITs do little to increase the reach of interventions. Because clinicians incorporating augmented BITs provide the majority of clinical work in face-to-face sessions, the level of training to provide quality intervention is equally as high as training an in-person provider who does not use BITs. Furthermore, the provider’s ability to interact with the client via BITs is limited by the provider’s availability, therapy training, and knowledge of the technology.

How Does One Design and Implement a BIT?

After deciding to build a couple or family BIT, there are multiple decision points in the design process. Below, we include several considerations and their implications for BIT design.

Will the BIT will meet the intended need?

To ensure that one’s vision is aligned with couples’ or families’ perceived needs, conducting a series of focus groups is essential. The first two focus groups should consist of the target audience, with two groups being helpful to ensure that ideas converge across the two groups. In these groups, open-ended questions should dominate: Do they see a need? If so, what? What would they want to address that need? What would they be willing (and not willing) to do to address that need? Towards the end of the meeting, turning to more specific ideas for BIT content can be helpful. If one is contemplating building an Augmented BIT, utilizing an additional focus groups of providers that will be assigning and utilizing the BIT is important.

Who will be using the BIT?

With more than one user (e.g., both partners of a couple or coparenting team; a parent and child), one must consider which activities they will be doing together and which they will doing separately. Completing activities jointly allows for opportunities to discuss the material in the moment. However, joint completion also has its disadvantages. It may be more difficult for more than one person to view the content (if completed on a smartphone) and different reading speeds may create frustration. Working together may also limit each individual’s ability to reflect on their own internal experience, especially if users are asked to write in their responses. Completing the BIT together also requires both partners to be available and in the same place; in contrast, individual activities can be completed on the bus/train home from work or when parents work different schedules. Finally, if the dyad or family is experiencing conflict, joint completion of activities may increase the risk that the BIT would start an argument.

In the BITs we have developed, we have found a mixture of individual and joint activities to be the most useful when more than one user is involved (Doss, Benson, Georgia, & Christensen, 2013). For example, in the OurRelationship BIT (Doss et al., 2016), both members of a distressed couple work together to select, understand, and then solve a relationship problem. The majority of each phase – where users view tailored feedback about their relationship and write about how the ideas apply to their own relationship – is completed individually. At the end of each phase, couples are brought together in a joint activity to have a structured conversation where the program displays portions of what users wrote during the individual activities and the couple is encouraged to discuss them. However, other online BITs for relationships encourage couples to complete all activities together to maximize opportunities for discussion (e.g., Braithwaite & Fincham, 2014). Therefore, the best balance of joint and individual activities will depend on the BIT’s content, purpose, and audience.

Native apps versus web-based programs

Native apps, which are programs built for a particular device (usually a smartphone or tablet), have important advantages. They can be downloaded onto the device and subsequently used without an internet connection. Native apps also tend to interact more seamlessly with a device’s other functionality, which can be important if GPS data, motion sensors, or related functionality is a central part of the BIT. However, native apps also have their disadvantages. At a minimum, two versions need to be built – one for the Apple operating systems (iPhone, iPad) and one for Android systems (Android smartphones such as Samsung); additionally, depending on the functionality the app needs to harness, it may be necessary to have different versions for smartphones and tablets even within the same operating system. Those multiple builds increase costs for initial build and maintenance and additional complexity in managing the resulting data.

Web-based applications, in contrast, require an internet connection (via a computer or smartphone) and cannot take as much advantage of the device’s functionality. However, using a device’s internet browser, only a single version needs to be built (often at a lower cost than a single version of a native app), reducing both costs and complexity of a web-based BIT. In the rest of this article, we are going to focus on describing web-based Guided BITs for two reasons: a) currently, they are more cost effective for investigators seeking to develop their first BIT and b) we expect any information we would provide on native apps would become quickly outdated; in contrast, the recent rise of HTML5 as the dominant web language across devices and operating systems forecasts a relative stability of functionality required for web-based BITs.

Necessary components of a web-based BIT

If a web-based BIT (instead of a native app) is the best fit, one must consider the requirements of the eLearning content itself as well as the requirements of the platform in which the activities are housed.

If one’s goal is to deliver an intervention with text, audio, video, and some interactivity (e.g., clicking, drag-and-drop interactions), there are several excellent and relatively inexpensive options available; at this time, we recommend Articulate (www.articulate.com), Adobe Captivate, and Lectora. With a larger initial budget, the Adapt open-source functionality (www.AdaptLearning.org) is a good option. Content in Adapt is “responsive” (displays different content or using different layouts based on the size of the device’s screen). Additionally, Adapt has more options to interact with an online database, facilitating saving and recalling complex variables (e.g., calculated formulas) across the user’s sessions in the BIT.

In addition to the content of the BIT, one may also need to consider housing that content in a Learner Management System (LMS), such as Blackboard, Moodle, or Totara. An LMS allows one to require users to register for the intervention (restricting who has access to the content – critical for a randomized trial), make content availability contingent on completion of previous content, and tract usage of the content. An LMS will also be necessary if the online content needs to interface with a database – essential to save users’ responses in the program, to make content available across different sessions/devices, or to pull information from different accounts (e.g., allowing the wife to see a subset of the husband’s responses).

The role of coaches in a Guided BIT

Coaches in a Guided BIT tend to help users apply program content, reinforce fundamental skills, brainstorm solutions to any issues that may arise, and ensure that users are not misinterpreting content (Schueller et al., 2016). Coaches also provide a direct contact for technical assistance and problems with program functionality.

The professional training level of coaches does not seem to impact clinical outcomes. For example, Titov et al. (2010) found no significant differences between technician-based and clinician-based support in a BIT for depression, with both significantly improving symptom outcomes more than a waitlist control group. This pattern suggests that professional support may not be necessary for BITs to function successfully (Titov et al., 2010), especially since many Guided BITs are tightly scripted. As such, the use of paraprofessionals or graduate level trainees as coaches is both cost efficient and effective.

Furthermore, less may actually be more when it comes to the frequency of coach support. Indeed, BITs with about 90 total minutes of coach contact throughout the course of the intervention have been shown to outperform BITs that used therapist support as primary delivery of program content, suggesting that more frequent contact might negatively impact consumer volition (Schueller et al., 2016). Therefore, Schueller and colleagues (2016) recommend that BITs include few and regularly spaced opportunities for coach support. For example, users in the OurRelationship BIT are able to complete online content at their own pace until they reach three program milestones, at which points they are asked to meet with their coach before continuing. This model helps to ensure that program content is well understood before it is applied and expanded upon in later modules, while at the same time allowing users considerable flexibility to complete activities on their own schedules during the interim.

Contact modality, such as email, synchronous messaging, phone calls, and video conferencing, plays an important role in BIT adherence. Asynchronous emails have been found to help increase overall program use (Mohr et al., 2013); moreover, texts may be a more efficient medium than email (Schueller et al., 2016).

How Does One Recruit for a BIT?

For Augmented BITs, targeting the providers who will use the BIT will likely be the most efficient recruitment strategy; this type of recruitment approach is described in the Telemental Health section. For Automated or Guided BITs, effective recruitment strategies will likely include a combination of agreements with existing organizations and online recruitment targeting couples/families directly. If problem recognition and/or education about BITs is the advertising goal, then reaching out to the target audience – in person or online – is warranted. For example, on Facebook and using Google display ads, one can target potential participants based on their geographic location, race, ethnicity, education, income, and other interests / websites visited. In contrast, if one hopes to recruit samples who are already seeking help (but may or may not have decided to seek a BIT), then advertising where the population is likely to seek that help can be effective. For example, on online search engines such as Google and Bing, one can target advertisements to individuals’ searches related to presenting problems or interventions. If the BIT is an app, then advertising on iTunes and/or Google Play are natural ways to target individuals searching for apps related to your BIT.

How Does One Test a BIT?

There are four broad considerations in conducting an online RCT of a BIT – assessment, random assignment, study duration, and selection of appropriate control groups.

Assessment

Currently, the most popular online assessment programs are Qualtrics and SurveyMonkey; we suspect that most universities in the US have an institutional subscription to one or the other. Both programs provide numerous question types and complex logic functionality. Alternatively, if a BIT is paired with an LMS, many eLearning software programs such as Articulate and Adapt include integrated assessment functionally (enhancing user experience); however, the functionality tends to be more limited than dedicated assessment functionality such as Qualtrics. Additionally, given the relatively high rate of attrition in BITs, it is important to conduct assessments early in the intervention to obtain estimates of change for all participants. These early data points can be used to reduce bias in subsequent data analyses.

Outcomes

As described in detail in the telemental health section, measures of satisfaction and engagement are important for couple and family BITs as well. To construct cost-effectiveness estimates, in addition to the training and delivery costs detailed in the telemental health section, it is important to maintain accurate estimates of costs associated with initial development (and any subsequent refinement) of the BIT. Initial development costs for BITs are often significantly higher than ongoing costs to deliver the treatment and thus weigh heavily on cost estimates. In particular, costs associated with construction of the app / online content, LMS (if relevant), and online advertising should be recorded.

Duration

Given the rapidly-changing pace of technology, some authors have argued that it is inappropriate to restrict changes to an intervention during evaluation because, by the time the RCT is complete, the tested intervention is outdated (e.g., Mohr et al., 2015). If rapid recruitment is not possible, then authors have recommended conducting “trials of intervention principles” (Mohr et al., 2015) or SMART designs (Collins, Nahum-Shani, & Almirall, 2014) that permit inclusion of alternative or updated versions of an intervention.

Control conditions

When selecting a control group for a BIT RCT, many of the same considerations apply as for in-person studies. However, when recruiting participants online, it is especially important to consider that any participants assigned to a control group will likely have other options (outside of the study) at their fingertips. To reduce the number of control couples or families who seek services elsewhere, it may be beneficial to make the control group a wait-list control group (if the waiting period is not too long) or a viable alternative treatment (either another active intervention or a placebo-control intervention).

What are Important Ethical / Legal Considerations in Delivering and Testing a BIT?

Testing and delivering couple and family BITs adds several important ethical and legal considerations to those typically involved in face-to-face interventions.

License jurisdiction

In many cases, a BIT can be considered a self-help intervention (with or without “coach”, not “therapist”, support) and thus does not require a license to deliver. However, if the BIT provides services that can be understood to represent the practice of psychology that would otherwise require a license to provide, then one must be careful that the therapist is licensed in the state or jurisdiction where participants reside. Issues of license jurisdiction are described in more detail in the Telemental health section above.

Online assessment

In conducting online assessments, one important consideration is whether your data will need to be HIPAA compliant (including the HITECH Act – “Health Information Technology for Economic and Clinical Health” – and/or related DHHS rules). At the time of writing, both Qualtrics and SurveyMonkey require a premium level of services and a special contract that designates them as a Business Associate to be HIPAA compliant.

Video

If the couple or family intervention is subject to HIPAA regulations, then HIPAA-compliant video services need to be used. Although several are available, VSee is a provider that specializes in HIPAA-compliant videoconferencing at affordable rates. If an intervention is not subject to HIPAA, investigators should consider whether using a videoconferencing software that will be more familiar to users might be a better fit (especially for participants who are not technologically savvy). For example, FaceTime offers “end-to-end” encryption, meaning that Apple can access the content of the calls. Conversations over Skype and Google Hangouts are encrypted; however, this encryption is not currently “end-to-end”.

Emergencies

When concerning levels of domestic violence or suicidality are reported, we suggest participants should be immediately provided with both the website and phone number for the relevant hotline – either the National Domestic Violence Hotline or the National Suicide Prevention lifeline. Most assessment functionality allows for this contact information to be provided in real time in a conditional manner based on a user’s response to a previous item.

Mandatory reporting

Finally, if implementing a couple or family BIT across state lines (or internationally), investigators must consider that mandatory reporting guidelines for child and elder abuse vary by state. There are several helpful compilations of state laws regarding child abuse (https://www.childwelfare.gov/topics/systemwide/laws-policies/state/) and elder abuse (https://ncea.acl.gov/whatwedo/policy/state.html).

Technology’s Impact on the Conceptual Framework for Couple and Family Interventions

To this point, we have focused on the impact that technology can have in improving the efficacy or reach of interventions for couples and families. However, it is also important to consider how technology can affect the field’s conceptual framework. First, the increased reach of technology-enhanced interventions allows for more diverse and larger samples of couples and families, which in turn permits testing treatment moderators such as race, ethnicity, acculturation, income, education, or type of relationship problems. For instance, despite guidance on how to tailor interventions for African American couples (e.g., Boyd-Franklin, Kelly, & Durham, 2008), there have been only a handful of studies that have examined differential outcomes by race or ethnicity (e.g., Doss et al., 2012; Rhoades, 2015).

Second, technology allows us to collect new measures with increasing sophistication and specificity, improving our ability to identify and test mechanisms of our treatments. One of the central challenges of assessing mechanisms is that, to permit the strongest evidence of their role, mechanisms should be measured after they have been affected by the intervention but before they have a chance to change the ultimate outcome variable. Technology, with its ability to integrate more frequent measures of putative mechanisms, allows the opportunity to assess mechanisms without making a priori assumptions about the exact timeframe. More frequent assessments also increase the chance of detecting change in an individual before it affects the broader couple or family system. Additionally, technology increases the opportunities to collect implicit or behavioral measures (e.g., automated calculation of vocal stress during real-world conversations), increasing the chances that change in the mechanism can be captured before a participant can self-report on that change (and thus before it has the opportunity to affect the participant’s global evaluations).

Third, the availability of technology-enhanced interventions may inform repackaging of our treatments. For example, in the individual adult literatures, transdiagnostic interventions have gained significant momentum (e.g., Sauer-Zavala, Gutner, Farchione, Boettcher, Bullis, & Barlow, 2017); recent studies have further demonstrated that web-based transdiagnostic interventions seem to have similar effects across anxiety and depressive disorders (e.g., Dear et al., 2015; Titov et al., 2015). Although there have been recent calls to combine couple- and child-focused interventions, studies to date have utilized relatively small samples (e.g., Zemp, Milek, Cummings, Cina, & Bodenmann, 2016) which may have missed small- and medium-sized benefits of this combination. Furthermore, efforts to develop and test transdiagnostic interventions for youth (Queen, Barlow, & Ehrenreich-May, 2014) have been initiated but large-sample web-based studies would allow more detailed examination of these possibilities.

Conclusion

We believe the role of technology in couple and family interventions is expanding rapidly; the question is no longer if technology integration can be helpful but how and when it can be helpful. Indeed, technology has enormous potential to improve the efficacy of our face-to-face interventions, facilitate skill acquisition and application to real-world settings, decrease provider burden, and expand the reach of our interventions to couples and families who would otherwise not receive services. However, technology is not a panacea; if utilized incorrectly or inappropriately, it could decrease the efficacy of our treatments and increase risks for our clients. In this article, we have identified key considerations and provided conceptual and methodological guidance for investigators in the successful development, refinement, and evaluation of their couple and family telemental health interventions or BITs. Although the relatively young field of technology-based and technology-facilitated treatments has already amassed an impressive initial base of knowledge, many of the most important answers are still ahead of us. As in any new area of study, advances have emerged quickly, and we are just now starting to see consistency in terminology and consensus in methodology. Research and development efforts need to stay agile in order to keep pace with the rapidly evolving possibilities of modern technology. The conceptual frameworks and perspectives on technology-based interventions outlined throughout this paper can serve to guide the next wave of needed innovation and research in a manner that maximizes both scientific rigor and clinical utility.

Acknowledgments

This research was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under award R01HD059802 (to Brian D. Doss), and by the National Institute of Mental Health under award K23 MH090247 (to Jonathan S. Comer). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Brian D. Doss is co-creator of the intellectual property of the OurRelationship program and could gain royalties from any future commercialization of this intellectual property.

Contributor Information

Brian D. Doss, Department of Psychology, University of Miami

Leah K. Feinberg, Department of Psychology, Florida International University

Karen Rothman, Department of Psychology, University of Miami.

McKenzie K. Roddy, Department of Psychology, University of Miami

Jonathan S. Comer, Department of Psychology, Florida International University

References

  1. Aardoom JJ, Dingemans AE, Spinhoven P, van Ginkel JR, de Rooij M, van Furth EF. Web-based fully automated self-help with different levels of therapist support for individuals with eating disorder symptoms: A randomized controlled trial. Journal of Medical Internet Research. 2016;18(6):e159. doi: 10.2196/jmir.5709. http://doi.org/10.2196/jmir.5709. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. American Telemedicine Association. Practice guidelines for telemental health with children and adolescents. Washington, DC: American Telemedicine Association; 2017. [DOI] [PubMed] [Google Scholar]
  3. Batastini AB. Improving rehabilitative efforts for juvenile offenders through the use of telemental healthcare. Journal of Child and Family Psychopharmacology. 2016;26:273–277. doi: 10.1089/cap.2015.0011. [DOI] [PubMed] [Google Scholar]
  4. Braithwaite SR, Fincham FD. Computer-based prevention of intimate partner violence in marriage. Behaviour Research and Therapy. 2014;54:12–21. doi: 10.1016/j.brat.2013.12.006. [DOI] [PubMed] [Google Scholar]
  5. Boyd-Franklin N, Kelly S, Durham J. African American couples in therapy. In: Gurman AS, editor. Clinical Handbook of Couple Therapy. New York: Guilford Press; 2008. pp. 681–697. [Google Scholar]
  6. Centers for Disease Control and Prevention. Marriages and Divorces. 2015 Retrieved from http://www.cdc.gov/nchs/nvss/marriage-divorce.htm.
  7. Center for Telehealth and eHealth Law. 50 state Medicaid statute survey. Washington, D.C.: CTel; 2011. Retrieved from http://ctel.org/wp-content/uploads/2011/06/CTeL-50-State-Medicaid-Statute-Survey-Part-I.pdf on May 20, 2017. [Google Scholar]
  8. Chou T, Bry LJ, Comer JS. Overcoming traditional barriers only to encounter new ones: Doses of caution and direction as technology-enhanced treatments begin to “go live.”. Clinical Psychology: Science and Practice (in press) [Google Scholar]
  9. Chou T, Comer JS, Turvey CL, Karr A, Spargo G. Technological considerations for the delivery of real-time child telemental healthcare. Journal of Child and Adolescent Psychopharmacology. 2016;26:192–197. doi: 10.1089/cap.2015.0043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Collins LM, Nahum-Shani I, Almirall D. Optimization of behavioral dynamic treatment regimens based on the sequential, multiple assignment, randomized trial (SMART) Clinical Trials. 2014;11:426–434. doi: 10.1177/1740774514536795. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Comer JS, Barlow DH. The occasional case against broad dissemination and implementation: Retaining a role for specialty care in the delivery of psychological treatments. American Psychologist. 2014;69:1–18. doi: 10.1037/a0033582. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Comer JS, Furr JM, Cooper-Vince C, Madigan RJ, Chow C, Chan PT, Eyberg SM. Rationale and considerations for the Internet-based delivery of Parent-Child Interaction Therapy. Cognitive and Behavioral Practice. 2015;22:302–316. doi: 10.1016/j.cbpra.2014.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Comer JS, Furr JM, Kerns CE, Miguel E, Coxe S, Elkins RM, Freeman JB. Internet-delivered, family-based treatment for early-onset OCD: A pilot randomized trial. Journal of Consulting and Clinical Psychology. 2017;85:178–186. doi: 10.1037/ccp0000155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Comer JS, Furr JM, Miguel EM, Cooper-Vince CE, Carpenter AL, Elkins RM, Chase R. Remotely delivering real-time parent training to the home: An initial randomized trial of Internet-delivered Parent-Child Interaction Therapy (I-PCIT) Journal of Consulting and Clinical Psychology. doi: 10.1037/ccp0000230. (in press) [DOI] [PubMed] [Google Scholar]
  15. Dear BF, Staples LG, Terides MD, Karin E, Zou J, Johnston L, Gandy M, Titov N. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: A randomized controlled trial. Journal of Anxiety Disorders. 2015;36:63–77. doi: 10.1016/j.janxdis.2015.09.003. [DOI] [PubMed] [Google Scholar]
  16. Doss BD, Atkins DC, Christensen A. Who’s dragging their feet? Husbands and wives seeking marital therapy. Journal of Marital and Family Therapy. 2003;29:165–177. doi: 10.1111/j.1752-0606.2003.tb01198.x. [DOI] [PubMed] [Google Scholar]
  17. Doss BD, Benson LA, Georgia EJ, Christensen A. Translation of Integrative Behavioral Couple Therapy to a web-based intervention. Family Process. 2013;52:139–153. doi: 10.1037/t02125-000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Doss BD, Cicila LN, Georgia EJ, Roddy MK, Nowlan KM, Benson LA, Christensen A. A randomized controlled trial of the web-based OurRelationship program: Effects on relationship and individual functioning. Journal of Consulting and Clinical Psychology. 2016;84:285–296. doi: 10.1037/ccp0000063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Doss BD, Rhoades GK, Stanley SM, Markman HJ. Marital therapy, retreats, and books: The who, what, when, and why of relationship help-seeking. Journal of Marital and Family Therapy. 2009;35(1):18–29. doi: 10.1111/j.1752-0606.2008.00093.x. [DOI] [PubMed] [Google Scholar]
  20. Doss BD, Rowe LE, Morrison KR, Libet J, Birchler GR, Madsen JW, McQuaid J. Couple therapy for military veterans: Overall effectiveness and predictors of response. Behavior Therapy. 2012;43:216–227. doi: 10.1016/j.beth.2011.06.006. [DOI] [PubMed] [Google Scholar]
  21. Jones DJ, Forehand R, Cuellar J, Parent J, Honeycutt A, Khavjou O, Newey GA. Technology-enhanced program for child disruptive behavior disorders: Development and pilot randomized control trial. Journal of Clinical Child & Adolescent Psychology. 2014;43:88–101. doi: 10.1080/15374416.2013.822308. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Karver MS, Handelsman JB, Fields S, Bickman L. Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review. 2006;26(1):50–65. doi: 10.1016/j.cpr.2005.09.001. [DOI] [PubMed] [Google Scholar]
  23. Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science. 2011;6:21–37. doi: 10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
  24. Kelders SM, Bohlmeijer ET, Pots WTM, van Gemert-Pijnen JE. Comparing human and automated support for depression: Fractional factorial randomized controlled trial. Behaviour Research and Therapy. 2015;72:72–80. doi: 10.1016/j.brat.2015.06.014. [DOI] [PubMed] [Google Scholar]
  25. Kramer GM, Kinn JT, Mishkind MC. Legal, regulatory, and risk management issues in the use of technology to deliver mental health care. Cognitive and Behavioral Practice. 2015;22:258–268. [Google Scholar]
  26. Lannin DG, Vogel DL, Brenner RE, Abraham WT, Heath PJ. Does self-stigma reduce the probability of seeking mental health information? Journal of Counseling Psychology. 2016;63:351–358. doi: 10.1037/cou0000108. [DOI] [PubMed] [Google Scholar]
  27. Luxton DD, Sirotin AP, Mishkind MC. Safety of telemental healthcare delivered to clinically unsupervised settings: A systematic review. Telemedicine and e-Health. 2010;16:705–711. doi: 10.1089/tmj.2009.0179. [DOI] [PubMed] [Google Scholar]
  28. Marsch L, Lord S, Dallery J, editors. Behavioral healthcare and technology: Using science-based innovations to transform practice. New York: Oxford University Press; 2014. [Google Scholar]
  29. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, Begale M. A randomized controlled trial evaluating a manualized telecoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS ONE. 2013;8:e70086. doi: 10.1371/journal.pone.0070086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Mohr DC, Schuller SM, Riley WT, Brown CH, Cuijpers P, Duan N, Cheung K. Trials of intervention principles: Evaluation methods for evolving behavioral intervention technologies. Journal of Medical Internet Research. 2015;17:1–13. doi: 10.2196/jmir.4391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mukolo A, Heflinger CA, Wallston KA. The stigma of childhood mental disorders: A conceptual framework. Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49(2):92–198. doi: 10.1097/00004583-201002000-00003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Muñoz RF. The Efficiency Model of Support and the Creation of Digital Apothecaries. Clinical Psychology: Science and Practice. 2016 doi: 10.1111/cpsp.12174. [DOI] [Google Scholar]
  33. Myers K, Vander Stoep A, Zhou C, McCarty CA, Katon W. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: A community-based randomized controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry. 2015;54:263–274. doi: 10.1016/j.jaac.2015.01.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. National Organization of State Offices of Rural Health. Statement on reducing behavioral health disparities in rural communities. Sterling Heights, MI: National Organization of State Offices of Rural Health; 2011. [Google Scholar]
  35. Nelson E, Bui T. Rural telepsychology services for children and adolescents. Journal of Clinical Psychology: In Session. 2010;66:490–501. doi: 10.1002/jclp.20682. [DOI] [PubMed] [Google Scholar]
  36. Olfson M, Blanco C, Wang S, Laje G, Correll CU. National trends in the mental health care of children, adolescents, and adults by office-based physicians. JAMA Psychiatry. 2014;71(1):81–90. doi: 10.1001/jamapsychiatry.2013.3074. [DOI] [PubMed] [Google Scholar]
  37. Queen AH, Barlow DH, Ehrenreich-May J. The trajectories of adolescent anxiety and depressive symptoms over the course of a transdiagnostic treatment. Journal of Anxiety Disorders. 2014;28(6):511–521. doi: 10.1016/j.janxdis.2014.05.007. [DOI] [PubMed] [Google Scholar]
  38. Rhoades GK. The effectiveness of the Within Our Reach relationship education program for couples: Findings from a federal randomized trial. Family Process. 2015;54:672–685. doi: 10.1111/famp.12148. [DOI] [PubMed] [Google Scholar]
  39. Robles TF, Slatcher RB, Trombello JM, McGinn MM. Marital quality and health: A meta-analytic review. Psychological Bulletin. 2014;140(1):140–187. doi: 10.1037/a0031859. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Roddy MK, Nowlan KM, Doss BD. A randomized controlled trial of coach contact during a brief online intervention for distressed couples. Family Process. 2016 doi: 10.1111/famp.12262. [DOI] [PubMed] [Google Scholar]
  41. Rogers JL, Howard KI, Vessey JT. Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin. 1993;113:553–565. doi: 10.1037/0033-2909.113.3.553. [DOI] [PubMed] [Google Scholar]
  42. Sareen J, Jagdeo A, Cox BJ, Clara I, ten Have M, Belik SL, de Graaf R, Stein MB. Perceived barriers to mental health service utilization in the United States, Ontario, and the Netherlands. Psychiatric Services. 2007;58:357–364. doi: 10.1176/ps.2007.58.3.357. [DOI] [PubMed] [Google Scholar]
  43. Sauer-Zavala S, Gutner CA, Farchione TJ, Boettcher HT, Bullis JR, Barlow DH. Current definitions of ‘transdiagnostic’ in treatment development: A search for consensus. Behavior Therapy. 2017;48:128–138. doi: 10.1016/j.beth.2016.09.004. [DOI] [PubMed] [Google Scholar]
  44. Schueller SM, Tomasino KN, Mohr DC. Integrating human support into behavioral intervention technologies: The efficiency model of support. Clinical Psychology: Science And Practice. 2016 doi: 10.1111/cpsp.12173. [DOI] [Google Scholar]
  45. Smit F, Cuijpers P, Oostenbrink J, Batelaan N, de Graaf R, Beekman A. Costs of nine common mental disorders: implications for curative and preventive psychiatry. Journal of Mental Health Policy and Economics. 2006;9:193–200. [PubMed] [Google Scholar]
  46. Smith A. [Retrieved May 2, 2017];Record shares of Americans now own smartphones, have home broadband. 2017 www.pewresearch.org/fact-tank/2017/01/12/evolution-of-technology/
  47. Stephan S, Levar N, Bernstein L, Edwards S, Pruitt D. Telemental health in schools. Journal of Child and Adolescent Psychopharmacology. 2016;26:266–272. doi: 10.1089/cap.2015.0019. [DOI] [PubMed] [Google Scholar]
  48. Stewart JW, Bradford K, Higginbotham BJ, Skogrand L. Relationship help-seeking: A review of the efficacy and reach. Marriage & Family Review. 2016;52(8):781–803. doi: 10.1080/01494929.2016.1157559. [DOI] [Google Scholar]
  49. Titov N, Dear BF, Staples LG, Terides MD, Karin E, Sheehan J, McEvoy PM. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided internet-delivered treatment for major depressive disorder and comorbid anxiety disorders: A randomized controlled trial. Journal of Anxiety Disorders. 2015;35:88–102. doi: 10.1016/j.janxdis.2015.08.002. [DOI] [PubMed] [Google Scholar]
  50. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: A randomized controlled trial comparing clinician vs. technician assistance. Plos ONE. 2010;5(6):1–9. doi: 10.1371/journal.pone.0010939. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Vander Stoep A, McCarty CA, Zhou C, Rockhill CM, Schoenfelder EN, Myers KM. The children’s attention-deficit hyperactivity disorder telemental health treatment study: Caregiver outcomes. Journal of Abnormal Child Psychology. 2017;45:27–43. doi: 10.1007/s10802-016-0155-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Wang M, Sandberg J, Zavada A, Mittal M, Gosling A, Rosenberg T, Jeffrey A, McPheters J. “Almost there”…Why clients fail to engage in family therapy: An exploratory study. Contemporary Family Therapy. 2006;28:211–224. [Google Scholar]
  53. Whisman MA, Beach SR, Snyder DK. Is marital discord taxonic and can taxonic status be assessed reliably? Results from a national, representative sample of married couples. Journal of Consulting and Clinical Psychology. 2008;76:745. doi: 10.1037/0022-006X.76.5.745. [DOI] [PubMed] [Google Scholar]
  54. Zemp M, Milek A, Cummings EM, Cina A, Bodenmann G. How couple- and parenting-focused programs affect child behavioral problems: A randomized controlled trial. Journal of Child and Family Studies. 2016;25:798–810. doi: 10.1007/s10826-015-0260-1. [DOI] [Google Scholar]

RESOURCES