Paving the Way to Successful Implementation: Identifying Key Barriers to Use of Technology-Based Therapeutic Tools for Behavioral Health Care

Abstract

This study aimed to identify barriers to use of technology for behavioral health care from the perspective of care decision-makers at community behavioral health organizations. As part of a larger survey of technology readiness, 260 care decision-makers completed an open-ended question about perceived barriers to use of technology. Using the Consolidated Framework for Implementation Research (CFIR), qualitative analyses yielded barrier themes related to characteristics of technology (e.g., cost, privacy), potential end-users (e.g., technology literacy, attitudes about technology), organization structure and climate (e.g., budget, infrastructure), and factors external to organizations (e.g., broadband accessibility, reimbursement policies). Number of reported barriers was higher among respondents representing agencies with lower annual budgets and smaller client bases relative to higher budget, larger clientele organizations. Individual barriers were differentially associated with budget, size of client base, and geographic location. Results are discussed in light of implementation science frameworks and proactive strategies to address perceived obstacles to adoption and use of technology-based behavioral health tools.

Introduction

Rapid advancements in Internet and mobile technologies have given rise to the development and use of such technology-based tools for behavioral health care, including prevention and education,^1–3 screening and assessment,^4,5 treatment,^6–11 recovery support,^12,13 and wellness monitoring.¹⁴ Increasingly, technology-based therapeutic tools are recognized as having great potential for behavioral health care, including substance use and mental health^4,15 Technology-based therapeutic tools generally represent an approach to delivering evidence-based interventions (e.g., cognitive-behavioral therapy, motivational interviewing, contingency management) through technological (e.g., computer-, web-, mobile-based) platforms, either as stand-alone programs or as augments to care. Studies have consistently demonstrated that technology-based therapeutic approaches can work as well as, or better than, traditional therapeutic approaches delivered by trained clinicians.^{1–3,15–17} Technology-delivered therapeutic tools for behavioral health care offer the potential for on-demand access to care across time and geographic location, and broadened reach of services to those who are traditionally more disenfranchised or perceive stigma regarding service use.

Current trends in health care delivery support the need for flexible care processes that extend care outside the boundaries of the clinic, and technology is increasingly seen as a powerful tool to meet expanded care demands. The Patient Protection and Affordable Care Act of 2010 calls for methods to reduce health care costs and enhance treatment-related efficiencies that include health information technology (i.e., electronic health records) and use of technology-based treatment approaches to foster efficiencies in care delivery.¹⁸ With health care reform, demand for behavioral health care services is likely to exceed provider capacity, and technology-based care approaches have the potential to help bridge these increased service needs.

There is also strong consumer desire for technology-based health care services.¹⁹ For example, in one report, a majority of clients with severe mental illness were interested in receiving a variety of mental health services (e.g., reminders about appointments or medications, regular check-ins with provider) via mobile technologies.²⁰ Use of online and mobile technologies is increasingly ubiquitous across age, race/ethnicity, and geography.²¹ Increasingly, consumers rely on Internet and smartphone-based tools for health information and tracking.²¹

Despite growing evidence for technology-based approaches to behavioral health care, barriers to using such approaches are not well understood. Most research has focused on identifying barriers to implementation of electronic health record (EHR) systems, and the most prominent barriers to successful use of these systems include low IT literacy, provider resistance to change, cost, lack of adequate software, and data security concerns.^22–24 While similarities may exist, there are unique potential roadblocks associated with implementation of mobile- and web-based therapeutic tools relative to electronic health systems.¹³ For example, dynamic behavioral health technologies may present unique challenges regarding training and acceptability for both providers and clients relative to electronic health systems. Recent studies call for an examination of perceived barriers to implementing computer- and mobile-assisted interventions to uncover sources of provider resistance and organizational impediments to the use of these innovative tools.^25,26 To address this gap in knowledge, the current study aimed to answer the following question: What are the primary barriers to use of technology-based therapeutic tools identified by care decision-makers (i.e., health care providers with consistent influence over the content and types of clinical care delivered at their organizations) in behavioral health care settings?

This research was guided by the Consolidated Framework for Implementation Research (CFIR).²⁷ This unifying framework, derived from the diffusion of innovations²⁸ and related implementation models,²⁹ summarizes constructs related to adoption and implementation of health services innovations in four domains, including intra-organization characteristics (Inner Setting), characteristics of the external context within which an organization functions (External Setting), characteristics of the individuals that might use an intervention (Individual Characteristics), and attributes of an intervention itself (Intervention Characteristics). Each domain includes multiple constructs that represent key factors that influence implementation of health services innovations.

Intra-organization (Inner Setting) characteristics include structural features (i.e., age and leadership/staff stability, and size of the organization), communication dynamics between leadership and staff, and cultural values and norms with the organization. Implementation climate is another key Inner Setting characteristic. Successful implementation is more likely in climates with motivation to change, that are flexible for embracing innovation, and that have leadership support and infrastructure resources to support the innovation.

External (Outer Setting) factors associated with innovation adoption include policies, regulations, and incentives that could influence implementation, such as reimbursement policies and payer requirements. Inter-organization competitive pressure, the degree to which an organization is networked with other organizations, and consideration of patient needs and resources, are other examples of Outer Setting characteristics.

Characteristics of Individuals associated with innovation adoption include demographics (e.g., age, gender), professional experience and attitudes about new treatment approaches, and innovation-specific factors, such as knowledge and attitudes about the innovation, and prior experience and perceived self-efficacy using or promoting the innovation. Providers’ readiness to change their treatment approach is another key individual-level characteristic associated with adoption of innovations. Finally, Intervention Characteristics associated with implementation include the extent to which potential end-users perceive a clear advantage for using the innovation relative to other approaches, the ease of use of the innovation and extent to which it can be adapted to meet potential user needs, the strength of evidence for the innovation and quality of its presentation, the cost of the innovation, and the compatibility of the innovation with end-user values and needs.²⁷ Identification of barriers to adoption and implementation of technology-based therapeutic tools in each of these domains can provide a lens for guiding development of targeted solutions to promote adoption and implementation.

Previous research has indicated higher rates of health information technology use in agencies that maintain higher operating budgets,³⁰ have larger client bases,²³ and are located in non-rural areas,³¹ relative to lower budget, smaller and rural organizations. Given the importance of context for understanding challenges and developing appropriate implementation strategies, the authors also sought to inform the following question: Do identified barriers to implementation of technology-based behavioral health tools differ based on structural organization factors (i.e., agency size, annual operating budget, geographic location)?

It is also possible that individual respondent characteristics, such as age, gender or professional experience, could influence perceptions of barriers to use of technology-based therapeutic approaches. For instance, younger providers may be more “technology-savvy” than older providers, or individuals working in a position for a longer period of time may be more resistant to new innovations or, alternatively, feel more equipped to handle workplace changes, relative to those newer to the job. The current study also explores potential individual attributes related to perceived barriers to use of technology-based tools for behavioral health care.

Method

The data presented in this report were collected as part of a larger survey study to assess readiness to use technology-based therapeutic tools in behavioral health care conducted in partnership with the National Council of Behavioral Health (The National Council), a non-profit advocacy organization that services adults, children, and families nationwide. The National Council membership network is comprised of administrative leadership of approximately 1950 community behavioral healthcare agencies, including outpatient mental health and substance use organizations as well as primary care and federally qualified health centers (FQHCs). Through convenience sampling, an online survey hosted by professional-version survey software (SurveyMonkey) was distributed to the entire network. The recruitment email indicated that the goal of the study was to learn how technology-based therapeutic tools could foster new models of behavioral health care, particularly in light of health care reform. Technology-based behavioral health tools were described as education, screening, assessment, intervention, recovery support, or treatment monitoring delivered by way of web-based programs or mobile devices – either as stand-alone tools or to augment care. The following screening question at the outset of the online survey assessed eligibility: “Are you a clinician, clinical supervisor or manager, or an administrator who makes decisions about behavioral health care?” Respondents that answered in the affirmative received the survey. The study protocol was granted exemption by the Dartmouth College Institutional Review Board and was approved by the administration at the National Council.

Measures

The larger, parent survey included closed-ended items to assess organization climate and provider characteristics,³² as well as readiness to use technology in behavioral health care.³³ The survey also assessed individual respondent characteristics, including age, job position (i.e., administrator, director/supervisor, clinician) and job tenure (“How many years have you been in your current position?”), as well as organization characteristics, including annual operating budget, number of clients served per year, and geographic location (urban, rural, suburban). An open-ended item at the end of the survey asked participants to identify perceived barriers to implementing technology for behavioral health care at their agency, “In your view, what are the biggest barriers to use of technology-based therapeutic tools to enhance care delivery at your agency?” The current study explores the results of this open-ended question. Any response that identified at least one factor perceived to contribute to difficulties in the successful adoption, implementation, or sustainability of technology-based behavioral health tools was included in the analysis.

Sample

A total of 408 targeted participants completed the parent survey. Of these, 268 (66%) responded to the open-ended question. Participants represented at least 189 different organizations, but as self-identifying organization name was not required, the organizational reach is very likely to be even higher than this. These rates are consistent with, or more favorable than, prior research of similar survey methodology.^34,35 Eight respondents provided comments that could not be classified as a barrier. Therefore, valid respondents consisted of 260 administrative care decision makers. The truncated sample did not differ from the overall survey respondent sample with regard to demographic characteristics. Sample characteristics are depicted in Table 1, reflecting broad respondent demographics and geographic diversity [e.g., age range 27–74 years (M = 51); organizations represented 42 states and the District of Columbia].

Table 1.

Demographics of the Sample

Variables	N	%
Respondent Characteristics
Gender (Female)	158/259	61
Race/Ethnicity (Caucasian)	239/257	93
Position	253
CEO/Administrator	66	26
Director/Supervisor	166	66
Clinician/Provider	21	8
Organization Characteristics
Annual Operating Budget	260
< 1 million	5	2
< 1 – 4 million	45	17
5 – 10 million	53	20
> 10 million	116	45
Don’t Know	41	16
Clients Served per Year	259
< 500	13	5
500 – 900	25	10
1000 – 3000	70	27
> 3000	130	50
Don’t Know	21	8
Services Offered	260
Mental Health	245	94
Drug	178	69
Alcohol	176	68
Primary Care	49	19
Sexual Health	29	11
Medical Specialty Care	12	5
Service Regions	260
Rural	160	62
Urban	153	59
Suburban	128	49
All Three Regions	61	24

Analytic Strategy

Open-ended responses were exported into a qualitative coding software program (ATLAS.ti, version 5.5) for coding and content analysis through a directed approach.³⁶ All relevant comments were assigned a thematic code using a coding directory, which was established using a combination of deductive and inductive approaches.^37–39 Prior to the directed content analysis, several a priori barrier themes were entered into the coding directory (i.e., Cost, Privacy, Technology Literacy, Client Access, and Provider Resistance); these themes were based on prior research^22–24 and grounded in the key domains from the CFIR implementation framework.²⁷

Data were coded by two study personnel trained in qualitative data analysis. The primary coder developed an initial coding directory. As anticipated, several concepts were addressed that did not fit with the a priori themes. As new concepts surfaced, additional codes were entered into the coding directory and linked to relevant comments. The content analysis process was iterative so that all responses were reviewed subsequent to development of new thematic codes.

A second coder conducted separate content analyses with a subset of responses using the unlinked open-ended responses and the coding directory with the thematic codes. Inter-rater agreement was evaluated in terms of assignment of the same specific code to each relevant comment (87.1% agreement; kappa=85.3%). Coding discrepancies were addressed and amended collaboratively by both coders, and the full set of responses was then coded (464 comments). Within a participant’s overall response, multiple codes could be assigned but duplicate codes were avoided.

Finalized codes were entered into PASW Statistics, version 18.0 (SPSS, Inc., 2009) on a respondent level. For each respondent, all barrier themes were coded to reflect whether or not the barrier was identified by the respondent (0=no; 1=yes). Primary barrier themes are reported and discussed below. ANOVA and linear and logistic regression analyses were conducted to assess differences among primary barriers based on individual respondent (age, position, job tenure) and organization factors (annual budget, number of clients per year, geographic location).

Results

Primary Barrier Themes

Directed content analysis yielded 11 distinct barrier themes aligned with each of the thematic domains of the CFIR implementation framework, and related to characteristics of the technology (Intervention Characteristics), potential end-users (Characteristics of Individuals), inner organization structure and climate (Inner Setting), and factors external to organizations that could influence implementation (Outer Setting). As seen in Figure 1, themes coalesced around: 1) Funding and cost (mentioned by 46% of respondents); 2) Privacy and security (34%); 3) Need for knowledge and skill building (19%); 4) Equipment and infrastructure (16%); 5) Perceived negative impact or previous bad experiences (15%); 6) Client access or maintenance of tools (15%); 7) Provider or agency openness or buy-in (14%); 8) Work and time demands (10%); 9) Staffing and IT support (7%); 10) Client Internet connectivity (5%); and 11) Billing and reimbursement (5%).

Figure 1.

Frequency of Endorsing Barriers to Implementation of Behavioral Health Technologies

Detailed conceptualization of each barrier theme is depicted in Table 2. Table 2 also maps each identified barrier theme with one or more of the relevant CFIR domains (i.e., Intervention Characteristics, Outer Setting, Inner Setting, and Characteristics of Individuals) and one or more of the accompanying CFIR constructs within a domain (e.g., Complexity, External Policies, Implementation Climate, Individual Stage of Change). Barrier themes corresponded in varying degrees with each key CFIR domain. The Inner Setting domain was represented by six of the barrier themes, the Outer Setting domain was represented by five of the barrier themes, the Intervention Characteristics domain was represented by four of the barrier themes, and the Characteristics of Individuals domain was represented by three of the barrier themes (see Table 2). Five of the barrier themes were characterized by only one key CFIR domain, five of the barrier themes corresponded with two key CFIR domains, and one of the barrier themes was related to three key CFIR domains.

Table 2.

Primary Barriers Themes

Barrier Theme	CFIR Domains(s)	CFIR Construct(s)	Conceptualization	Representative Quotes
Funding / Cost	Intervention Characteristic	Cost	Barriers involving high cost or affordability of technology-based tools, budget constraints or cuts, lack of internal or external funding prioritization or investment.	Budget strains and changes in funding model that expect more for less put major strain on clinicians and administrators; Agencies cannot afford high-tech systems that would connect us in real time to other behavioral healthcare providers.
	Outer Setting	External Policies		The cost factor is daunting. We continue to purchase workstations and signature pads to meet the demands of staff growth. There is often not enough money to address the other technology needs on the wish list.
Privacy / Security / Regulations / Ethics	Outer Setting	External Policies	Concerns about privacy, security, confidentiality, and HIPAA violations regarding patient health information. Also included were perceived barriers related to organizational (agency-level regulations, restrictions, and ethical considerations of technology use in care delivery.	Protecting health information (trying to address privacy issues and the expansion of phone technology - i.e. for adolescents texting is the preferred mode of communication, and it seems the “rules” need to catch up to technology growth).
	Inner Setting	Implementation Climate		I think the biggest barrier is protection of confidential information. I believe there should be concerted efforts to change 42CFR in order that care coordination can occur more easily via technology.
Need for Knowledge / Skill Building	Characteristic of Individuals	Knowledge about Intervention / Self-efficacy	Barriers related to a lack of necessary provider knowledge, skills, or abilities to use technology-based tools, or the perceived need for skills training or other professional development resources.	Some of the barriers that concern me are related to the process of learning how to use new technological tools.
	Inner Setting	Available Resources / Access to Information and Knowledge		Also, lack of awareness. As an agency, we are currently transitioning to electronic medical records. We also use computer-based measures for assessment administration and scoring… However, beyond this, I think there is no real knowledge or understanding about how technology could be used in the therapy room.
Equipment / Infrastructure	Intervention Characteristic	Complexity	Implementation barriers due to old, outdated, unreliable, or inadequate equipment, technology, systems, or IT infrastructure.	The short life span of technology and constant updating creates challenges when different information systems have to work together.
	Inner Setting	Available Resources		Biggest current barriers are outdated computer system that prevents newer technology. Some employees have desk top computers and many others have “thin clients” with very little power for many new technologies.
Negative Impact / Bad Experiences	Intervention Characteristic	Relative Advantage	Barriers related to perceptions of a potentially detrimental impact of technology-based tools on the provider-client relationship or quality of care delivery, or concerns about previous negative experiences utilizing technology.	Nothing, in my estimation, can replace face to face contact with clients. The biggest barrier that I can see is that technology- based tools place a chasm of mistrust between client and treatment that is very valuable in developing an ongoing recovery.
	Characteristic of Individuals	Beliefs about Intervention		As therapists, I think there is a significant need for healthy boundaries, and I would not want my clients to be able to contact me via smart phone any day or time of the week. I also have major concerns about client emailing…email interface changes the dynamic of communication and yet as therapists we are still accountable for what is said and how we respond. This could be quite difficult to track and monitor via email.
Client Access / Maintenance of Tools	Outer Setting	Patient Needs and Resources	Barriers related to clients’ inability to access, obtain, understand, maintain, or properly utilize technology-based tools	Clients have limited resources and are often quite mobile (frequent change of location and phone numbers); they also lack consistency in following the suggested approaches which makes tracking or assessing effectiveness of technology based interventions a challenge.
				Our customer base is primarily indigent and in a recent survey over 90% reporting not having access to a computer, a smartphone and/or internet in their living environment.
Provider / Agency Openness / Buy-In	Characteristic of Individuals	Beliefs about Intervention	Barriers related to resistance or lack of openness or acceptance to using technology-based tools in clinicians, administrators, other health care providers, or the health care agency as a whole.	Certain staff are resistant to the use of technology-based therapeutic tools which affects other staff’s progress in this area.
	Inner Setting	Implementation Climate		We are an evidence-based CMHC, so before we “embrace” any new approaches, we want to review credible, unbiased evidence supporting that approach. We have yet to see any interventional trials that compare treatment as usual (TAU) vs. technology alternatives. We would demand such evidence before encouraging our clinicians (or spending money) on new approaches. We see this conservative approach as an important extension of our stewardship of community resources.
	Intervention Characteristic	Evidence Strength and Quality
Demands / Time	Inner Setting	Available Resources	Barriers related to an excess of current work demands and a lack of time to implement technology-based tools.	We have turned down great, free training opportunities if they require more than 1 day commitment due to pressures to meet program outcomes/contractual obligations or revenue demands.
				Time needed is a significant problem for direct care staff, supervisors and directors, and leadership.
Staffing / Support	Inner Setting	Available Resources/ Access to Information and Knowledge	Concerns regarding a lack of appropriate staff members or necessary information technology (IT) support to accommodate the needs of technology implementation.	We do not have adequate IT staff to support the organization--programmers, analysts etc.
				The lack of modern systems savvy people to ensure proper acquisition of EFFECTIVE technology to meet client and staff needs.
Client Internet Connectivity / Service	Outer Setting	Patient Resources	Concerns related to clients’ inability to connect to the Internet or receive mobile service due to lack of service coverage, inadequate service providers, or other related issues.	The variance in ISP [internet service provider] upload speed affects clients’ ability to access telehealth products, especially group therapy.
				Getting service reception in remote rural areas is an issue.
Billing / Reimbursement	Outer Setting	External Policies and Incentives	Concerns regarding an inability to charge a fee for service, bill, or otherwise be reimbursed for services provided via technology.	The use of electronic tools increase the amount of time the clinicians spend with non-direct contact which is not billable.
				Treatment using new technologies is often not covered under fee for service.

Number of Perceived Barriers by Organization and Respondent Characteristics

Individual respondents endorsed a range of barriers (range: 1–6; Mean = 1.87, Median = 2.00, SD = 0.98). ANOVA and multiple regression analyses were conducted to examine differences in number of reported barriers based on organization (budget, size, geographic location) and respondent (age, job tenure) characteristics.

Analyses revealed two factors associated with number of barriers endorsed (see Table 3). Comparatively fewer implementation barriers were reported by respondents that represented agencies with higher annual operating budgets and higher numbers of clients served per year (i.e., greater size), relative to their lower budget and smaller counterparts. More specifically, respondents from agencies with annual operating budgets of greater than $10 million (M = 1.77; SD = 0.86) reported significantly fewer barriers to the implementation of technology-based behavioral health tools than those with budgets of $10 million or less (M = 2.08; SD = 1.12), F(1, 217) = 5.379, p = .021, d = 0.31 (small-to-medium effect). Similarly, respondents of agencies serving more than 3,000 clients per year (M = 1.70; SD = 0.87) reported significantly fewer implementation barriers than those serving 3,000 clients or less per year (M = 2.08; SD = 1.08), F(1, 236) = 9.234, p = .003, d = 0.39 (small-to-medium effect). Organization geographic location was not related to number of reported barriers. There were also no significant differences in number of barriers for any respondent-level characteristics (i.e., gender, age, years on the job).

Table 3.

Group Differences in Number of Barriers Endorsed

Variables	Number of Barriers
	N	Mean (SD)	d	P
Annual Operating Budget	219	---		*
Up to 10 million	103	2.08 (1.12)
> 10 million	116	1.77 (0.86)	0.31
Clients Served per Year	238	---		**
Up to 3000	108	2.08 (1.08)
> 3000	130	1.70 (0.87)	0.39
Rural-only Setting	260	---		ns
No	202	1.91 (1.01)
Yes	58	1.71 (0.84)	0.22

Note. ns P ≥ .05;

^*P < .05;

^**P < .01;

^***P < .001

Organization and Respondent Characteristics Associated with Perceived Barriers

A series of logistic regressions was conducted to examine the association of organization characteristics with the different barriers. As shown in Table 4, logistic regression analyses indicated that each key organization characteristic (i.e., budget, clients served, geographic setting) was associated with one implementation barrier. Annual operating budget was inversely related to endorsement of the Privacy/Security barrier, such that respondents of agencies with an annual operating budget of less than $10 million were more than twice as likely to identify privacy- or security-related concerns as those with a budget of $10 million or more. Number of clients served per year was negatively associated with endorsement of the Funding/Cost barrier, such that respondents from smaller agencies (i.e., serving less than 3,000 clients per year) were more than twice as likely to report concerns related to high cost or lack of funding as those from larger agencies (3,000 clients or more per year). Finally, respondents from rural organizations were more than 10 times as likely to endorse Client Internet Connectivity/Service as a barrier to technology use as those from other geographic locations. There were no other differences in reported barriers by respondent organization characteristics. There were also no respondent-level differences for perceived barriers.

Table 4.

Logistic Regression Analyses of Organizational Factors Predicting Implementation Barriers

Predictor	Client Connectivity
	B	OR	(95% CI)	Wald χ2	df	p
Annual Operating Budget	.038	1.038	0.337–3.195	0.004	1	.948
Number of Clients	−1.052	.349	0.104–1.167	2.919	1	.088
Rural-only Setting	2.333	10.312	3.101–34.295	14.485	1	<.001
	Funding
Annual Operating Budget	−.375	.687	0.403–1.172	1.896	1	.168
Number of Clients	−.759	.468	0.278–0.788	8.161	1	.004
Rural-only Setting	.130	1.138	0.634–2.043	0.189	1	.664
	Privacy / Security
Annual Operating Budget	−.794	.452	0.257–0.796	7.573	1	.006
Number of Clients	−.436	.646	0.379–1.104	2.556	1	.110
Rural-only Setting	.397	1.488	0.816–2.711	1.683	1	.194

Discussion

This study aimed to enhance the knowledge base regarding the perceived barriers of clinical care decision makers to implementation of technology-based therapeutic tools for delivery of behavioral health care. To the best of the authors’ knowledge, this is the first survey of a large network of behavioral health care organizations to examine barriers to use and implementation of such tools. Several key findings emerged that will help guide further research, practice and policy. As expected, the barriers identified in this study, as well as the associated organizational factors (i.e., budget, size, geographic location), also align closely with the key domains comprising the CFIR implementation framework.

The highly prevalent funding or cost-related concerns are certainly an attribute of the Outer Setting (i.e., external funding climate), but may also be a function of an Intervention Characteristic (i.e., costs). Given the limited resources in most health service agencies, purchasing of mobile apps or licenses for internet-based assessment or treatment programs are often perceived as untenable. The current results suggest that it is imperative to clearly demonstrate the added value and short- and long-term cost savings of integrating innovative technology-based tools into care delivery by way of rigorous comparative effectiveness and cost-effectiveness studies. There is some research to suggest that organizations are able to recoup the high initial costs of technology-based data collection tools in as little as three months, but rigorous cost analysis data is largely unavailable, indicating a significant gap in the field.^40,41 Organizations will be acutely aware of the upfront costs to implementing technology-based tools, so scientific evidence to predict recoupment of losses, as well as potential long-term cost savings, will be critically important to gaining greater buy-in. Such research should include clear metrics of value and efficiency for providers (i.e., improved workflow, increased client reach, more time to focus high need clients) as well as organizations (i.e., return-on-investment). Demonstration trials of technology-based approaches using different payer models may help to identify reimbursement models that maximize outcomes in relation to fiscal impact. In all cases, research results should be communicated to health care agencies, payers and policy makers in ways that are accessible, relevant and meaningful to the array of stakeholders.

The second most mentioned barrier—concerns surrounding privacy and security of information—relates to both the Outer Setting (i.e., external policy regulations) and Inner Setting (i.e., internal restrictions on technology use) domains. Much of the reported concern among respondents reflects the current system-wide policy regulations and restrictions regarding privacy, confidentiality, and security of protected health information (e.g., HIPAA, 42CFR Part II). However, responses also suggest that some organizations are more willing to accept certain levels of risk in health care delivery or have greater capacity to understand and navigate the external policy regulations than others, which influences the agency-level guidelines and restrictions put into place. The salience of privacy and security concerns indicates that the health care field must collectively work to clearly explicate protocols for the development of technology-based therapeutic tools that will facilitate client protection and regulatory guideline compliance.

The identified need for knowledge and skill building comprises a characteristic of health care providers adopting a tool (i.e., Characteristic of Individuals domain) but may also be a function of the availability of resources and access to information and knowledge within an organization (i.e., Inner Setting domain). Easy to access, clear and comprehensive education about specific ways that individuals can protect themselves with use of different technologies is also sorely needed. A substantial portion of respondents reported a lack of requisite knowledge about how technologies can be used for behavioral health care. This finding suggests a strong need for broad dissemination of information about available evidence-based technological tools, accessible and thorough training in the use of technology-based tools within different systems of care, and ongoing technical assistance to help stakeholders within organizations build the skills and confidence to implement technology-based tools within their settings. Future research should compare the effectiveness of these dissemination and implementation strategies to determine how to best enhance provider knowledge and skill sets.

Several of the primary barriers (e.g., Openness/Buy-In) were centered on collective attitudes and perspectives of those within the organization, such as leaders in the position of making clinical care (and therefore adoption) decisions, about technology-based approaches. These barriers align closely with the Inner Setting domain (i.e., implementation climate). Responses along this barrier theme were also representative of the Characteristic of Individuals (i.e., individual readiness for change) and Intervention Characteristics (i.e., perceived validity of evidence supporting technology) domains of the CFIR. Strategies to address these complex barriers should be multi-faceted, but dissemination of empirical evidence throughout an organization represents a key starting point. To promote adoption of technology, dissemination efforts should aim to debunk fears cited by respondents, such as compromised patient care and job replacement. Instead, the introduction of these tools should be framed as a way to allow providers to work at their highest level of training and focus on the most high need issues. Additionally, careful efforts must be made in the development, dissemination, and implementation of technology-based behavioral health tools to consider the important role of interaction between clients and clinicians.

The underlying mechanisms of provider resistance and lack of openness to use technology-based care approaches may be multi-faceted, including limited awareness of established benefits, an organizational climate characterized by skepticism or unwillingness to try new approaches, or a demand for more research on the effectiveness and safety of these tools. The fit of any innovation with the attitudes and values of the agency and providers adopting it is critical to the acceptability, efficiency, and effectiveness of the implementation process.^42,43 The concept of perceived fit is also reflected in the Inner Setting domain of the CFIR. The Implementation Climate and Compatibility constructs of the Inner Setting domain suggests that alignment between an adopter’s perceptions of an intervention (as shaped by his or her personal attitudes, values, and needs) and the larger organization’s interpretation of the intervention, contributes to the likelihood of successful implementation.²⁷ Fortunately, through investment in comparative effectiveness research, demonstration pilots that allow for first-hand experience of these tools by providers and consumers, and enhanced dissemination of key findings to both frontline providers and organizational leaders, perceived fit can be improved at all levels, in turn enhancing openness and buy-in for the innovation.

Although respondent-level factors did not seem to play a role in the number or type of barriers endorsed by respondents, the current authors cannot rule out the possibility that respondent-level differences (i.e., between different provider roles) would not have emerged in a more heterogeneous sample. While this study was focused on key care decision-makers, future research efforts should strive to capture the perspectives of a more diverse group of healthcare workers. Regarding organization-level differences, more frequent endorsement of barriers in lower budget and smaller agencies (i.e., structural characteristics of the inner organizational setting) may reflect the multiple challenges in technology-related implementation faced by agencies with relatively few resources. Special efforts should be made with lower resource agencies to employ strategies responsive to multiple implementation barriers and to perform demonstration trials in these organizations as a potential means for increasing subsequent confidence, skills, and desire for more sustained uses of technology.

Interestingly, perceived barriers were fairly robust across agencies of varying operating budgets, client capacity, and geographic setting. Internet connectivity and mobile service are clearly issues external to organizations that remain primary impediments to agencies’ provision of technology-delivered services within rural areas. While wireless network coverage is slowly expanding, other technology-related strategies can be implemented in rural areas in the meantime, such as mobile applications that include full offline capabilities, for instance.

Limitations

One limitation of the study is that perceived barriers were assessed with a single open-ended item. Richer qualitative information could be obtained through key informant interviews, focus groups, and observational methods. However, participants largely provided detailed, thoughtful responses that reflected their perceptions of the most salient issues. Further, as the intent was to capture perceived implementation barriers, the survey-based method represented an efficient and effective way to capture relatively rich qualitative data from a large and diverse group of representative respondents.

Another limitation of the study is that clinician-level respondents were under-represented. The current study intended to identify perceived implementation barriers among care decision-makers within mental and behavioral health care agencies, recognizing that this would likely include but under-represent front-line providers. These respondents may comprise a highly involved and actively influential group of clinicians, making them appropriate for inclusion in this sample. However, comparable research that centrally focuses on clinician-level providers, other front-line staff, and consumers would add important perspectives to further elucidate strategies to promote adoption and implementation of technology-based behavioral health tools.

Implications for Behavioral Health

Stakeholders involved in the development, dissemination, and implementation of technology-based behavioral health tools must become acutely aware of the primary impediments to the adoption and use of these tools within behavioral health care agencies. Rapid development and evaluation of technology-based interventions and assessments, without adequate knowledge of, or attention to, organizational bottlenecks and other implementation barriers may be contributing to a growing research-practice gap. The current research identifies these barriers, highlights the organizational contexts in which these impediments are particularly salient, and relates each barrier to key domains of the highly-regarded CFIR framework. Use of conceptual frameworks, such as the CFIR, can help guide implementation science research to elucidate mechanisms of successful adoption and implementation of technology-based therapeutic tools. These findings emphasize the importance of developing technology-based therapeutic tools that are lower cost, safer, and more responsive to the needs and perspectives of behavioral health care providers. These results may also help inform stakeholders seeking proactive strategies to address major obstacles to the implementation and use of technology-based behavioral health tools within community health care settings. Together, these advancements may facilitate the process of adoption and contribute to greater spread, scale-up, and sustainability of technology-based therapeutic tools.