Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Nov 1.
Published in final edited form as: Adm Policy Ment Health. 2015 Nov;42(6):672–681. doi: 10.1007/s10488-014-0602-0

It Takes a Village: A Mixed Method Analysis of Inner Setting Variables and Dialectical Behavior Therapy Implementation

Matthew S Ditty 1, Sara J Landes 2,3, Andrea Doyle 4, Rinad S Beidas 5
PMCID: PMC4400206  NIHMSID: NIHMS635625  PMID: 25315183

Abstract

Guided by the Consolidated Framework for Implementation Research (CFIR), this mixed method study explored the relationship between inner setting variables and Dialectical Behavior Therapy (DBT) implementation. Intensively trained DBT clinicians completed an online quantitative survey (n = 79) and a subset were sequentially interviewed using qualitative methods (n = 20) to identify relationships between inner setting variables and DBT implementation. Four interpersonal variables - team cohesion, team communication, team climate, and supervision - were correlated with the quantity of DBT elements implemented. Qualitative themes corroborated these findings. Additional variables were connected to implementation by either quantitative or qualitative findings, but not both.

Keywords: Dialectical Behavior Therapy, implementation, inner setting

Evidence-based psychotherapies (EBPs) have displayed meaningful clinical outcomes in research trials, but because EBPs are largely underutilized in real world settings, individuals in need often do not receive them (Hogan, 2003; McHugh & Barlow, 2010). If EBPs are to benefit more individuals, implementation processes in real world settings must be examined in order to provide effective care for those in need (Aarons & Palinkas, 2007; Fixsen et al., 2005; Fixsen et al., 2009).

A number of frameworks have identified constructs that impact implementation at the individual, organizational, and system-levels (Aarons, Hurlburt, & Horwitz, 2011; Damschroder et al., 2009). The individual-level includes characteristics of stakeholders involved in service delivery (e.g., a provider's knowledge of Dialectical Behavior Therapy; DBT). The organizational level includes characteristics of the setting where services are provided (e.g., an organization's culture of treating suicidal individuals). The system level refers to characteristics of the broader economic, social, and legal context where services are delivered (e.g., societal attitudes toward individuals with Borderline Personality Disorder; BPD; Aarons, Hurlburt, & Horwitz, 2011; Damschroder et al., 2009). Each level is addressed in The Consolidated Framework for Implementation Research (CFIR), a framework that includes a comprehensive list of constructs thought to influence implementation (Damschroder et al., 2009).

DBT (Linehan, 1993) is an EBP developed to treat suicidal behavior, BPD, and other high-risk behaviors (e.g., non-suicidal self-injury). DBT is a principle-based intervention with four standard modes of treatment in an outpatient setting: weekly individual therapy, weekly group skills training, skills coaching by phone, and weekly therapist consultation team. Within and across each mode are specified strategies, forming a complex array of treatment elements (e.g. each consultation team designates a team leader). To date, there have been at least 11 randomized controlled trials of DBT (Bohus et al., 2004; Carter et al., 2010; Koons et al., 2001; Linehan et al., 1991; Linehan et al., 1999; Linehan et al., 2002; Linehan et al., 2006; McMain et al., 2009; Safer, Telch, & Agras, 2001; Telch, Agras, & Linehan, 2001; Verheul et al., 2003). DBT increases treatment retention while reducing symptoms of BPD, non-suicidal self-injury, substance abuse, binging and purging, depression, and anger (Lynch et al., 2007), earning recommendations from the Substance Abuse and Mental Health Services Administration (SAMHSA), American Psychiatric Association (APA), and U.K. National Institute of Health and Clinical Excellence (APA, 2001; National Institute for Health and Clinical Excellence, 2009; SAMHSA, 2010). DBT also results in substantial cost savings by reducing emergency care utilization (Amner, 2012; Priebe et al., 2012).

In order to maximize its impact in the real world, DBT must be offered in more organizations. Two studies in public health systems have examined barriers to DBT implementation through qualitative interviews with clinicians (Carmel, Rose, & Fruzzetti, 2013) and administrators (Herschell et al., 2009). Both identified organizational barriers to DBT implementation, including staffing problems (e.g., staff turnover), difficulties with program development (e.g., identifying appropriate clients), lack of administrative or organizational support, resource concerns (e.g., reimbursement issues), and how DBT fits with existing practices and opinions.

Swales, Taylor, and Hibbs (2012) examined the sustainability of DBT programs implemented in the UK following a large roll out. They found that programs attempting to provide DBT were vulnerable to drift from the model and termination in the second year. Just 57% of active DBT programs offered each primary mode of DBT, and organizational support was the most commonly reported challenge. Solutions included assessment of the match between site needs and DBT, careful selection of staff and patients, training, and monitoring of programs (Swales, Taylor, & Hibbs, 2012).

While these studies highlight the importance of organizational support to the implementation of DBT, more detail is needed to understand the nature of that support. To provide such detail, an exploration of organizational variables and DBT implementation guided by a framework such as the CFIR is warranted. Specifically, inner setting CFIR constructs provide a checklist for systematically exploring relationships between organizational variables and DBT implementation. However, terminological disagreements and instrumentation deficiencies among implementation researchers remain (Martinez et al., 2014; Proctor et al., 2013). Even though it does not resolve all conceptual disputes or measurement disparities among implementation researchers, utilizing CFIR inner setting constructs as a checklist enables systematic exploration.

Inner setting refers to the structural, political, and cultural context within an organization where an intervention resides (Damschroder et al., 2009). Five subdomains exist within the CFIR's inner setting. Structural characteristics include the social architecture, age, maturity, and size of an organization. Networks and communication refers to the nature and quality of an organization's webs of social network as well as its formal and informal communications. An organization's culture refers to its norms, values, and basic assumptions; implementation climate involves an organization's shared receptivity by individuals to an intervention. Readiness for implementation refers to the tangible and immediate indicators of an organization's commitment to an intervention (Damschroder et al., 2009).

Researchers have examined the relationship between constructs within inner setting and the implementation of EBPs other than DBT. Glisson and Green (2011) demonstrated the necessity of having a low-stress, engaged organizational climate within mental health settings. Beidas et al. (2012) found that supervision and consultation improved therapist behavior after training. Torrey et al. (2012) concluded that effective and engaged leadership is vital for implementing mental health interventions. Damschroder and Lowery (2013) conducted a qualitative exploration of CFIR constructs impacting the implementation of a weight loss program. These results revealed the importance of interpersonal phenomena within organizations, but their generalization to DBT implementation remains unknown. By identifying inner setting variables that foster DBT implementation specifically, organizations can strategically support the treatment.

Therefore, this study aims to build on the work of previous DBT implementation research by exploring the relationship between inner setting constructs and DBT implementation. The question for research is: what inner setting variables are related to the quantity of DBT elements implemented by DBT programs in real world settings? For a complete list of CFIR variables analyzed, see Table 1. Findings were organized by the following inner setting subdomains: (1) structural characteristics, (2) networks and communication, (3) culture and implementation climate, and (4) readiness for implementation.

Table 1.

Inner Setting Variables and Means of Measurement

Variable Means of Measurement
Structural Characteristics
Organizational Affiliation Is your DBT program a stand-alone entity (such as a private practice), or are you affiliated with a larger organization (such as a hospital or parent corporation)?
Age of Team How many years have at least two members of your current DBT team been practicing together as members of your team?
Size of Team How many individuals are members of your current DBT team?
Size of Program How many individuals are directly involved with your DBT program (including team-members and non-team members, such as support staff)?
Level of Care PETQ 50, 53, 58
Networks and Communication
Cohesion ORC “Cohesion” Subscale
Communication ORC “Communication” Subscale
Culture and Climate
Team Climate for Innovation TCI-14
Readiness for Implementation
Provides Ongoing Supervision PETQ “Provides Ongoing Supervision” Subscale
Educational Background – % of Team at the Bachelors Level How many individuals on your DBT team have less than a Masters degree? (Answer divided by Size of Team)
Educational Background – % of Team at the Masters Level How many individuals on your DBT team have a Masters degree, but not a Doctoral degree? (Answer divided by Size of Team)
Educational Background – % of Team at the Doctoral Level How many individuals on your DBT team have a Doctoral degree or more? (Answer divided by Size of Team.)
Reimbursement for Individual Describe your reimbursement for individual therapy.
Reimbursement for Group Skills Describe your reimbursement for group skills training.
Reimbursement for Between Session Coaching Describe your reimbursement for between session coaching.
Reimbursement for Clinical Team Meetings Describe your reimbursement for clinical team meetings.
Office Space Does your DBT program have adequate office space to carry out all modes of DBT (individual therapy, group skills training, team meetings, and between session consultation)?
Open in a new tab

Methods

The current study employed a sequential mixed methods design. Phase 1 involved a quantitative survey (n = 79) measuring inner setting variables and the quantity of DBT elements implemented. Phase 2 involved qualitative interviews with a subset of participants from Phase 1 (n = 20). Open-ended Phase 2 questions explored the relationship between inner setting constructs and DBT implementation.

Participants

Participants were a purposive sample of English-speaking mental health providers who completed DBT intensive training through Behavioral Tech, LLC more than one year prior to study participation. Seventy-nine providers participated in the quantitative survey and of those, twenty (25%) also participated in qualitative interviews. Providers who completed intensive training from other organizations were excluded from the sample. To reduce the time burden of the quantitative survey, demographic information was not collected.

Measures

Networks and Communication

The Organizational Readiness for Change (ORC; Lehman et al., 2002) is a self-report measure designed to assess organizational constructs that influence implementation. The ORC consists of 115 items and respondents rate each item on a 5-point Likert scale (1 = disagree strongly; 5 = strongly agree), with some items reverse scored. The scale is organized into 18 subscales. Two of the subscales, “Cohesion” and “Communication,” were selected for use in the current study for their determined alignment with the CFIR.

The ORC Cohesion Subscale is a 6-item scale focusing on work-group trust and cooperation (Lehman et al., 2002). Items include “staff members at your program work together as a team” and “staff members at your program get along very well.” Its alpha coefficient at the director, staff, and program level is reported at 0.83, 0.84, and 0.92 respectively (Lehman et al., 2002). This subscale was used as a proxy for the CFIR construct of networks, which refers to the nature and quality of social networks (Damschroder et al., 2009).

The ORC Communication Subscale is a 5-item scale focusing on the adequacy of information networks within an organization (Lehman et al., 2002). Items include “your program staff is always kept well informed” and “the formal and informal communication channels in your program work very well.” Its alpha coefficient at the director, staff, and program level is reported at 0.67, 0.80, and 0.82 respectively (Lehman et al., 2002). This subscale was used as a proxy for the CFIR construct of communication, which refers to the nature and quality of formal and informal communications within an organization (Damschroder et al., 2009).

Culture and Implementation Climate

The Team Climate Inventory-short version (TCI-14; Kivimaki & Elovainio, 1999) is a self-report scale designed to measure facets of workgroup climate supporting innovation. Each of its fourteen items are rated on a 5-point Likert scale (1 = disagree strongly, 5 = strongly agree) and tallied for a total score. Items include “team members feel understood and accepted by each other” and “people in the team cooperate in order to help develop and apply new ideas.” When tested with a Finnish sample, the TCI-14 had an alpha coefficient of 0.94 (Kivimaki & Elovainio, 1999). Loo and Loewen (2002) tested an English version of the TCI-14 on a Canadian sample and found high alpha coefficients at two administrations (0.90 and 0.93). The English version was used as a proxy for the CFIR constructs of culture and implementation climate.

Other Inner Setting Variables

Researcher-developed questions. Twelve close-ended researcher-developed questions measured additional CFIR inner setting constructs. All items were developed and refined using cognitive interviewing techniques specified by Fowler (1995). Items included questions about structure of the program, team variables, and financial concerns.

DBT Implementation

The Program Elements of Treatment Questionnaire (PETQ; Schmidt III, Ivanoff, Korslund, & Linehan, 2008) is a self-report questionnaire that assesses the extent of DBT program elements implemented and is designed to be completed by the team leader. It consists of 85 items. Items 1-60 make up five categories of program elements required by DBT, including elements specific to DBT, consultation team, client treatment and support, tracking outcomes, and documentation. The rest of the items relate to organizational characteristics supporting DBT elements (e.g. training). Response options for implementation of different elements of DBT are “yes” that it has been implemented, “some” of the element has been implemented, “planned” for elements that are planned but not yet implemented, and “no” for those not implemented.

The PETQ was developed as an assessment tool for DBT program accreditation and has been distributed as a self-assessment tool for DBT programs. As such, no formal scoring procedures exist and no psychometric data are available. In the absence of alternatives, the first 60 items were used to measure DBT implementation outcomes, and the PETQ score was calculated as a percentage of “yes” items.

Items 50, 53, and 58 asked if a program was categorized as “outpatient,” “milieu treatment/ day program,” or “inpatient/ residential.” These items were removed from the tally of implementation outcomes, and instead used to categorize the level of care of each program. Level of care was categorized as a structural characteristics variable. The percentage of “yes” PETQ supervision items (items 81-82) were considered a readiness for implementation variable. According to the CFIR, readiness for implementation includes leadership engagement, training, and access to knowledge, and the nature of supervision was determined congruent with these constructs.

For a summary of all inner setting variables and their measurement, see Table 1.

The qualitative interview guide was developed from the CFIR's inner setting constructs. Interviews began with broad, open-ended questions about the participant's organization and DBT implementation, such as “How does your organization help you and your colleagues provide DBT?” The list of CFIR inner setting constructs was used as a checklist during each interview to ensure each was discussed. Interview guide questions were used at the end of each interview to inquire about any construct not already discussed. Examples include, “How do you think the size of your organization has impacted your ability to do DBT?”

Procedure

All procedures were approved by the University of Pennsylvania Institutional Review Board. Data were collected from August 2012 through January 2013.

Recruitment

Recruitment emails for the online survey were sent via three routes. First, a recruitment email was sent to the international DBT listserv on three occasions. This listserv is available to DBT providers who have completed intensive training in DBT and includes approximately 1570 recipients. Second, a recruitment email was sent to the Association of Behavioral and Cognitive Therapies (ABCT) listserv. This listserv is open to any member of ABCT and currently has approximately 4700 members. Third, individual emails were sent to the approximately 250 DBT programs with email addresses listed on the Behavioral Tech, LLC website. In addition, a handout was distributed at the 2012 International Study for the Improvement and Teaching of DBT (ISITDBT) conference. This conference is for DBT providers and researchers and generally has approximately 150-200 attendees. Phase 2 participants were recruited via a question on the online survey. Twenty-eight individuals indicated a willingness to participate in Phase 2, and a follow up email was sent to each. Twenty individuals responded to the follow up email and completed the interview.

All Phase 1 online survey participants were entered into a raffle where three individuals won $150, $75, and $25. All 20 Phase 2 participants received a $10 Starbucks gift card.

Phase 1

Participants completed the online survey that included informed consent, the PETQ, ORC subscales, TCI-14, and researcher-developed questions. The entire online survey took approximately 30 minutes to complete.

Phase 2

The qualitative interviews were conducted online via Skype and were recorded with consent. Interviews lasted approximately one hour.

Analysis

A sequential analytic strategy involved quantitative analysis followed by qualitative analysis (i.e., QUAN → qual). The data were triangulated to confirm relationships between inner setting variables and DBT implementation. Complementarity was assessed between the quantitative and qualitative findings (Palinkas et al., 2011).

Quantitative inner setting variables and PETQ scores were compared in SPSS using bivariate statistical analyses. Nominal inner setting variables were compared to PETQ scores with t-tests or ANOVA procedures. Ordinal variables (e.g., TCI-14) and ratio variables (e.g., number of team members) were compared to PETQ scores with regression analyses.

Qualitative interviews were transcribed and coded by the first author in nVivo. To check reliability, a second rater coded one of the 20 interviews and discussed findings with the first author. There was 100% agreement on themes with minor elaborations. Following Creswell's (2007) grounded theory guidelines, open codes were further analyzed through axial coding processes. In total, 2,399 open codes were organized into 10 DBT categories, 8 structural characteristics categories, 13 networks and communications categories, 19 culture and implementation climate categories, 23 readiness for implementation categories, and 3 categories peripheral to the inquiry.

Results

DBT Implementation Outcomes

The mean PETQ score of survey respondents was 0.70; meaning on average, respondents selected “yes” on 70% of the items. The PETQ had good internal reliability (α = 0.87). Percentages of respondents indicated their program's utilization of each primary mode of DBT as follows: (1) individual therapy – 96%, (2) group skills training – 99%, (3) skills coaching/ telephone consultation – 87%, (4) therapist consultation team – 97%. The following optional modes of DBT were utilized by the following percentage of respondents: (5) individual skills training – 61%, (6) DBT pharmacotherapy – 27%, (7) DBT case management – 32%, and (8) DBT support/ group process therapy – 33%.

Structural Characteristics

The majority of providers (63%) reported working with a DBT program nested within an organization and the remainder (37%) reported working in stand-alone DBT programs. Providers reported working in treatment settings with varying levels of care, including outpatient (90%), inpatient/residential (8%), and milieu/day treatment (2%). Additional structural characteristics are reported in Table 2.

Table 2.

Means and Standard Deviations for Structural Characteristics of Provider's DBT Programs

Characteristic M (SD)
Age of Team 7.35 years (6.51)
Size of Team 7.94 people (4.48)
Size of Program 19.61 people (46.64)
Open in a new tab

Respondents representing stand-alone DBT programs had significantly higher PETQ scores than those representing teams nested within organizations, t(75) = 2.13, p < .05. The main effect of a program's level of care on PETQ scores was not significant, F(2, 78), p = n.s. The age of the team (r = 0.10, p = n.s.) was not significantly correlated with PETQ scores. The size of the team was significantly correlated with PETQ scores, r = 0.28, p < 0.05. The size of the program was not significantly correlated with PETQ scores, r = -0.07, p = n.s.

Qualitative findings concerning the influence of structural characteristics on DBT implementation were largely unclear and inconsistent. Participants had experiences in a wide range of settings, yet few statements were made regarding the impact of structural characteristics on the implementation of DBT elements. Disagreement existed within the few statements made on structural characteristics and implementation. One participant stated, “I don’t know that the structure matters.”

The mixed method findings on structural characteristics and DBT implementation are listed in Table 3.

Table 3.

Complementarity of Findings - Structural Characteristics and DBT Implementation

Method Quantitative Qualitative
Question Do stand-alone DBT programs implement more DBT elements than those nested within an organization? If a DBT program is nested within an organization, does that impact implementation?
Answer Yes: Stand-alone programs had higher implementation scores than those nested in an organization with moderate significance. No: None of the participants indicated that nesting within an organization impacted implementation. However, affiliation with a university was thought to positively influence implementation.
Question Do DBT programs in outpatient, inpatient/ residential, or day/ milieu settings differ in their DBT implementation? Does a program's level of care influence its ability to implement DBT?
Answer Unclear: DBT implementation did not significantly differ among levels of care. However, only 8 individuals were not in outpatient settings, so sample sizes were small. No: Participants did not indicate that level of care influenced DBT implementation.
Question Is the age of a DBT consult team correlated with DBT implementation? Does a team's age impact DBT implementation?
Answer No: The years since a team was formed was not correlated with implementation. No: Participants did not indicate that the age of a team impacted implementation.
Question Is team size correlated with DBT implementation? Does team size impact DBT implementation?
Answer Yes: Team size was positively correlated with DBT implementation with moderate significance. Unclear: Two participants with team-sizes of three individuals desired a larger team. Others indicated that successful implementation drew members to them, suggesting that team size results from implementation rather than causing it.
Question Is the size of an organization correlated with DBT implementation? Does the size of an organization impact implementation?
Answer No: The number of individuals in a program's organization was not correlated with DBT implementation. Unclear: Some participants indicated that larger organization had more moving parts and were harder to implement in. Others indicated that the size of an organization does not impact DBT implementation.
Open in a new tab

Networks and Communication

With regard to cohesion and communication, participants scored an average of 25.75 (SD = 4.13) on the ORC Cohesion subscale and an average of 18.66 (SD = 4.49) on the ORC Communication subscale. Both the ORC Cohesion (α = 0.88) and ORC Communication (α = 0.86) subscales had good internal reliability.

Scores on the ORC Cohesion and Communication subscales were both significantly positively correlated with PETQ scores, r = 0.43, p < 0.01 and r = 0.49, p < 0.01 respectively. As shown in Table 4, qualitative data complemented and elaborated upon these quantitative findings, explicitly linking cohesion and communication to DBT implementation.

Table 4.

Complementarity of Findings – Cohesion and Communication and DBT Implementation

Method Quantitative Qualitative
Question Does team cohesion correlate with the amount of DBT implementation? Does team cohesion impact DBT implementation?
Answer Yes: Cohesion subscale scores were positively correlated with the amount of implemented DBT elements with strong significance. Yes: Cohesion, working as a team, liking team members, and being vulnerable with each other were identified as important. Weak networks were also identified as a threat to DBT programs. Themes clearly linked cohesion to implementation.
Question Does team communication correlate with the amount of DBT implementation? Does communication impact DBT implementation?
Answer Yes: Communication subscale scores were positively correlated with the amount of implemented DBT elements with strong significance. Yes: The importance of open communication and communication style were identified as themes. Each was suggested as causes of DBT implementation. Communication beyond the team to administration, ancillary staff, and the community was also identified as important for an effective DBT program.
Open in a new tab

Culture and Climate

With regard to culture and climate, participants had an average score of 59.85 (SD = 8.51) on the TCI-14. The TCI-14 had excellent internal reliability (α = 0.94). The TCI-14 and the PETQ were significantly positively correlated, r(72) = 0.58, p < 0.01. As shown in Table 5, qualitative data further elaborated these quantitative findings and linked culture and implementation climate to DBT implementation.

Table 5.

Complementarity of Findings – Culture and Implementation Climate and DBT Implementation

Method Quantitative Qualitative
Question Does team climate for innovation correlate with the amount of DBT implementation? Do culture and implementation climate impact DBT implementation?
Answer Yes: Team climate for innovation was positively correlated with the amount of implemented DBT elements with strong significance. Yes: Many participants spoke of the importance of sharing goals, vision, and a collective energy for having a DBT program. Many also spoke of developing a DBT-specific culture in their setting, including speaking in a DBT-language. DBT skills, such as interpersonal effectiveness, were also seen as important implementation strategies.
Open in a new tab

Readiness for Implementation

In regards to DBT supervision, the average score of the PETQ supervision subscale was 0.69 (SD = 0.43). It had good internal reliability (α = 0.81). Regarding the academic background of teams, participants indicated that 11% of their team's members had less than a Masters degree, 68% had a Masters degree, and 9% had a Doctoral degree. Reimbursement for the primary modes of DBT is reported in Table 6. Finally, the majority of respondents (77%) reported having adequate office space.

Table 6.

Percentage of participants receiving reimbursement from different sources for each mode of DBT.

Self-Pay Private Insurance Public Insurance None
Reimbursement for individual therapy 18% 22% 55% 5%
Reimbursement for group skills training 22% 18% 54% 6%
Reimbursement for between session coaching 0 1% 35% 64%
Reimbursement for consultation team 0 0 29% 71%
Open in a new tab

The PETQ supervision subscale was positively correlated with the overall PETQ score (r = 0.61, p < 0.001). Percentage of team members with less than a Masters degree was not significantly correlated with PETQ score, r = -0.08, p = n.s. Percentage of team members with a Masters degree but not a Doctoral degree was also not significantly correlated with PETQ score, r = -0.03, p = n.s. The percentage of team members with a Doctoral degree was negatively correlated with PETQ score, r = -0.53, p < 0.05.

The main effect of reimbursement on PETQ score was not significant for (a) individual therapy F(3, 74) = 0.49, p = n.s. or (b) group skills training F(3, 74) = 1.06, p = n.s. The differences in PETQ scores of those who received reimbursement were not significantly different from those who did not receive reimbursement for (a) between session coaching t(29) = 0.89, p = n.s. or (b) consultation team meetings t(42) = -0.58, p = n.s. Those who reported having adequate office space had significantly higher PETQ scores than those who did not, t(76) = 2.32, p<.05.

Qualitative support for the importance of supervision was clear and strong. It was less clear for the importance of office space, and it was non-existent for the impact of team members with doctoral degrees. Qualitative data that further elaborated quantitative findings regarding readiness for implementation variables and DBT implementation, including supervision and office space are presented in Table 7.

Table 7.

Complementarity of Findings – Readiness for Implementation and DBT Implementation

Method Quantitative Qualitative
Question Does having DBT supervision correlate with the amount of DBT implementation? Does DBT supervision impact DBT implementation?
Answer Yes: DBT supervision subscale scores positively correlated with the amount of implemented DBT elements with strong significance. Yes: Many participants expressed the impact of DBT supervision on implementation. Some mentioned the use of recording and reviewing therapy session as an important tool for supervision and implementation.
Question Is the percentage of team members with (a) less than a Masters degree, (b) a Masters degree, and (c) a Doctoral degree correlated with the amount of DBT implementation? Does the educational background of individuals in a DBT program impact implementation?
Answer Yes and No: The percentage of team members with less than a Masters degree was not correlated with DBT implementation, nor was the percentage of team members with a Masters degree. However, the percentage of team members with a Doctoral degree was negatively correlated with DBT implementation with moderate significance. Unclear: Some stated that having individuals with strong research knowledge was important to DBT implementation, and some thought this could be attained from some degrees more than others. Others stated that educational background does not impact implementation. Many participants highlighted the importance of knowing DBT, however.
Question Do programs with funding for each standard DBT mode implement better than those without funding for each mode? Does funding for each mode impact DBT implementation?
Answer Unclear: The amount of DBT elements implemented did not differ between those that did and did not receive funding for each mode. However, some conditions had too few individuals to analyze properly, such as just five respondents representing programs with no funding for skills groups. Unclear: All participants spoke about the importance of receiving funding for the sustainability of their program. However, many participants also stated that funding did not impact the quantity or quality of DBT elements provided. One participant mentioned acceptance of funding received, then working to change it.
Question Do programs with adequate office space have more DBT implementation than those without? Does adequate office space impact use of DBT?
Answer Yes: Respondents endorsing adequate office space implemented more DBT elements than those without adequate office space with moderate significance. Unclear: Some participants claimed that office space is important for implementing DBT (e.g., space for groups and team meetings). However, most statements made about office space alluded to its ability to foster other important aspects in programs (e.g., communication with team members, a DBT culture, etc.). Others spoke about creatively implementing without adequate office space.
Open in a new tab

Discussion

Major findings of this research are as follows: team cohesion, team communication, team climate, and supervision were significantly correlated with the quantity of DBT elements implemented. Qualitative themes strengthened and elaborated these findings while clearly implicating each variable as a possible facilitator for DBT implementation. All four inner setting variables can be described as interpersonal characteristics within organizations. Therefore, the results of this systematic exploration of inner setting CFIR phenomena most clearly link these aspects of human behavior within organizations as facilitating DBT implementation. These findings support previous implementation research on inner setting variables and interventions other than DBT.

While interpersonal phenomena within organizations appear paramount to implementation, other findings from the current research are noteworthy. Four additional quantitative inner setting variables were moderately correlated to the quantity of implemented DBT elements (the four interpersonal variables noted above were strongly correlated). However, each of these four additional variables lacked qualitative support. Programs nested within an organization implemented fewer DBT elements than stand-alone programs, but qualitative support for this finding was mixed. Team size was positively correlated with DBT implementation, but qualitative themes suggested that team size may have resulted from implementation rather than causing it. The percentage of team members with a doctoral degree was negatively correlated to DBT implementation, and no qualitative statements supported this finding. Programs with adequate office space implemented more elements than those without, but qualitative themes suggested that office space fosters other inner setting variables such as communication and culture.

Additional inner setting barriers and facilitators were suggested by qualitative themes, but were unconfirmed by quantitative analyses (e.g., affiliation with a university). Because these findings have either quantitative or qualitative support and not both, interpretation of each individual finding requires increased caution. Despite the weakness of each independent finding, the quantity of additional findings is noteworthy. Furthermore, some of the inner setting variables unconfirmed in this study may also be significant. For example, Swales et al. (2012) found that the age of a program relates to whether a DBT program is active or inactive, while the present study did not find a relationship between the age of a team and implementation. However, the former measured whether a DBT program was active or inactive while this study analyzed the quantity of elements implemented by active programs. Therefore, while interpersonal inner setting constructs appear particularly important for DBT implementation, they are likely insufficient on their own.

Several limitations of this study are notable, including several regarding measurement. First, implementation outcomes were measured via self-report to determine the quantity of DBT elements utilized. Accuracy of the number of elements utilized cannot be verified, nor can adherence or fidelity to these elements be analyzed. These concerns are intensified by the PETQ's lack of previous psychometric properties. Additionally, given the lack of scoring procedures for this measure, all items in the measure were given equal weight (e.g., implementing a skills group was counted the same as tracking outcomes). Future work should address this inequality in the measure for more refined measurement of implementation. Second, inner setting variables were sampled at the individual level, and individual reports may contain inaccuracies. Third, because data were collected following implementation, quantitative findings cannot establish if inner setting variables existed prior to implementation or resulted from it.

Other limitations concern sampling. Even though the sample was purposive, the research used a small sample of self-selected participants who received training from a specific company. The results may not generalize across DBT programs, especially to those programs created by therapists trained by other sources, including self-training, companies other than Behavioral Tech, LLC, or university programs.

Future prospective research should test hypotheses derived from this research. Implementation efforts targeting team cohesion, team communication, team climate for innovation, and supervision should be monitored and tested longitudinally to verify if these variables do in fact impact DBT implementation outcomes. Additionally, understanding relationships between inner setting variables presents another future line of inquiry. For example, does adequate office space foster team communication? Alternate statistical analyses to explore the relationships between the findings of this research may identify potential mediators or moderators to key determinants of DBT implementation.

Despite the limitations and necessity of future inquiry, these results have potentially profound implications. Considering the complexity of DBT as a treatment and the many elements to be implemented, identifying key facilitators for successful DBT implementation can enhance efforts to offer the treatment. Specifically, this study attempted to more precisely identify the organizational support required for such efforts by systematically exploring inner setting variables. In the current economic climate, results of this study offer hope that DBT implementation is fostered by interpersonal phenomena within organizations, which differ from the costly resources most commonly considered as organizational support (e.g., funding). Efforts to support team cohesion, team communication, team climate for innovation, and supervision may result in the implementation of DBT in more community settings, whereby increasing access to care for high-risk, difficult to treat individuals in need.

These results support the conclusion that the quantity of implemented DBT elements increases with improvements in human and interpersonal processes within settings - including team cohesion, team communication, team climate, and supervision. Proponents seeking to implement DBT should pay special attention to these four variables within their settings.

Acknowledgement

The authors thank Dr. Kelly Koerner, Dr. Tony Dubose, Dr. Erin Miga, and Dr. Jeffrey Applegate for advising this study's design. The authors also thank Dr. Thomas Lynch for strategizing qualitative interviewing, Dr. Minseop Kim for reviewing all statistical analyses, and Dr. Allison Gonzalez for coding a qualitative interview for triangulation. Finally, the first author thanks Dr. Shari Manning and Dr. Katherine Comtois for manuscript preparation advice.

This research is based on results from the doctoral dissertation of Matthew Ditty. Funding for this research project was supported by NIMH grants to Beidas (MH099179). Additionally, the preparation of this article was supported in part by the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and Quality Enhancement Research Initiative (QUERI), Department of Veterans Affairs Contract, Veterans Health Administration, Office of Research & Development, Health Services Research & Development Service. Dr. Beidas is an IRI fellow.

References

  1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:4–23. doi: 10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Palinkas LA. Implementation of evidence based practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:411–419. doi: 10.1007/s10488-007-0121-3. [DOI] [PubMed] [Google Scholar]
  3. Amner K. The effect of DBT provision in reducing the cost of adults displaying the symptoms of BPD. British Journal of Psychotherapy. 2012;28:336–352. [Google Scholar]
  4. American Psychological Association Practice guideline for the treatment of patients with borderline personality disorder. American Journal of Psychiatry. 2001;158:261–276. [PubMed] [Google Scholar]
  5. Beidas RS, Edmunds JE, Marcus SC, Kendall PC. Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services. 2012;63:660–665. doi: 10.1176/appi.ps.201100401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Beidas RS, Koerner K, Weingardt KR, Kendall PC. Training research: Practical recommendations for maximum impact. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:223–237. doi: 10.1007/s10488-011-0338-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bohus M, Haaf B, Simms T, Limberger MF, Schmahl C, Unckel C, Linehan MM. Effectiveness of inpatient dialectical behavioral therapy for borderline personality disorder: a controlled trial. Behaviour Research and Therapy. 2004;42:487–499. doi: 10.1016/S0005-7967(03)00174-8. [DOI] [PubMed] [Google Scholar]
  8. Carmel A, Rose ML, Fruzzetti AE. Barriers and solutions to implementing dialectical behavior therapy in a public behavioral health system. Administration and Policy in Mental Health and Mental Health Services Research. 2013:1–7. doi: 10.1007/s10488-013-0504-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Carter GL, Willcox CH, Lewin TJ, Conrad AM, Bendit N. Hunter DBT project: Randomized controlled trial of dialectical behaviour therapy in women with borderline personality disorder. Australian and New Zealand Journal of Psychiatry. 2010;44:162–173. doi: 10.3109/00048670903393621. [DOI] [PubMed] [Google Scholar]
  10. Creswell JW. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 2nd Ed. Sage Press; Thousand Oaks, CA: 2007. [Google Scholar]
  11. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4:50. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Science. 2013;8:51. doi: 10.1186/1748-5908-8-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fixsen DL, Naoon SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, the National Implementation Research Network; Tampa, FL: 2005. [Google Scholar]
  14. Fixsen D, Blase K, Naoom S, Wallace F. Core implementation components. Research on Social Work Practice. 2009;19:531. [Google Scholar]
  15. Fowler FJ. Improving Survey Questions: Design and Evaluation. Vol. 38. Sage Publications; Thousand Oaks, CA: 1995. [Google Scholar]
  16. Glisson C, Green P. Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect. 2011;35:582–591. doi: 10.1016/j.chiabu.2011.04.009. [DOI] [PubMed] [Google Scholar]
  17. Herschell AD, Kogan JN, Celedonia KL, Gavin JG, Stein BD. Understanding community mental health administrators’ perspectives on dialectical behavior therapy implementation. Psychiatric Services. 2009;60:989–992. doi: 10.1176/appi.ps.60.7.989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hogan MF. New Freedom Commission report: The president's New Freedom Commission: recommendations to transform mental health care in America. Psychiatric Services. 2003;54:1467–1474. doi: 10.1176/appi.ps.54.11.1467. [DOI] [PubMed] [Google Scholar]
  19. Kivimaki M, Elovainio M. A short version of the Team Climate Inventory: Development and psychometric properties. Journal of Occupational and Organizational Psychology. 1999;72:241–246. [Google Scholar]
  20. Koons CR, Robins CJ, Lindsey Tweed J, Lynch TR, Gonzalez AM, Morse JQ, Bastian LA. Efficacy of dialectical behavior therapy in women veterans with borderline personality disorder. Behavior Therapy. 2001;32:371–390. [Google Scholar]
  21. Landes SJ, Linehan MM. Dissemination of dialectical behavior therapy: An intensive training model. In: Barlow DH, McHugh RK, editors. Dissemination and Implementation of Evidence-based Psychological Interventions. Oxford University Press; New York: 2012. pp. 187–208. [Google Scholar]
  22. Lehman WE, Greener JM, Simpson DD. Assessing organizational readiness for change. Journal of Substance Abuse Treatment. 2002;22:197–209. doi: 10.1016/s0740-5472(02)00233-7. [DOI] [PubMed] [Google Scholar]
  23. Lewis CC, Borntrager C, Martinez R, Fizur P, Comtois K. A comprehensive review of dissemination and implementation science instruments.. Symposium at the Convention of The Association for Behavioral and Cognitive Therapies; Toronto, ONT, Canada. November, 2011. [Google Scholar]
  24. Linehan MM. Cognitive-Behavioral Treatment of Borderline Personality Disorder. Guilford; New York, NY: 1993. [Google Scholar]
  25. Linehan MM, Armstrong HE, Suarez A, Allmon D, Heard HL. Cognitive-behavioral treatment of chronically parasuicidal borderline patients. Archives of General Psychiatry. 1991;48:1060–1064. doi: 10.1001/archpsyc.1991.01810360024003. [DOI] [PubMed] [Google Scholar]
  26. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, Lindenboim N. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Archives of General Psychiatry. 2006;63:757–766. doi: 10.1001/archpsyc.63.7.757. [DOI] [PubMed] [Google Scholar]
  27. Linehan MM, Dimeff LA, Reynolds SK, Comtois KA, Welch SS, Heagerty P, Kivlahan DR. Dialectical behavior therapy versus comprehensive validation therapy plus 12-step for the treatment of opioid dependent women meeting criteria for borderline personality disorder. Drug and Alcohol Dependence. 2002;67:13–26. doi: 10.1016/s0376-8716(02)00011-x. [DOI] [PubMed] [Google Scholar]
  28. Linehan MM, Schmidt H, Dimeff LA, Craft JC, Kanter J, Comtois KA. Dialectical behavior therapy for patients with borderline personality disorder and drug dependence. The American Journal on Addictions. 1999;8:279–292. doi: 10.1080/105504999305686. [DOI] [PubMed] [Google Scholar]
  29. Lynch TR, Morse JQ, Mendelson T, Robins CJ. Dialectical behavior therapy for depressed older adults: A randomized pilot study. The American Journal of Geriatric Psychiatry. 2003;11:33–45. [PubMed] [Google Scholar]
  30. Lynch TR, Trost WT, Salsman N, Linehan MM. Dialectical behavior therapy for borderline personality disorder. Annual Review of Clinical Psychology. 2007;3:181–205. doi: 10.1146/annurev.clinpsy.2.022305.095229. [DOI] [PubMed] [Google Scholar]
  31. Loo R, Loewen P. A confirmatory factor-analytic and psychometric examination of the team climate inventory: full and short versions. Small Group Research. 2002;33:254–265. [Google Scholar]
  32. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implementation Science. 2014;9:118. doi: 10.1186/s13012-014-0118-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist. 2010;65:73–84. doi: 10.1037/a0018121. [DOI] [PubMed] [Google Scholar]
  34. McMain S, Links P, Gnam W, Guimond T, Cardish R, Korman L, Streiner D. A randomized trial of dialectical behavior therapy versus general psychiatric management for borderline personality disorder. American Journal of Psychiatry. 2009;166:1365–1374. doi: 10.1176/appi.ajp.2009.09010039. [DOI] [PubMed] [Google Scholar]
  35. National Institute for Health and Clinical Excellence January). Borderline personality disorder: Treatment and management. 2009 Retrieved from http://www.nice.org.uk/nicemedia/pdf/CG78NICEGuideline.pdf. [Google Scholar]
  36. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:44–53. doi: 10.1007/s10488-010-0314-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Priebe S, Bhatti N, Barnicot K, Bremner S, Gaglia A, Katsakou C, Zinkler M. Effectiveness and cost-effectiveness of dialectical behaviour therapy for self-harming patients with personality disorder: A pragmatic randomised controlled trial. Psychotherapy and Psychosomatics. 2012;81:356–365. doi: 10.1159/000338897. [DOI] [PubMed] [Google Scholar]
  38. Proctor EK, Powell BJ, McMillen J. Implementation strategies: Recommendations for specifying and reporting. Implementation Science. 2013;8:139. doi: 10.1186/1748-5908-8-139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Safer DL, Telch CF, Agras WS. Dialectical behavior therapy for bulimia nervosa. American Journal of Psychiatry. 2001;158:632–634. doi: 10.1176/appi.ajp.158.4.632. [DOI] [PubMed] [Google Scholar]
  40. Schmidt III H, Ivanoff A, Korslund K, Linehan MM. Program Elements of Treatment Questionnaire. Behavioral Research & Therapy Clinics; Seattle, WA: 2008. [Google Scholar]
  41. Substance Abuse and Mental Health Service Administration. (SAMHSA Report to congress on borderline personality disorder. 2010 http://store.samhsa.gov/shin/content//SMA11-4644/SMA11-4644.pdf.
  42. Swales MA, Taylor B, Hibbs RA. Implementing dialectical behaviour therapy: Programme survival in routine healthcare settings. Journal of Mental Health. 2012;21:548–555. doi: 10.3109/09638237.2012.689435. [DOI] [PubMed] [Google Scholar]
  43. Telch CF, Agras WS, Linehan MM. Dialectical behavior therapy for binge eating disorder. Journal of Consulting and Clinical Psychology. 2001;69:1061–1065. doi: 10.1037//0022-006x.69.6.1061. [DOI] [PubMed] [Google Scholar]
  44. Torrey WC, Bond GR, McHugo GJ, Swain K. Evidence-based practice implementation in community mental health settings: The relative importance of key domains of implementation activity. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39:353–364. doi: 10.1007/s10488-011-0357-9. [DOI] [PubMed] [Google Scholar]
  45. Verheul R, van den Bosch LM, Koeter MW, de Ridder MA, Stijnen T, Van Den Brink W. Dialectical behaviour therapy for women with borderline personality disorder 12-month, randomised clinical trial in The Netherlands. The British Journal of Psychiatry. 2003;182:135–140. doi: 10.1192/bjp.182.2.135. [DOI] [PubMed] [Google Scholar]
  46. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Science. 2011;6:78. doi: 10.1186/1748-5908-6-78. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES