Abstract
Objectives
Community mental health clinics are increasingly utilizing independent contractors to provide clinical services. At the same time, many organizations are participating in initiatives intended to increase implementation of evidence-based practices (EBPs). The primary aim of this study was to understand the associations of utilizing independent contractors with clinician and organizational characteristics associated with implementation of EBPs; as well as to understand the potential impact of using independent contractors on mental health services delivery from the perspective of organizational leadership.
Methods
Quantitative data were collected from 130 therapists in 23 organizations; qualitative data were collected from executive administrators in 9 of the 16 organizations participating in EBP initiatives sponsored by the City of Philadelphia. Regression with random effects was used to estimate the associations between worker status (i.e., contractor, employee) and clinician attitudes towards EBPs, knowledge of EBPs, and organizational culture and climate. Qualitative inquiry was used to understand the impact of relying on independent contractors on organizational participation in EBP initiatives.
Results
Independent contractors endorsed less positive attitudes towards EBPs and scored lower on knowledge of EBPs. Interviews revealed four main themes: reasons for using independent contractors, general consequences of independent contractors, specific impact of independent contractors on participation in EBP initiatives, and suggestions for alternatives.
Conclusions
A growing number of community mental health clinics are relying on independent contractors. There may be consequences of this paradigm shift which deserve exploration.
Over the past fifteen years, there has been an increased emphasis on the crisis facing the behavioral health workforce, including lack of providers and concerns about effectiveness of services (1). An effort to address these issues was undertaken by the Annapolis Coalition on the Behavioral Health Workforce, culminating in a document outlining strategic goals (2) including broadening the concept of workforce, strengthening the workforce, and building structures to support the workforce (3). Concurrently and relatedly, the last fifteen years have also seen a growing interest in the implementation of evidence-based practices (EBPs) by community mental health clinics, largely in response to policy mandates (4). This is consistent with the recommendations made by the Annapolis Coalition to buttress the competencies of the behavioral workforce in EBPs (5).
One important matter related to the behavioral health workforce that has received little attention is the growing number of independent contractors in specialty community mental health (Manderscheid, personal communication, July 24th, 2015). Salaried employees receive benefits, and have productivity requirements, whereas independent contractors typically do not receive benefits or have productivity requirements. The organizational management literature suggests that utilizing independent contractors destabilizes the workforce because it may result in conflict between salaried employees and independent contractors (6) and increase turnover for both groups (7). In community mental health, the increasing reliance on independent contractors may have sequelae for both mental health services delivery and implementation of EBPs. Given that independent contractors are not employees per se, organizations may be less likely to invest in their professional development resulting in potentially poorer knowledge and attitudes towards EBPs. Relying on independent contractors may also affect organizational culture and climate by increasing stress within an organization. Conversely, the use of independent contractors may have a positive impact on services and/or EBP implementation because it may allow organizations to be more nimble in hiring individuals with the skillsets they deem necessary for the setting. Given that both clinician (i.e., knowledge and attitudes) and organizational characteristics (i.e., culture and climate) have been found to be predictors of implementation of EBPs (8, 9), it is important to explore the relationship between independent contractors and these variables.
The first aim of this study was to explore the associations between independent contractors and factors found to be associated with implementation of EBP (8, 9). No a priori hypotheses were specified. Using quantitative methods, we compared attitudes and knowledge of EBPs in independent contractor and salaried clinicians. Next, we examined organizational culture and climate while taking into account the ratio of independent contractors within each organization. The second aim of the study was to use qualitative methods to explore stakeholder perspectives on the impact of using independent contractors in a subset of organizations participating in EBP initiatives.
Method
Setting
The Philadelphia Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) has supported EBPs in the public mental health system through four initiatives: cognitive therapy (10), prolonged exposure, trauma-focused cognitive behavioral therapy, and dialectical behavior therapy. These initiatives include training and consultation in line with treatment developer recommendations and entail a significant allocation of resources on the part of clinicians, organizations, and the system.
Agencies
More than 100 community public mental health agencies in Philadelphia provide outpatient services to youth (Community Behavioral Health, personal communication, January 3rd, 2013). We used purposive sampling to recruit outpatient public mental health agencies which served the largest percentage of youth. Out of the total number of agencies, 29 served approximately 80% (n =23,354) of youth receiving publicly-funded mental health outpatient care, thus we targeted those agencies. Of these 29 agencies, 18 (62%) agreed to participate. Representatives of another organization participating in an EBP initiative asked to participate, resulting in a final sample of 19 agencies with 23 sites, 130 clinicians, and 36 supervisors. There were no exclusion criteria for staff. Approximately 60% (N=130) of therapists providing services in the 23 organizations participated in the study; 57% (n=74) of therapists were independent contractors.
Procedure
This research was approved by the City of Philadelphia and [masked] Institutional Review Boards. The leader of the organization was approached to solicit organizational participation. A two-hour meeting was scheduled, during which we obtained informed consent. All participants were compensated $50 and provided lunch. Quantitative data were collected in 2013.
The qualitative data were collected as part of a follow-up study where we conducted interviews with agency leadership to understand the process of participating in EBP initiatives sponsored by the City of Philadelphia (11). We used purposive sampling to specifically target agencies participating in EBP initiatives in our sample (K = 16) and completed 9 interviews. When approaching agencies, we asked them to identify leadership who would have an understanding of operations, workforce issues, and agency and system factors needed to support EBP initiatives. Data were collected in 2013–2014. Respondents received an additional $50 for participation.
Measures
Demographics were assessed using the Therapist Background Questionnaire (12). We also asked participants to report on worker status (i.e., independent contractor or salaried), hours worked weekly, and participation in DBHIDS EBP initiatives. Supervisors provided information on the number of clinicians in their unit and their worker status. From that information, we determined percentage of independent contractors. DBHIDS provided us with a list of organizations that had participated in EBP initiatives, the year they began participation, and that completion year (if applicable). From that information, we calculated the cumulative years each organization participated in DBHIDS EBP initiatives.
Dependent Variables
Clinician attitudes were assessed using the Evidence-Based Practice Attitude Scale (EBPAS; (13)), a self-report questionnaire that assesses appeal of EBP, requirements to use EBP, general openness to new practices, and divergence between EBP and usual practice. The EBPAS demonstrates good internal consistency and validity (14).
Clinician knowledge about EBP was measured using the Knowledge of Evidence-Based Services Questionnaire, a 40-item self-report instrument that measures general knowledge of EBP. Knowledge is measured on a continuum from 0 to 160, with higher scores indicative of more knowledge (KEBSQ; (15)). Psychometric data suggests temporal stability, discriminative validity, and sensitivity to training (15).
Organizational culture and climate were measured using the Organizational Social Context Measurement System (OSC; (16), a 105-item measure of the social context of mental health organizations. Organizational culture includes proficiency, rigidity, and resistance, and climate includes engagement, functionality, and stress. Proficient cultures are those in which clinicians prioritize the well-being of clients and are expected to be competent. Rigid cultures are those in which clinicians have little autonomy. Resistant cultures refer to ones in which clinicians are expected to be apathetic. Engaged climates refer to ones in which clinicians feel they can accomplish worthwhile things and remain invested. Functional climates are ones in which clinicians can get their job done effectively. Stressful climates refer to ones in which clinicians feel emotionally exhausted. Organizational culture and climate are measured with T-scores (mean of 50 and standard deviation of 10) based on a normed sample of 100 community mental health clinics (16). The OSC has strong psychometric properties (17).
Semi-structured interview
We developed a semi-structured interview guide to collect information about participants’ experiences with the four DBHIDS EBP initiatives focusing on inner (i.e., agency characteristics) and outer context (i.e., system characteristics) factors (11). We included follow-up questions specifically oriented towards understanding the relationships between utilizing independent contractors and the association with participation in EBP initiatives.
Data Analytic Plan
Quantitative analysis
Analyses were conducted using PROC MIXED and PROC REG in SAS 9.0. Missing data for predictor variables were less than 10%; series means were imputed for missing predictor variables. Dependent variable missing data were minimal (3%).
We used five mixed-effects linear regression models to evaluate the associations among clinician factors (i.e., worker status, age, hours worked per week, participation in EBP initiatives (yes/no)) and knowledge (i.e., total knowledge score as measured by the KEBSQ) and attitudes (EBPAS requirements appeal, openness, divergence). These mixed-effects models included random intercepts for organization and fixed effects for staff factors.
We also used six fixed-effects regression models in which the organization was the unit of observation to evaluate the associations among (a) the ratio of independent contractors and (b) cumulative years the organization participated in city-sponsored EBP initiatives, and organizational culture (proficiency, rigidity, and resistance) and climate (engagement, functionality and stress) as measured by the OSC. Organizational measures are included in analyses by aggregating individual responses within the organization (18, 19). On all organizational variables, both statistics were substantially above the suggested .60 level as measured by awg and/or rwg (19, 20); therefore participant responses to organizational constructs were averaged within each organization.
Qualitative
Transcripts were analyzed in an iterative process based upon modified grounded theory (21–24). Through a close reading of eight transcripts, the investigators developed a set of codes that were applied to the data. A codebook was developed that included codes emerging from the transcripts (e.g., workforce) as well as a priori codes (e.g., barriers) derived from the original research questions which pertained to barriers and facilitators of participating in EBP initiatives. A subset of 12 transcripts (20%) was coded by two investigators, and inter-rater reliability was excellent (k = .98) (25). Through an inductive process, the first author independently read through the data to examine workforce themes related to independent contractors, producing memos including examples and commentary to reach consensus regarding newly-derived, emergent themes (22, 23). The second author read through and coded 2 (20%) of the same data for consensus.
Results
Quantitative Results
Table 1 provides demographic information about the clinicians who participated in the quantitative portion of the study. Table 2 presents the results of the analyses predicting attitudes and knowledge. Table 3 presents the results of the analyses predicting organizational culture and climate.
Table 1.
Variable | N | % |
---|---|---|
Gender | ||
Male | 30 | 23 |
Female | 99 | 76 |
Transgender | 1 | 1 |
Hispanic/Latinoa | ||
Yes | 26 | 20 |
No | 97 | 75 |
Ethnicitya | ||
Asian | 6 | 5 |
Black or African American | 27 | 22 |
White | 67 | 55 |
Hispanic/Latino | 13 | 11 |
Multiracial | 5 | 4 |
Other | 5 | 4 |
Academic backgrounda | ||
Bachelor’s degree | 5 | 4 |
Master’s degree | 107 | 82 |
Doctoral degree | 12 | 9 |
Licensure statusa | ||
Yes | 32 | 25 |
No | 51 | 39 |
In process | 42 | 32 |
Age (M±SD) | 38.09±11.63 | |
Years at current organization (M±SD) | 3.35±4.65 |
Does not add up to 100% because of missing responses
Table 2.
EBPAS requirements |
EBPAS appeal |
EBPAS openness |
EBPAS divergence |
KEBSQ knowledge |
|
---|---|---|---|---|---|
Variable | Predictor Estimates | ||||
Independent contractor worker status (referent = salaried) |
.09 | −.28* | .02 | −.16 | −5.12* |
Age | −.01 | −.01* | −.01 | .01 | −.18* |
Hours worked weekly | −.02* | −.01 | .00 | −.00 | −.16* |
Participated in EBP initiative (referent = did not participate) |
.22 | .02 | .18 | −.09 | 4.94* |
EBPAS = Evidence Based Practices Attitude Scale - EBPAS subscales range from 0 (not at all) to 4 (very great extent); KEBSQ = Knowledge of Evidence Based Services Questionnaire - KEBSQ ranges from 0–160 where higher scores are indicative of more knowledge; EBP = evidence-based practice
= p <.05.
Table 3.
Variable | Proficiency | Rigidity | Resistance | Engagement | Functionality | Stress |
---|---|---|---|---|---|---|
Ratio IC | 7.54 | −2.57 | −4.75 | 1.87 | 9.40 | −11.17^ |
Cumulative years participating in EBP initiative |
1.77^ | .56 | .56 | .19 | 1.85^ | .04 |
IC = independent contractor; EBP = evidence-based practice
= p ≤.10; Each measure of organizational culture and climate is measured with T-scores with a mean of 50 and standard deviation of 10
Participation in EBP initiatives
We completed an ad hoc analysis using a logistic mixed effects model among organizations participating in EBP initiatives (K = 16), and found no association between worker status and participation in an EBP initiative. Of the therapists working in the 16 organizations participating in EBP initiatives, 41 participants participated; 43 did not.
Attitudes
Independent contractors had less favorable attitudes towards adopting EBPs. Specifically, they endorsed being less willing to adopt EBPs even if they found them appealing. Additionally, older clinicians were also less likely to adopt EBPs if they found them appealing. Clinicians who worked fewer hours per week had more open attitudes towards adopting EBPs if required.
Knowledge
We found that independent contractors, younger clinicians, clinicians who worked fewer hours per week, and clinicians not participating in EBP initiatives scored lower on knowledge of EBPs. Given that both worker status and participation in EBP initiatives were significant predictors of knowledge, we created an interaction term that combined the 2 variables (worker status*participation in EBP initiatives) to our model. The interaction term was not significant; therefore, we removed it.
Organizational culture and climate
Ratio of independent contractors and cumulative years participating in EBP initiative in the organization did not significantly predict proficiency, rigidity, resistance, engagement, functionality, or stress. A marginally significant association between cumulative years participating in EBP initiatives and proficiency (p=.06) and functionality (p=.10) was observed. A marginally significant negative association between proportion of independent contractor staff and stress was observed (p=.10).
Qualitative results
Participant characteristics
Six participants were female; three were male. Six participants were white, two participants were Hispanic/Latino, and one participant was American Indian/Alaskan Native. Mean age was 45.0±8.4. Participants came from both salaried organizations (defined as >50% of outpatient therapists were salaried; K = 4) and independent contractor organizations (defined as >50% of outpatient therapists were contractors; K = 5). Of the nine represented organizations, two had transitioned to using only independent contractors within the past year.
See Table 4 for relevant quotes. All individuals noted the financial difficulties of maintaining salaried employees in the current fiscal environment for mental health services, leading to the need for independent contractors. A minority of participants noted that independent contractors remove financial risk from the organization and places it on the therapist (i.e., they do not have to pay therapists if clients do not attend session). Participants also noted the general ramifications of independent contractors including turnover, employment instability, lack of connection to the organization, and potentially reduced quality of services.
Table 4.
Theme | N | % | Example |
---|---|---|---|
Reasons for switching to the independent contractor model | |||
Financial difficulties of the salaried model |
6 | 67 | Outpatient was so horribly underpaid so we made that transition [to independent contractors], and so now outpatient is not the huge loss leader. I ran outpatient when it was salaried and I cannot tell you the financial stress of running that because your financial goals were literally impossible, and you knew no matter what would do, you were going to be extremely under budget and looking at the faces of folks that are working so very hard. |
Contractor model transfers financial risk to the clinician |
2 | 22 | If a patient doesn’t come, the agency doesn’t pay the clinician. The organization wants to pass that fee on to a contractor and say, ‘You know, we’ll support you going, but we can’t pay for it.’ So, you’re talking about travel expenses and all that, plus the not being paid while they’re going to the training. |
Consequences of the independent contractor model (general) | |||
Turnover | 3 | 33 | [The contractor model] does lend itself to that kind of instability, kind of wondering. And you know, and I have yet to have a contractor give me proper notice. |
Time = money | 3 | 33 | Contractor therapists are paid a piece rate, which means that they see somebody for an hour, and their rate is about 25.50 an hour. And, if the person doesn’t show up, they get nothing. Understood. They know that coming in. They work full time hours, and they are making a lot of money, but in the back of their mind they are thinking about the other stuff they are doing that they are not getting paid for (e.g., note writing, team meeting). I recruited people who I can sense have a desire to learn things because I figured that would work best. But, it’s difficult, even if you have a desire to learn things, you still have to eat. |
Lack of recognition/connection to the organization |
2 | 22 | The [agency] sometimes forget about contractors in a sense that they do a lot of nice things for employees. Employees just got a raise or they are getting ready to get a raise. We have a day that we all get together and celebrate and when we have that day, contractors go home generally. You know, so they are kind of left out, but at this point, you’re looking at that being almost half the agency. People that are contractors are not connected to the agency. So if you have a mission and vision driven work, you’re going to have a hard time making that connection, They don’t go to all staff meetings, they don’t go to group meetings; they come do the work, they go. |
Uncertainty | 1 | 11 | You know, so that also limits the kind of people that I can recruit because I have to find someone who is able to maintain themselves with no insurance and be able to consider that they might not get all the money every week. |
Reduced quality of services | 2 | 22 | You lose a quality element as well. I’m not saying that independent contractors do poor work, what I’m saying is that my ability to develop and enhance and train individuals is also very limited. |
Consequences of the independent contractor model (EBP-specific) | |||
Impacts staff selection | 3 | 33 | We only have contractors. So, the agency, even if they do have money, because we do have money for training, is not willing to invest money to send a contractor to an expensive training…that they may or may not get a return on. |
Contractors can not meet initiative requirements |
4 | 44 | But where the challenge has been is that we have people who are contractors, and therefore, their ability to commit time to the project is limited. |
Contractors have to be committed to learning EBP because they are not paid for extra time |
1 | 11 | I have to find contracts who are willing to do [EBP], which I am not paying them for, including training and consultation, which means ultimately they would just have to be committed to the fact that they want to learn a skill. |
Incentive for contractors to use EBP |
1 | 11 | Per diem staff [i.e., independent contractors] are only paid if they’re doing a session. So, they are going to do what works with the people they see. There is an incentive there to do that. |
Alternatives or ways to improve the independent contractor model | |||
Treating contractor clinicians with respect/providing opportunities for professional development |
5 | 56 | We feel strongly enough about the initiative and we want them to participate. We are paying for their training time as per their hourly fee-for-service rate. And they’re not salaried employees. So the agency takes on a certain amount of cost for that as well as some reduced productivity because people spend more time in supervision and beyond the actual face-to-face session with the consumer. What we have tried to do, the only thing we changed [when we transitioned to the contractor model] is how the staff are paid. Other than that, we behave as a department with staff who receives supervision. |
Shared risk model | 1 | 11 | For employees we have tuition reimbursement. When you sign up for tuition reimbursement, you also sign off that you will be at the agency for 2 years or you have to pay the money back. So similarly, I think they could include this in the contract of independent contractor clinicians. |
Primary care integration |
1 | 11 | I think we’re going to keep the [salaried] model because we believe in it and in addition to that, it is going to be integrated in primary care and behavioral health. We’re going to try to go as far as we can with the health home models. |
EBP = evidence-based practices
Participants also noted the specific impact of independent contractors on participation in EBP initiatives. Respondents stated that they would not send contractors to EBP trainings because of the potential loss of investment due to contractors’ perceived transiency and turnover. Furthermore, the additional requirements of EBP initiatives (e.g., training and ongoing consultation) made it difficult to include contractors in these efforts.
A majority of participants shared their opinions of alternatives or improvements that could be made. Several participants noted that it is important to treat independent contractors with respect and provide them with professional development opportunities to increase their connection to the organization and reduce turnover. They gave examples including increasing the hourly rate of independent contractors trained in EBPs, paying for parking when independent contractors go to trainings, and providing clinical support. Some participants advocated paying a reduced training rate for independent contractors attending clinical support meetings.
Discussion
Nationally, a growing number of community mental health clinics are moving towards relying on independent contractors. We used mixed methods to begin to understand the associations between worker status and factors associated with implementation of EBPs (9). Our quantitative results suggest that independent contractors reported less positive attitudes towards EBPs and less knowledge than salaried clinicians. Our qualitative results provide preliminary insights into why some organizations are using independent contractors and consequences in the organizations we studied.
Our quantitative data suggest that independent contractors were not less likely than salaried therapists to participate in EBP initiatives; however this finding was not corroborated by the qualitative interviews. Furthermore, independent contractors demonstrated poorer knowledge of and attitudes towards EBPs, suggesting potentially less access to professional development opportunities. Administrators shared that their organizations would be less likely to include independent contractors in EBP initiatives because independent contractors were more likely to turnover (26) and less able to participate in initiative requirements (10, 27), suggesting that the increase in the number of organizations using independent contractors may pose a threat to implementation of EBPs. Although preliminary, these findings have important implications for the calls to action to better train the behavioral health workforce (1–3, 5, 28, 29). There may be a segment of the specialty provider workforce receiving less access to training opportunities. Raising awareness about this is an imperative for future efforts to support the behavioral health workforce. It is important to note that the clinician and organizational variables studies are only factors associated with implementation. Future research should investigate the impact of using independent contractors on actual implementation of EBPs.
The proportion of independent contractors in an organization did not affect organizational culture and climate. However, we were underpowered to detect small effects at the organizational level. For this reason, we included results significant at p < .1. These results suggest that organizations with more independent contractors may have less stressful climates. This finding may be explained by productivity requirements given that salaried staff are expected to bill a certain number of weekly patient hours whereas our informal conversations with stakeholders suggest that independent contractors decide how many hours they would like to work each week. Given the high no-show rate in community mental health (30), this may contribute to increased pressure to bill and/or carry a high case load which may be reflected in a stressful climate. Although the results do not indicate causation, it is important to attend to the role of climate as it is impacted by worker status given that organizational climate is predictive of job performance (31, 32) and client outcomes (33, 34).
A number of important preliminary insights were gleaned from the qualitative interviews. Participants discussed the rationale for using independent contractors and their perceptions of the impact of such a model on their organization and staff. Overwhelmingly, participants discussed the dire financial environment in community mental health (26, 35, 36), and the reality that they could not provide salary and benefits to their therapists. There may be alternatives to improve organizational fiscal health such as teaching organizations better business practices (37) or changing federal and state policies to support these organizations. The general consequences of relying on independent contractors were perceived as negative for both the organization and the staff. Stakeholders suggested that independent contractors were more likely to turnover, less likely to feel connected to their organization, and more likely to provide poorer clinical services.
A number of study limitations should be noted. First, and most importantly, we did not have information about therapist behavior, thus precluding us from drawing conclusions on the actual implementation of EBPs. Second, not all therapists in the organizations participated in the quantitative portion and we did not directly talk with therapists in the qualitative portion. Third, our interviews were conducted with a small number of organizations in Philadelphia, so the generalizability is unknown. Fourth, administrators may be biased towards their fellow employees (e.g., salary increases are tied to one another). Finally, we did not collect information regarding whether cases are assigned differentially to clinicians based on their employment status. Assigning cases according to the unique strengths of contractors and employees may resolve some of the concerns identified in this article.
Conclusions
The findings from this study have important implications. Very little has been written about the increasing number of independent contractors in the behavioral health workforce; in a report to Congress on mental health workforce issues, no mention was made of this phenomenon (35). Non-scholarly discourse around this topic has been plentiful, as many individuals, organizations, and systems working within and with specialty mental health outpatient services are impacted by this shift. It is our hope that this initial investigation launches a systematic research agenda that delineates the number and characteristics of independent contractors in the behavioral health workforce, the characteristics of organizations utilizing independent contractors, and the associations with organizational, clinician, client, and EBP implementation efforts. Although this is an important topic to examine from an implementation science lens, it has a much broader potential impact on mental health services delivery that is in critical need of study.
Acknowledgments
Disclosures and acknowledgements.
Disclosures. Dr. X receives royalties from Oxford University Press and has served as a consultant for Kinark Child and Family Services. Dr. ZZ has received grant support from Ortho-McNeil Janssen and Forest Research Institute and has served as a consultant to AstraZeneca and Alkermes.
Acknowledgements. We are especially grateful for the support that the Department of Behavioral Health and Intellectual disAbility Services has provided for this project, and for the Evidence Based Practice and Innovation (EPIC) group.
Funding. Funding for this research project was supported by the following grants from National Institute on Mental Health: (K23 MH099179, Dr. X; F32 MH103960, Dr. Y; F32 MH103955, Dr. Z). Additionally, the preparation of this article was supported in part by the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and Quality Enhancement Research Initiative (QUERI), Department of Veterans Affairs Contract, Veterans Health Administration, Office of Research & Development, Health Services Research & Development Service. Dr. X was an IRI fellow from 2012–2014.
Contributor Information
Rinad Beidas, University of Pennsylvania - Psychiatry, 3535 Market Street 3015, Philadelphia, Pennsylvania 19104, rbeidas@upenn.edu.
Rebecca Stewart, University of Pennsylvania.
Courtney Benjamin Wolk, University of Pennsylvania - Psychiatry, Philadelphia, Pennsylvania.
Danielle Adams, University of Pennsylvania - Psychiatry, Center for Mental Health Policy and Services Research 3535 Market Street 3rd Floor, Philadelphia, Pennsylvania 19104.
Steven C. Marcus, University of Pennsylvania - School of Social Policy & Practice, Philadelphia, Pennsylvania
Arthur Evans, Philadelphia Department of Behavioral Health, Philadelphia, Pennsylvania.
Kamilah Jackson, Community Behavioral Health.
Geoffrey Neimark, Community Behavioral Health.
Matthew Hurford, Philadelphia Department of Behavioral Health and Intellectual disAbilities, Pennsylvania.
Joan Erney, Community Behavioral Health.
Ronnie Rubin, Community Behavioral Health, Philadelphia, Pennsylvania.
Trevor R. Hadley, Univ Of Pennsylvania - Ctr For Mental Health, 3600 Market St, Rm 717, Philadelphia, Pennsylvania 19104
Frances Barg, University of Pennsylvania.
David S. Mandell, University of Pennsylvania Perleman School of Medicine - Department of Psychiatry, Philadelphia, Pennsylvania
References
- 1.Hoge MA, Stuart GW, Morris J, et al. Mental health and addiction workforce development: federal leadership is needed to address the growing crisis. Health Affairs. 2013;32:2005–2012. doi: 10.1377/hlthaff.2013.0541. [DOI] [PubMed] [Google Scholar]
- 2.Hoge MA, Morris J, Daniels A, et al. An action plan on behavioral health workforce development. Rockville, Md: US Department of Health and Human Services, Substance Abuse and Mental Health Services Administration; 2007. [Google Scholar]
- 3.Hoge MA, Morris JA, Stuart GW, et al. A national action plan for workforce development in behavioral health. Psychiatric Services. 2009;60:883–887. doi: 10.1176/ps.2009.60.7.883. [DOI] [PubMed] [Google Scholar]
- 4.Cooper JL, Aratani Y. The status of states' policies to support evidence-based practices in children's mental health. Psychiatric Services. 2009;60:1672–1675. doi: 10.1176/ps.2009.60.12.1672. [DOI] [PubMed] [Google Scholar]
- 5.Hoge MA, Morris JA. Implementing best practices in behavioral health workforce education-building a change agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2004;32:85–89. doi: 10.1023/b:apih.0000042741.19527.a8. [DOI] [PubMed] [Google Scholar]
- 6.Davis-Blake A, Uzzi B. Determinants of employment externalization - a study of temporary workers and independent contractors. Administrative Science Quarterly. 1993;38:195–223. [Google Scholar]
- 7.Davis-Blake A, Broschak JP, George E. Happy together? How using nonstandard workers affects exit, voice, and loyalty among standard employees. Academy of Management Journal. 2003;46:475–485. [Google Scholar]
- 8.Glisson C, Schoenwald SK, Hemmelgarn A, et al. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology. 2010;78:537–550. doi: 10.1037/a0019160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Beidas RS, Marcus S, Aarons GA, et al. Predictors of community therapists' use of therapy techniques in a large public mental health system. JAMA Pediatrics. 2015;169:374–382. doi: 10.1001/jamapediatrics.2014.3736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Stirman SW, Bhar SS, Spokas M, et al. Training and consultation in evidence-based psychosocial treatments in public mental health settings: the ACCESS Model. Professional Psychology-Research and Practice. 2010;41:48–56. doi: 10.1037/a0018099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Beidas RS, Stewart RE, Adams DR, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research. doi: 10.1007/s10488-015-0705-2. in review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Weisz J. Therapist Background Questionnaire. Los Angeles: University of California; 1997. [Google Scholar]
- 13.Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS) Mental Health Services Research. 2004;6:61–74. doi: 10.1023/b:mhsr.0000024351.12294.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Aarons GA, Glisson C, Hoagwood K, et al. Psychometric properties and U.S. National norms of the Evidence-Based Practice Attitude Scale (EBPAS) Psychological Assessment. 2010;22:356–365. doi: 10.1037/a0019188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Stumpf RE, Higa-McMillan CK, Chorpita BF. Implementation of evidence-based services for youth assessing provider knowledge. Behavior Modification. 2009;33:48–65. doi: 10.1177/0145445508322625. [DOI] [PubMed] [Google Scholar]
- 16.Glisson C, Landsverk J, Schoenwald S, et al. Assessing the Organizational Social Context (OSC) of mental health services: implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:98–113. doi: 10.1007/s10488-007-0148-5. [DOI] [PubMed] [Google Scholar]
- 17.Glisson C, Green P, Williams NJ. Assessing the Organizational Social Context (OSC) of child welfare systems: implications for research and practice. Child Abuse & Neglect. 2012;36:621–632. doi: 10.1016/j.chiabu.2012.06.002. [DOI] [PubMed] [Google Scholar]
- 18.James LR, Demaree RG, Wolf G. Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology. 1984;69:85–98. [Google Scholar]
- 19.Brown RD, Hauenstein NMA. Interrater agreement reconsidered: an alternative to the r(wg) indices. Organizational Research Methods. 2005;8:165–184. [Google Scholar]
- 20.Bliese P. Multilevel Theory, Research, and Methods in Organizations. San Francisco: Joseey-Bass; 2000. Within-group agreement, non-independence, and reliability: implications for data aggregation and analysis. [Google Scholar]
- 21.Beidas RS, Edmunds JM, Cannuscio CC, et al. Therapists perspectives on the effective elements of consultation following training. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40:507–517. doi: 10.1007/s10488-013-0475-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hill CE, Knox S, Thompson BJ, et al. Consensual qualitative research: an update. Journal of Counseling Psychology. 2005;52:196–205. [Google Scholar]
- 23.Hill CE, Thompson BJ, Williams EN. A guide to conducting consensual qualitative research. Counseling Psychologist. 1997;25:517–572. [Google Scholar]
- 24.Stirman SW, Miller CJ, Toder K, et al. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science. 2013;8:1–12. doi: 10.1186/1748-5908-8-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
- 26.Beidas RS, Marcus S, Benjamin-Wolk C, et al. A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research. doi: 10.1007/s10488-015-0673-6. (Epub ahead of print, July 16, 2015) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Creed T, Stirman S, Evans A, et al. A model for implementation of cognitive therapy in community mental health: the Beck Initiative. The Behavior Therapist. 2014;37:56–65. [Google Scholar]
- 28.Hewitt A, Larson S, Edelstein S, et al. A synthesis of direct service workforce demographics and challenges across intellectual/developmental disabilities, aging, physical disabilities, and behavioral health. Washington, DC: National Direct Service Workforce Resource Center; 2008. [Google Scholar]
- 29.Hoge MA, Paris M, Jr, Adger H, Jr, et al. Workforce competencies in behavioral health: an overview. Administration and Policy in Mental Health and Mental Health Services Research. 2005;32:593–631. doi: 10.1007/s10488-005-3259-x. [DOI] [PubMed] [Google Scholar]
- 30.Defife JA, Conklin CZ, Smith JM, et al. Psychotherapy appointment no-shows: rates and reasons. Psychotherapy. 2010;47:413–417. doi: 10.1037/a0021168. [DOI] [PubMed] [Google Scholar]
- 31.Judge TA, Thoresen CJ, Bono JE, et al. The job satisfaction-job performance relationship: a qualitative and quantitative review. Psychological Bulletin. 2001;127:376–407. doi: 10.1037/0033-2909.127.3.376. [DOI] [PubMed] [Google Scholar]
- 32.James LR, Choi CC, Ko CHE, et al. Organizational and psychological climate: a review of theory and research. European Journal of Work and Organizational Psychology. 2008;17:5–32. [Google Scholar]
- 33.Glisson C, Green P. Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect. 2011;35:582–591. doi: 10.1016/j.chiabu.2011.04.009. [DOI] [PubMed] [Google Scholar]
- 34.Williams NJ, Glisson C. Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: a United States national study. Child Abuse & Neglect. 2014;38:757–767. doi: 10.1016/j.chiabu.2013.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Hyde PS. Report to Congress on the Nation's Substance Abuse and Mental Health Workforce Issues. US Department for Health and Human Services, Substance Abuse and Mental Health Services Administration. 2013 [Google Scholar]
- 36.Stewart R, Adams DR, Mandell D, et al. The perfect storm: collision of the business of mental health and the implementation of evidence-based practices. Psychiatric Services. doi: 10.1176/appi.ps.201500392. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Lloyd D. How to Maximize Service Capacity: Nuts and Bolts Solutions for Implementing Change in Behavioral Healthcare Organizations Below the Senior Management Level. Rockville, Md: National Council for Community Behavioral Healthcare; 1998. [Google Scholar]