Quality Indicators for Multidisciplinary Team Functioning in Community-Based Children’s Mental Health Services

Abstract

The current study examined the organizational social context in 21 community-based programs serving youth at-risk for out-of-home care due to emotional or behavioral disorders and their families and program performance on five quality indicators of team functioning in teams that included a family support specialist. Results indicate that programs with higher performance on structures to facilitate teamwork, informal communication mechanisms among team members, and the ability to integrate family support specialists as equal members of the team showed more positive organizational functioning. Implications for the role of quality indicators in health care reform efforts are discussed.

Introduction

Multidisciplinary teams, which are comprised of staff who vary in their educational and professional experiences, bring together diverse perspectives, expertise, and skills (Kozlowski and IIgen 2006). Driven by an increased demand for innovative and increasingly complex services that require different levels of expertise (Roberge and van Dick 2010), multidisciplinary teams have become commonplace across health and mental health settings, and are widely considered fundamental to service delivery (Leipzig et al. 2002; Schofield and Amodeo 1999). With increasing specialization in healthcare, and expanding knowledge about health and mental health conditions, having a forum for assembling groups that reflect different areas of specialization is increasingly necessary. Some of the benefits of multidisciplinary team models include improved consumer health outcomes and functioning, enhanced quality of life, reduced costs, and utilization of medical services (e.g., Wensing et al. 2006). They have shown particular promise in mental health settings, which typically have high rates of staff burnout and turnover, as they safeguard against fragmented care among patients requiring long-term treatment (Vinokur et al. 1995).

The growth in popularity of these models has resulted in a growing literature base about team functioning and processes that facilitate and impede their effectiveness. Most theories of team functioning derive from McGrath’s (1964) IPO algorithm, in which Inputs (I), refers to team composition, Processes (P) describes team activities that impact functioning, and Outputs (O) describes the performance and cohesiveness of the team as viewed by persons external to the team, meeting the needs of team members, and members’ agreement to continue to be part of the team (Hackman 1987).

Processes that encourage successful team functioning include establishing mutual trust among team members, open communication, maintaining equality among team members, and providing and receiving feedback (Firth-Cozens 2001; Molyneux 2001). These processes are consistent with Grumbach and Bodenheimer’s (2004) conceptualization of five elements underlying successful team functioning, which are: (1) clear and defined goals, both on an organizational level via the setting’s mission, and team goals that are measurable, (2) systems, including a set of procedures for given activities (e.g. scheduling appointments, obtaining prescriptions), (3) division of labor based upon each team member’s role, (4) training in each team member’s respective area, and (5) communication.

Multidisciplinary teams can both enhance and impede team functioning by virtue of their diversity (Roberge and van Dick 2010). Evidence suggests team heterogeneity encourages creative problem solving and produces more and better quality decisions than homogeneous teams. However, team integration is harder to achieve among members of varying disciplines, because members have different perspectives and experiences. Important processes, such as open communication and coming to a mutual agreement, are also more difficult to attain, and the potential for competition and conflict is higher (Lichtenstein et al. 1997; van de Ven et al. 2008).

One emerging multidisciplinary team model within the child mental health system is a peer-based model, in which adult family members of children with serious emotional and behavioral disorders work as family support specialists (FSS) alongside case managers, mental health clinicians, and other professionals, providing outreach, information, advocacy, and emotional support to families (Cavaleri et al. 2010; Hoagwood and Kolko 2009; Hoagwood et al. 2010; Winters and Metz 2009). FSSs, who have experience caring for a child with identified mental health needs and navigating child-serving systems, are in a special position to interact with parents whose children are being treated in the children’s mental health system. They are able to build and earn trust, instill hope, and facilitate engagement with service providers of parents whose children are being treated in the children’s mental health system (Hoagwood et al. 2010).

Perhaps the most widely recognized implementation of a multi-disciplinary team approach in children’s mental health is the Wraparound model, whose primary goal is to blend the perspectives of families and providers and build successful partnerships between them. Wraparound is a structured team planning process that incorporates a dedicated full-time facilitator, who has access to family and youth support specialists, called Family Partners. The facilitator engages the youth and family to build a service team, whose members hold periodic face-to-face meetings with family members to build, access and monitor an agreed-upon service plan (Miles et al. 2011; Walker 2008). These meetings occur at least once a month for the next 12–24 months. Compared to traditional services models, families receive similar services as provided in Wraparound service plans but the Wraparound process has several unique features. These include: having a dedicated facilitator for the team, a services plan that follows a fixed set of Wraparound principles to ensure family voice and choice are primary, and flexible funding streams to meet unique family needs (Suter and Bruns 2009).

The Family Partner, who has lived experience as a parent of a child with emotional or behavioral challenges, is a formal member of the Wraparound team. A Family Partner has been described as a mediator, a facilitator, or a bridge between families and agencies to encourage the process of collaboration by being a model and coach (Penn and Osher 2008). The Family Partner’s primary role is to help ensure each family is heard and their individual needs are being addressed and met. To accomplish this, the Family Partner helps the family to engage and actively participate on the team and to make informed decisions that drive the process.

Experienced Family Partners report that the Wraparound principle of collaboration, that is, the process of team members developing a single Wraparound service plan, tests their skills the most. Family Partners must align themselves with the family and support the family’s choices, not letting their own personal views influence the decision making process, simultaneously building a partnership with the Wraparound Facilitator and other team members (Penn and Osher 2008). Because group problem-solving process is central in the Wraparound process, it is essential to have a highly skilled facilitator with a working knowledge of both group processes and participatory decision-making to ensure family-driven care is achieved (Walker 2008). The function of the Family Partner in the Wrap Around approach, however, is not limited to being a Wraparound team member as the function of the Family Partner is dependent on the resources and capabilities of the community in which it is being implemented (Miles 2008). In some communities, who operate without the benefit of a wraparound meeting, the Family Partner role may be limited to providing peer support to families while other communities may use the Family Partner to deliver direct services and intervention to caregivers or assisting with start-up activities as families enter the services process, or filling the role of a Wrap Around Facilitator.

An analysis of the implementation literature on the Wraparound process (Bertram et al. 2011) revealed limited empirical studies examining the relationship between staff education, experiential backgrounds, and the use of specific Wraparound training content. Additionally, little evidence documents the fidelity of implementation of training in the Wraparound process. Walker et al. (2011) suggest additional barriers in the engagement and planning process is a lack of knowledge of what promotes a blending of perspectives and the skills needed to carry out these processes.

Presently, while there are numerous discussions and descriptions on the role and functions of the FSSs in the children’s mental health field (Davis et al. 2010), there is a lack of empirical investigations that examine the functioning of peer/non-peer teams working within community-based mental health centers, a commonly implemented model within the child mental health system (Center for Health Care Strategies 2012). This alternate approach to the Wraparound process is implemented by community-based mental health organizations where team members are managed by the mental health organization.

Studies that examine working with adults with serious mental illnesses (SMI) where the models are more established may help inform the effective use of FSSs within multidisciplinary teams. A series of qualitative studies of teams composed of peers and mental health professionals found peers experienced negative and stigmatizing staff attitudes, role ambiguity, insufficient supervision and training, low rates of pay, and difficulty transitioning from a consumer of services to a staff member (Chinman et al. 2008; Gates and Akabas 2007; Gates et al. 2010; Moll et al. 2009). Non-peers (mental health professionals) voiced their own concerns about peers maintaining professional boundaries, lack of clarity about how to integrate the peer as part of their team, and concerns about self-disclosure and the peer’s emotional stability (fears that the peer will become symptomatic or will not be able to manage the stress from their job (Chinman et al. 2008; Gates and Akabas 2007; Gates et al. 2010; Moll et al. 2009).

Although studies of the more general literature and studies of peers working as part of a team for adults provide some context about potential issues and facilitators to teamwork, the generalizability of this literature to teams within the child mental health system is limited. First, unlike peers who work in tandem with other professionals to deliver services to adult consumers, FSS in children’s services are often the primary provider whose foci are on the well-being of the child’s parents/caregivers and on ensuring that their values, preferences, and choices are included in the child’s service planning. Unlike the rest of the service team, whose focus is on the child, FSSs, representing the caregiver and family perspectives, may use different vocabularies and languages to reflect parent/caregiver concerns and to facilitate the integration of parent/caregiver needs into services.

Second, unlike adult peers who have personal experiences with the mental health system and who may function as case managers (Davidson et al. 2006), the primary goal of FSSs in children’s services is to help ensure the caregiver’s perspective and voice is integrated into treatment planning. The credentials for employment are largely based on their own “lived experience” rather than degrees or more traditional educational experiences that qualify team members as case managers or therapists. Thus, concerns on how to value this kind of experience and role may be either implicitly or explicitly felt among other team members.

Third, peers in the adult literature are typically consumers of mental health services, and may face a different kind of stigma than parents, who are usually not consumers, but who may have experienced marginalization and blame for their child’s emotional/behavioral issues within the mental health system (Davis et al. 2010; Hoagwood et al. 2010).

Finally, the literature on adult peers mainly describes models in which peers join an existing team and perform the role of another professional (e.g., peers acting as case managers). In contrast, team models with FSSs are typically hired for the specific purpose of representing the parent perspective, and not to supplant or perform the role of another professional. Any or all of these factors could potentially impact team functioning.

The Role of Social Context of Organizations

Studies on the social context of organizations in children’s mental health services systems show that organizational climate and culture have an effect on service implementation and outcomes in child-serving agencies (Aarons et al. 2012; Glisson 2002; Glisson and Green 2006). The socio-technical model of organizational effectiveness (Glisson et al. 2013) constructs mental health treatment as a core technology that is embedded in the social context of an organization.

Mental health services depend on social and technical processes, social norms and perceptions within organizations and affect how services are delivered and organizational cultures and climates may be barriers to the delivery, adoption, or implementation of services. The organizational social context (OSC) survey (Glisson et al. 2008), for example, examines social and technical relationships across three dimensions (1) expectations and priorities on how work is carried out within an organization (i.e., organizational culture), (2) psychological impact of the wellbeing and functioning of staff (i.e., organizational climate) and (3) satisfaction and commitment of staff (i.e., work attitudes or morale). Importantly, improved social context within mental health organizations resulted in better outcomes for children (Glisson et al. 2013). While it has been shown that child-serving agencies develop unique social contexts and these contexts guide service provision and influences the relationship that develops between service providers and consumers, little is known about how the social context of an organization affects the structure and process of multidisciplinary teams when the team includes a FSS.

Currently, there is a need to build the empirical base for quality measures that delineate appropriate benchmarks of care in the children’s mental health system, especially in less developed infrastructures associated with patient-centered care (Pincus et al. 2011). To address this deficit, the purpose of this study is to (1) explore the association between an organization’s social context and corresponding performance on quality indicators focused on multi-disciplinary team structures and processes where the team includes a FSS and all team members are employed by the same organization, and (2) describe the variability in organizational performance on specific quality indicators focused on team structures and process when the team includes a FSS. The results of this study will inform improvement efforts in the child mental health arena by isolating specific quality measures of FSS services and in turn, these measures may help to form the basis for developing organizational accountability and performance standards in providing high-quality mental health care for children and their families.

Methods

Design

This study is a correlational study of data gathered from a larger project of family support services. Quantitative data gathered through the OSC measure (Glisson et al. 2008) were compared to performance on five quality indicators of positive team functioning. The quality indicators were developed through an expert panel using a modified Delphi approach. To determine the level of performance on each indicator, semi-structured interviews were conducted with three staff members within 21 programs. Research staff independently rated the performance on each indicator based on the content of the staff interviews. The quantitative data is combined with quotes taken from staff interviews to illuminate barriers and facilitators to team functioning.

Participants

The participants for the current study were line and administrative staff from the Home and Community Based Services (HCBS) Waiver Programs operating throughout New York State. The HCBS Waiver Program is an intensive, community-based program serving children with serious emotional disturbances, at risk-for out of home placement, who are between 5 and 17 years of age (prior to their 18th birthday). HCBS Waiver Programs are federally approved programs authorized under title XIX of the Social Security Act; these programs provide states the flexibility to design and offer a mix of service options in order to meet the meet the needs of specific groups of individuals who would otherwise require institutional care.

Of the 33 eligible HCBS programs, nine programs chose not to participate in the current study; three programs were not eligible to participate as they either did not have at least a half-time FSS or an FSS on staff for at least 6 months. The 21 participating programs are representative of the HCBS programs across the state in terms of program capacity (number of families that can be served ranged from 12 to 144 families; mean = 48.9, SD = 37.5) and region of the state, with at least half of the programs in each of the five regions participating in the study. Approximately one-third (n = 7, 33 %) of the participating HCBS programs were located in the Western region of New York State, followed by five (23.8 %) located in the New York City region, four (19.1 %) located in the Central and Hudson River regions respectively, and one (4.8 %) located in the Long Island region of the state. Program capacity ranged from 12 to 144 families served (M = 48.9, SD = 37.5).

Two hundred forty-four (n = 244) line and administrative staff from these HCBS Waiver programs participated in the current study. Line staff were comprised of individualized care coordinators (ICCs), FSSs, skill-builders (SB), intensive in-home workers (IIW) and respite workers (RW). Two hundred twenty-three (n = 223) line staff completed a measure of organizational culture and climate, the OSC. In addition, semi-structured interviews were conducted by phone or face-to-face with one ICC, one FSS, and the program director (PD) from each participating program, yielding a total of 63 administrative and line staff that completed interviews.

Staff Characteristics

The sample that completed the OSC was primarily female (84.7 %) and Caucasian (65.0 %). Staff were 37.8 years of age on average (SD = 11.7): RW were the youngest (M = 23.9, SD = 1.2), and FSS the oldest (M = 48.3, SD = 10.9). Most staff were ICC (n = 136, 60.9 %), followed by FSS (n = 38, 17.0 %) and SB (n = 34, 15.2 %). Staff had ~10.0 years of experience (SD = 7.4); RW had the fewest years of experience (M = 2.13, SD 1.2), and ICCs the most (M = 11.8, SD 6.7), with the exception of the one intensive in-home services worker who had 14 years of experience.

Most PDs (71.5 %) had a Master’s degree in social work or family therapy/counseling, and were licensed social workers (61.9 %): Almost half (47.6 %) worked at their agency between 1 and 5 years; almost 40 % (38.1 %) worked at their agency between 6 and 10 years. Two (9.5 %) had been employed between 11 and 20 years.

The majority of ICCs either had a Master’s degree in social work or family/therapy counseling (n = 8, 38.1 %), or a Bachelor’s degree/some graduate school (n = 7, 33.3 %). Slightly under half (n = 9, 43 %) were credentialed, either with a social work license (n = 4, 19.1 %), a credential that was not specified (n = 4, 19.1 %), or a Family Development credential (n = 1, 4.8 %). All but 1 (n = 1, 4.8 %) had been employed at their agency for between 1 and 5 years; remaining ICCS were employed between six and 10 years.

Most FSSs earned a high school degree, had some college or an Associate’s degree (n = 15, 71.5 %), followed by four (19.1 %) who had a Bachelor’s degree. Almost three quarters (n = 15, 71.4 %) were credentialed, either for completing a Parent Empowerment Program (n = 10, 47.6 %), or by earning a Family Development credential (n = 3, 14.3 %). Consistent with the other two groups, most FSSs were employed at the agency for between 1 and 5 years (71.4 %), followed by three (14.3 %) who were employed for less than a year and three (14.3 %) who were employed between 6 and 10 years.

Procedure

The study’s research assistant contacted each HCBS Waiver PD by phone, described the study, and secured informed consent. Once consented, they were asked to complete a close-ended survey interview. Once the PD returned the survey, the study’s research assistant scheduled a 1-h meeting, by phone to review the questionnaire and administer a second, open-ended portion of the interview.

Program directors identified one ICC and one FSS from the waiver team to participate in the semi-structured interview. After the PD presented the study to their staff, the study’s research assistant contacted each team member by phone, described the study, and scheduled an in-person meeting to coincide with a scheduled site visit. During the site visit, the research team secured informed consent, and the research assistant administered the survey instrument, which lasted ~1-h in duration.

To recruit line staff to complete the OSC, PDs presented the study to eligible line staff prior to a site visit by the research team. During the subsequent site visit, the research team secured informed consent and administered the measure to line staff. To help encourage honesty in responses, no supervisors were present during the OSC administration. No participants refused to provide informed consent.

All participants were compensated $25.00 for completion of the interviews, provided in cash or gift card. Each HCBS waiver program was also compensated $250.00 for completion of study activities. The appropriate Institutional Review Board provided oversight for human subject’s protection and approved all study procedures.

Measures

Quality Indicators

The quality indicators used in this study were developed using a modified Delphi technique (Boulkedid et al. 2011) using an expert panel and is described in detail in Olin et al. (2013a). To assess the performance on each quality indicator, PDs completed a modified MacArthur Survey that included questions on infrastructure, referrals, supervision and training, and staff roles. PDs, along with ICCs, and FSSs completed a semi-structured interview that focused specifically on what services FSSs provide, how the FSSs works with families, and how FSSs work with other internal HCBS waiver program and external staff. All interviews were audio-taped and later transcribed.

Two members of the research team were trained to review the transcripts and rate the performance of each program on the 14 indicators; interviews were used to determine if programs were either performing below (score = 1), meeting (score = 2) or exceeding expectations (score = 3) on each of the indicators. The final rating for each indicator was the lowest score across the three respondents. Each coder independently rated the interviews and very good reliability across the two raters was obtained averaging 98 % with a range of 88–100 % (kappa = 0.96). Discrepancies in ratings were resolved through consultation with a senior research staff member.

The 14 Program Quality Indicators (PQI) fall within four domains: Role of the FSS (2 items), Fiscal (3 items); Standard to Guide Practice (3 items), and Structural (6 items), see Olin et al. 2013a). Two indicators were not able to be rated due to excessive missing information. Of the remaining 12 PQIs, five were relevant to the current study based on the literature demonstrating that trust, support structures, and communication are critical to high quality team functioning. Of the five PQIs used in the current study, two related to the role and supervision of the FSS: PQI-2: Staff clearly understands the role of the FSS within the goal of the program; and PQI-9: Specialized supervision in place for FSSs to maintain role on team. The remaining three indicators focus on the interactions among team members and the FSS: PQI-10: Structures are in place to facilitate team work, PQI-11: Informal communication occurs among team members; and PQI-14: Program integration of the FSS as an equal member of the team. While the score for each indicator can range from 1 to 3, the aggregate score across the five indicators can range from 5 to 15. Table 1 provides an example of the anchors used by raters to score PQI 14.

Table 1.

Anchors used by raters to score Program Quality Indicator 14

Rating	QI 14 Program integration of the Family Support Specialist as equal members and/or active members of the team in working with a family
Exceeds	Programs make sure that the family voice is being integrated into treatment plans by actively engaging with the FSS as a liaison with the family
Meets	Programs integrate FSS as equal and active members of the service team (e.g., whole team involved in treatment planning)
Below	Program does not seek FSS input on the team (FSS not treated/seen as equal partners on team)

Organizational Social Context

The OSC is a 20-min, 150-item paper and pencil questionnaire administered in person to front-line staff (i.e., staff who carry a caseload) who work at least half-time, are non-contracted, salaried employees within the HCBS waiver program. This included ICCs, FSSs, SB, RW, and IIWs. The measurement model includes three major dimensions of social context (i.e., organizational culture, organizational climate and work attitudes) and each has a unique history in the organizational literature (Glisson and James 2002; Peterson et al. 2000; Reichers and Schneider 1990; Verbeke et al. 1998). Organizational culture refers to the expectations and priorities in an organization that determine the way work is done and organizational climate refers to the psychological impact of the work environment on the personal wellbeing and functioning of employees (Verbeke et al. 1998). The third dimension, work attitudes, is an individual-level construct that includes job satisfaction and organizational commitment, which together capture the morale of those individuals who work in the organization (Glisson and Durick 1988). The OSC is a nationally-normed social context measure for child-serving agencies and has demonstrated excellent psychometric properties (Glisson et al. 2008).

Using Latent Profile Analysis and based on a national sample of child-serving mental health agencies, the OSC generates an organizational profile for each program. Using the scores on the culture and climate subscales, programs can be classified as either “best,” “average”, or “worst” functioning based on a combination of the six OSC dimensions when compared to a national sample of organizations. The Work Attitude or Morale dimension, a combination of the Job Satisfaction and Job Commitments subscales of the OSC, is standardized to a t score with a score of 50 being the mean with a standard deviation of 10.

Results

OSC and Team Functioning

Of the 21 programs participating in the study, seven (33 %) were classified as falling into the “best” organization functioning category on the OSC, 9 (43 %) fell into the worst category, and 5 (24 %) fell in the average category of functioning, see Table 2. The programs in the “best” category of the OSC had the highest percentage of indicators (34 %) in the “exceeds” performance expectations category on the PQIs while the programs in the “worst” category had highest number of PQIs (56 %) rated as being in the “below” performance expectations category.

Table 2.

Program scores on Five Program Quality Indicator’s by OSC profile and Work Attitude subscales

Program	Quality indicators						Summated scores across five Quality Indicatorsa	OSC work attitude subscalesb
	Number and percent of QI items that fell into three categories							Job satisfaction score	Job commitment score	Morale
	Below		Meets		Exceeds
“Worst”c (n = 9) mean	56 %		31 %		13 %		7.9 (2.50)	51.1(9.01)	57.0 (6.97)	54.8 (7.80)
1	2	40 %	1	20 %	2	40 %	10	64.24	70.04	68.78
2	1	20 %	3	60 %	1	20 %	10	44.28	49.75	47.22
3	4	80 %	1	20 %	0	–	6	56.40	57.40	57.45
4	3	60 %	2	40 %	0	–	7	47.27	48.82	48.02
5	0	–	2	40 %	3	60 %	13	56.01	62.47	60.38
6	4	80 %	1	20 %	0	–	6	61.26	59.86	61.18
7	4	80 %	1	20 %	0	–	6	47.27	59.86	54.79
8	4	80 %	1	20 %	0	–	6	47.27	50.31	48.93
9	3	60 %	2	40 %	0	–	7	35.91	54.22	46.13
“Average” (n = 5) mean	28 %		44 %		28 %		10.0(1.87)	61.8 (5.35)	63.6 (3.17)	63.7 (3.54)
10	1	20 %	1	20 %	3	60 %	12	66.75	66.50	67.76
11	2	40 %	3	60 %	0	–	8	64.06	59.17	62.03
12	2	40 %	3	60 %	0	–	8	55.43	61.45	59.49
13	1	20 %	2	40 %	2	40 %	11	56.67	65.12	62.31
14	1	20 %	2	40 %	2	40 %	11	66.04	65.87	67.05
“Best” (n = 7) mean	40 %		26 %		34 %		9.7 (3.10)	61.7 (3.54)	67.8 (8.20)	66.3 (6.17)
15	1	20 %	1	20 %	3	60 %	12	63.11	68.92	67.58
16	1	20 %	0	–	4	80 %	13	64.76	78.31	74.09
17	1	20 %	1	20 %	3	60 %	12	65.75	73.56	71.64
18	1	20 %	2	40 %	2	40 %	11	64.17	66.81	66.77
19	4	80 %	1	20 %	0	–	6	57.06	64.64	62.19
20	5	100 %	0	–	0	–	5	58.46	70.36	66.34
21	1	20 %	4	80 %	0	–	9	58.64	52.27	55.32
Grand mean	–		–		–		–	57.2 (8.43)	62.2 (8.10)	60.7 (8.20)

^aQuality Indicators scores 1 = Below, 2 = Meets and 3 = Exceeds

^bT-score with mean = 50 and standard deviation of =10

^cThe categories of worst, average and best are based on the Organizational Social Context (OSC) scale

The aggregated means across the five PQIs for the programs in each of the OSC categories also reflect this trend with the program in the “best” organizational functioning, and “worst” categories having means of 9.7 (SD = 3.10) and 7.9 (SD = 2.50), respectively, with overall scores on the PQIs ranging from 5 to 13. The scores on Work Attitude (i.e., Morale) reveal that programs in the “best” category on the OSC had a mean of 66.3 (SD 6.17) on Morale while programs in the “worst” category had a mean of 54.8 (SD = 7.80). Overall, programs falling into the best, average and worst categories on organizational functioning had corresponding patterns on the total scores on the Quality Indicators and the Work Attitude scores of the OSC.

These observations are supported with moderate correlations between the OSC categories of best, average and worst, and the aggregated means for the five PQIs [r_s(21) = 0.35, p > 0.05] and significant correlations with the aggregated means of the three PQIs that focus on the interactions among team members [r_s(21) = 0.46, p < 0.05]. Further, there were significant correlations between the OSC Morale, Job Satisfaction, Organizational Commitment subscales and the aggregated means for the five PQIs {[r_s(21) = 0.56, p < 0.01], [r_s(21) = 0.51, p < 0.05] [r_s(21) = 0.48, p < 0.05], respectively}. These results suggest programs that had positive organizational functioning also exhibited positive work attitudes and higher performance on the PQIs, indicating that climate, culture, and work attitudes are associated with overall levels of team functioning as measured by the five quality indicators.

Variation in Quality Indicators

FSS Role

The PQI-2 states that staff clearly understand the role of the FSS within the goals of the program. 43 % of programs that fell in the “best” category of organizational functioning and 33 % of the programs in the “worst” categories of the OSC were rated as exceeding performance expectations on this indicator, suggesting little difference in this indicator across OSC classification. The mean ratings also reflect this trend in the “best,” “average” and “worst” OSC categories with means of 2.0 (SD = 1.00), 2.0 (SD = 1.00) and 1.9 (SD = 0.93), respectively, see Table 3.

Table 3.

Program performance on five quality indicators by OSC profile

Program	Score on quality indicators
	PQI 2 Staff under-stand role of the FSS	PQI 9 Specialized supervision of FSS	PQI 10 Structures to facilitate teamwork	PQI 11 Informal communication among team	PQI 14 FSS as equal member of team
“Worst” programs (n = 9) means	1.9 (0.93)	1.4 (0.73)	1.4 (0.53)	1.6 (0.73)	1.6 (0.73)
1	3	3	1	1	2
2	3	1	2	2	2
3	2	1	1	1	1
4	1	1	1	2	2
5	3	2	2	3	3
6	1	1	2	1	1
7	1	2	1	1	1
8	1	1	2	1	1
9	2	1	1	2	1
“Average” programs (n = 5) means	2.0 (1.00)	1.2 (0.45)	2.2 (0.45)	2.6 (0.55)	2.0 (0.71)
10	3	1	3	3	2
11	1	1	2	2	2
12	1	2	2	2	1
13	3	1	2	3	2
14	2	1	2	3	3
“Best” programs (n = 7) means	2.0 (1.00)	1.3 (0.76)	2.0 (0.82)	2.3 (0.76)	2.1 (0.90)
15	1	3	3	2	3
16	3	1	3	3	3
17	3	1	2	3	3
18	3	1	2	3	2
19	1	1	1	2	1
20	1	1	1	1	1
21	2	1	2	2	2
Grand M	2.0 (0.90)	1.3 (0.70)	1.8 (0.70)	2.0 (0.80)	1.9 (0.80)

Quality indicators scores 1 = below, 2 = meets, 3 = exceeds

Almost all of the responses from the interviews indicated that the role of the FSS was to provide the team with the caregivers’ perspective and their needs and issues. One FSS summed up this sentiment by stating,

They [individualized care coordinators] do rely on us [family support specialists] as parents to give that parent perspective just to help with brainstorming, or what might work, or even just to help with the case load of the concrete stuff, you know, helping a parent in court, or transporting them, or helping find what they need, so it’s very positive.

Responses from ICCs noted that FSS provided the care-giver’s perspective. Some ICCs stated the FSSs facilitated engagement with the team and that the FSSs took direction from the ICCs regarding the way in which they should work with families and in carrying out the service plan.

Most of the responses from PDs agreed with the ICCs that the FSSs took direction from ICCs, but they also reflected that the FSSs’ role was to engage the family in working with the team, and providing the team with the caregiver’s insight. However, some PDs noted that role confusion and lack of clarity about responsibilities, both by staff members and the FSSs themselves, also hindered teamwork. As noted by one PD,

I think sometimes there’s somewhat of an overlap in the services, and we have to figure out roles and boundaries so we are not overstepping.

One FSS also described confusion over her own role when she said,

The aim seems to get lost I feel. I feel like parent sometimes [says] I have two workers, two people to take advantage of and tell them get me this, get me this, I need this, and it’s hard for me to remind parents of my purpose when my purpose almost sounds like the ICCs purpose. I get a little bit confused, like what is so specific and different about my role when the ICC is also doing a lot of the same things?

Supervision of FSS

PQI-9 states that specialized supervision is in place for FSSs to maintain their role on the team. Two programs, one in the “best” and one in the “worst” OSC categories, scored as exceeding performance expectations on this quality indicator; 16 (76 %) programs were rated as being “below” performance expectations on this quality indicator. This indicator also had the lowest mean rating [1.3 (SD = 0.70)] of all five indicators. Responses from the interviews reveal that FSSs are usually supervised by the PD or the ICC, with supervision focused on administrative and clinical issues rather than role-specific responsibilities of the FSS. This lack of attention in supervision to FSSs’ specific duties helps to explain the low performance for many programs on this quality indicator.

PDs also highlighted two additional areas related to supervision of multidisciplinary teams involving FSSs. The first area was a hierarchy of educational degrees due to differences in educational attainment. Differences in job requirements for ICCs, who are expected to have a college degree, and FSSs, who hold personal experience but not necessarily a degree, were noted as causing tension within the team and impeding team relationships.

The second problem area identified as problematic by the PDs was splitting, defined as flattering one staff member at the expense of another by family members. PDs noted that a family member would elevate the FSS and diminish the ICC, causing a rift between staff. As stated by a PD,

Sometimes people are flattered by that a parent will say “I like you so much better than the ICC” and that creates a wounded team. So right now, I’m dealing with one staff member who is feeling flattered by the parent, and the other who is feeling disparaged.

Structures to Facilitate Teamwork

The PQI10 states that structures are in place to facilitate teamwork. Programs in the “worst” category of the OSC did not have any programs that were rated as “exceeds performance expectations” on this indicator while the programs in the average and best categories of the OSC have 20 and 29 % of their programs rated as “exceeds performance expectations”, respectively. The majority of programs (52 %) were rated as “meeting” this performance expectation and the means reflected this trend across OSC categories of “best,” “average,” and “worst” with means of 2.0 (SD = 0.82), 2.2 (SD = 0.45) and 1.4 (SD = 0.53) respectively.

Communication, including ongoing communication and opportunities for communication via team meetings was the most commonly reported facilitator to team functioning in the staff interviews. For all administrative and line staff, ongoing communication was the most salient facilitator that was cited. Weekly meetings allowed continuity of conversations and effective problem-solving. Respondents believed team meetings were a critical venue where they could discuss their work. As noted by one ICC, “We have a staff meeting every week and in that if there’s any major incidents and case reviews, we communicate through that.”

The most common barrier to effective team functioning was scheduling difficulties. Conflicts centered mainly on the FSS’s schedule, and were both professional and personal. County regulations mandate that FSSs attend specific meetings. According to one PD, “Everyone’s busy and working on such limited time frames. They [FSS] have many families and obligations and need to balance their personal life. This can be difficult and sometimes impacts communication. It is a constant work in progress.”

Several barriers were reported by the FSSs. One problem reported by the FSSs was the lack of regular meetings or not always being invited to staff meetings. FSSs felt not having a regular meeting was detrimental to staff learning about family support services and the benefits of these services to families. As reported by one FSS,

We don’t ever really have team meetings, and I’m not saying we have to have a team meeting, but I think the workers should have an understanding of what family support services are, and not that they would be whatever you want them to be, whenever you need them to be, and that’s kind of what it feels like to me right now. There should be, of course within that you should have some leeway about what help you give families, but certainly it shouldn’t be that whatever needs to be at that moment, so we need to define that better.

Working in close proximity was also mentioned as a structure to facilitate team work. Sharing the same work-space enabled spontaneous meetings and communication about families, resulting in a more integrated team. As reported by an ICC,

We’re lucky enough that we’re all housed in the same office, so we run into [FSSs] five times a day. We can sit and have mini meetings about families and things like that.

Informal Communication

PQI-11 states that informal communication occurs among team members. Programs in the “worst” categories had fewer programs (11 %) that exceeded the performance expectation on this indicator than programs in the “average” (60 %) and “best” (43 %) categories of the OSC, reflecting a strong association between this quality indicator and the OSC profiles. This trend is also reflected in the means across OSC categories of “best”, “average,” and “worst” with means of 2.3 (SD = 0.76), 2.6 (SD = 0.55) and 1.6 (SD = 0.73), respectively.

Informal communication via phone calls, texting, and emails were described by ICCs and FSSs as ensuring close communication about their work with families, particularly given the variations in schedules and that much of their work they was conducted in the family’s home. As described by one ICC,

We use all means of communication, so basically emails, which [are] very helpful in our line of work because our schedules are different, so I can be in my office on time, the family support specialist may not be in [her] office, so email is very very helpful, or we just call on the phone.

Less effective programs discussed using informal communication less frequently and less optimally with communication seeming more one-sided. As one ICC mentioned, “We (informally) chat about a family. My family support worker will tell me how her meeting went and what happened during the meeting with a family.”

FSS as Equal Team Member

PQI-14 states that FSSs are equal members and/or active members of the team in working with a family. This indicator reflected a strong association with the three OSC categories more than any other indicator with 43 % of program exceeding performance expectations in the “best” OSC category while 20 and 11 % of the programs in the average and worst categories, respectively, exceeded performance expectations on this indicator. The means also reflect this trend across categories; “best” mean of 2.1 (SD = 0.90); “average” mean of 2.0 (SD = 0.71) and “worst” mean of 1.6 (SD = 0.73).

Staff reported mutual respect among team members as a facilitator to teamwork and to establishing FSSs as equal team members. Mutual respect was defined as respecting all team members’ perspectives and ideas about how to best work with a family. According to one PD, “if the family [support specialist] wants to work on something that perhaps was not part of the service plan; we will amend the plan so they can address that issue.” Further, one PD stated, “I treat the [FSS] as a professional equal and this trickles down to staff…they see her as a valuable tool in navigating the school system [and] going to court.”

Discussion

This is the first study to explore the functioning of multi-disciplinary teams comprised of FSSs and non-peer professionals serving families of children with mental health conditions at-risk for out-of-home care when team members are employed by a single agency. This study found considerable variation in OSC across HCBS waiver programs, which included a multidisciplinary team including a peer FSS. Additionally, five quality indicators associated with team functioning aligned with the program classifications on OSC, indicating that the OSC, a nationally normed measure, is sensitive to variations in team functioning in organizations that have multidisciplinary teams that include FSSs.

The examination of each of the five quality indicators also found variability in performance across indicators. The performance on understanding the role of the FSS within team members was not different across OSC classifications, with many programs rated as exceeding the performance expectation on this indicator. This high level of performance on this indicator was not unexpected. The participating programs in this study have a long history of incorporating family voices into their children’s mental health services. Since the mid-1980s when the federal government funded the Child and Adolescent Service System Program (CASSP; Stroul 1996), the New York Office of Mental Health has viewed family participation as an essential element in service quality. Currently, the role of the FSS in HCBS Waiver programs is to facilitate access to services and increase family voice in the treatment process. As the comments from staff in this study demonstrated, however, understanding the role of the FSS by team members is very different from effectively implementing this role within a multidisciplinary team structure.

Overlapping functions and lack of appreciation of the value and role of family support services have been cited as key impediments to team integration, staff conflict, and suboptimal treatment coordination or planning. A lack of shared clarity about the role of the peer and the services peers provide is a recurring issue in adult mental health services, as both peers and other professional staff have identified role ambiguity, conflict, confusion, poor job structure, and lack of support as common barriers to effective team functioning and undermining team integration (Chinman et al. 2008; Gates and Akabas 2007; Gates et al. 2010; Moll et al. 2009). Successful strategies for enhancing functioning of multidisciplinary teams, such as clarifying role definitions and scope of work, facilitating communication about peer roles and organizational policies, and providing training to peers on their duties and roles within the organization (Gates et al. 2010), are most likely to be effective when there is explicit guidance for organizational leadership and organizational endorsement of peer support workers and programs within the roster of clinical staff and services.

Overall performance on the specialized supervision for FSS to maintain their role on the team indicator was low with over three-quarters of programs falling below performance expectations. While most programs acknowledge a structure for the supervision of FSSs with PDs or ICCs assigned this duty, the quality and the content of the supervision did not focus on how to fully integrate and maintain the FSS role within the team, but rather focused on administrative or clerical issues during supervision. Hierarchies built on educational background and “splitting” (flattering one staff member at the expense of another by family member) were also clear supervisory challenges. This points out the need to have both structures and effective supervisory processes in place to (1) minimize conflict and competition and (2) maximize innovative and better quality solutions associated with a multidisciplinary perspective. Such supervisory or management practices will likely need to expand the focus beyond FSSs to consider effective management of multidisciplinary teams to foster the value of different perspectives, experiences and philosophies to promote high quality service programming.

The majority of programs were rated as meeting the performance indicator of having structures in place to facilitate teamwork (PQI-10). Interestingly, programs in the “worst” category of the OSC did not have any programs that were rated as “exceeds performance expectations” on this indicator while the best and average categories had several programs that exceeded expectations. The most common structures in facilitating teamwork were communicating through formal team meetings and the ability to work in close proximity; the most common structural barriers were scheduling difficulties and not including FSSs in team meetings.

There were also differences in the use of informal communication across team members and the perception that the FSS was an equal and active member of the team. Levels of performance on this measure showed that there were more programs in the “best” organizational classification on both indicators than in the average and worst categories.

Mutual respect among team members and ensuring the FSS as an equal team member were both mentioned as being a facilitator to teamwork. Having parents provide a professional role within the children’s mental health service system represents a major shift in service philosophy and may introduce tensions within multidimensional teams as these new service models introduce philosophical differences between peer-delivered support and more traditional clinical approaches. These quality indicators are in line with the important benchmarks of shared-decision making and patient-centered care within the service quality models in the overarching design of general health care reform (IOM 2006; Pincus et al. 2011).

Overall, the variation in the performance on these PQIs across agencies point to the lack of extant literature on how to build effective structures and processes within programs whose multidisciplinary teams include a FSS. While there is considerable literature on the need for family voice within treatment planning and the credentials of FSSs, evidence-based resources for program administrators on how to supervise an FSS within a team or how to maximize the family perspective in a multidisciplinary treatment team planning process appears to be lacking. These resources could build on the growing literature on family support programming as well as interventions aimed at enhancing the overarching social context of social service agencies. For example, the Availability, Responsiveness, and Continuity organizational intervention (ARC; Glisson et al. 2012) uses three overarching strategies to enhance the use of innovations by staff in child serving agencies.

The ARC model includes (1) enhancing the ability of staff to use tools to identify and address barriers to service innovation and agency effectiveness such as team work, goal setting, and feedback systems, (2) having program staff adopt a set of principles based on effective organizations (i.e., mission-driven, results-oriented and participatory-based), and finally (3) integrating cognitive models and attitudes among staff that are associated with adopting service innovation (i.e., openness to change, psychological safety in the work environment, and commitment to continuous learning). One area for future research is the application of the ARC model to programs that incorporate FSSs within their treatment teams. The resources and strategies emanating from these studies could be used to help assure the effective integration of the FSS into treatment planning and realize the full potential of the role of the FSS in effectively changing the perspective of the treatment team, and hence the effectiveness of the treatments offered to families.

Several limitations must be kept in mind when considering implications of the results. First, staff members that participated as respondents in this study were recruited exclusively from HCBS waiver programs in the State of New York; thus it is not clear whether the findings are generalizable to other child treatment settings. These programs, however, in many ways mirror a medical home model, whose key features include better access, care coordination, prevention, quality and safety (Sai et al. 2004), and hence may be a potential model that could be supported within the Affordable Care Act in the national healthcare reform era.

Additionally, the findings are based on the subjective reports of respondents rather than on direct observation. There was no objective way to verify whether or not a subjective report is an accurate representation of actual team functioning. This limitation is mitigated somewhat by interviewing PDs, ICCs, and FSSs, which allows for triangulation of the responses made by multiple types of staff members, all of whom may have different perspectives. It should be noted, however, that caregivers’ perceptions were not included, a perspective that would offer valuable information into the process. Also, an examination of the size of the agency on the results was not considered although agency size was not found to affect OSC scores in a study using the same agencies (Olin et al. 2013b). Finally, the associations found were not related to consumer outcomes, such as satisfaction with level of involvement and decision-making opportunities in the process. This is an important next step in demonstrating the importance of team functioning to quality service delivery standards.

Currently no performance standard exists for team-based, peer support services. Family support services have grown throughout the country over the last several decades with FSS employed in a variety of organizations (Walker 2008). With this rapid expansion, a variety of family support service models, with variations in the function and roles of an FSS and the emergence of necessary structures to support these roles. To keep pace with the new health care emphasis on accountability and performance standards, it is necessary to measure quality of services, including quality indicators of team functioning. The five indicators captured in this study represent important core indicators in the children’s mental health field when serving children-at-risk for out-of-home care. Uncovering two quality indicators on the role and supervision of FSSs and three quality indicators on the structural requirements necessary to effectively integrate FSSs into an organizations culture represent an important first step in building empirical support for performance standards for peer support services. The finding that programs with positive organizational cultures and climates also demonstrated higher performance on quality indicators on team functioning can contribute to building accountability and quality measurement models, both important cornerstones of health care reform.