Impact of Empowerment Training on the Professional Work of Family Peer Advocates

Abstract

A pilot study using a prospective design examined the impact of a collaboratively developed training model, called the Parent Empowerment Program (PEP), for professionally-employed family peer advocates who work with caregivers of children with mental health needs. This training used a combination of didactic, practice exercises, and group discussion. It targeted specific mental health knowledge content and collaborative skills to facilitate the work of family peer advocates in empowering caregivers. Co-delivered by a family peer advocate and clinician, the training consisted of a 40-hour face-to-face training, followed by six monthly face-to-face booster sessions. A total of 15 advocates participated in assessments conducted at baseline and post-training. This group of experienced family peer advocates showed no significant increase in knowledge about mental health content, but post-training assessments indicated increased collaborative skills and mental health services self-efficacy. This initial evaluation has implications for expanding training and support for the emergent workforce of professionally-employed family peer advocates in children’s mental health.

1. Introduction

In the past decade within the children’s mental health field, there has been an increase in the establishment of local family advocacy, support and education organizations (Koroloff & Friesen, 1997; Friesen & Koroloff, 1990; Hoagwood et al., 2008). These organizations offer a range of professional and voluntary roles for family members seeking to improve services for children. Families seek information and support from their peers, and this is an important adjunct to traditional mental health care. Specific training programs for families about children’s mental health service systems are being rapidly developed with the goal of creating knowledgeable and informed consumers of services (Farmer, Burns, Angold, & Costello, 1997). Few have been rigorously studied to examine their effectiveness (Hoagwood et al., 2010). One model, examined in the well-known Fort Bragg study, found increases in parent self-efficacy 12 months after training among military parents receiving a structured empowerment program (Bickman, Helfinger, Northrup, Sonnichsen, & Schilling, 1998).

Nationally, family peer advocates are increasingly employed throughout the country in a variety of organizations (Walker, 2008); estimates from national surveys of family organizations suggest that the number of family peer advocates in the children’s MH field has risen dramatically in the past decade (Hoagwood et al., 2008; Robbins et al., 2008), with a growing number of states developing certification standards (e.g., GA, MD, MN, ME, FL, NY). In New York State, the Office of Mental Health supports approximately 200 separate family support programs. These programs employ approximately 400 professional family peer advocates, many of whom are parents of children with special needs. Advocates serve as peers to model, coach, and support parents, and serve as important linkages to families. Five regional centers coordinate delivery of the family support programs throughout New York States’ 62 counties (Roussos, Berger, & Harrison, 2008). In New York City alone, an average of 1,068 families and 957 youth are served through these programs every month (Roussos et al., 2008). New York State will be doubling the number of FPA over the next two years, establishing a certification process for the professionalization of this new workforce, and allowing family peer advocates services to be billable under state regulations. In response to this emerging service model, a team of advocates, city and state policy-makers, and researchers collaborated over six years to develop and pilot a training and consultation program for new professional family peer advocates who work with diverse families across the State. This program, called the Parent Empowerment Program (PEP) is co-delivered by an experienced advocate and a mental health clinician. The PEP training follows a manualized curriculum consisting of a core 40-hour, in-person, group-based training; this curriculum is based on an edited book on Improving Children’s Mental Health Through Parent Empowerment: A Guide to Assisting Parents (Jensen and Hoagwood, 2008). The training targeted two broad areas: skills for developing effective working relationships with families, assessing family needs, and strategies for activating families to address their children’s mental health needs; and knowledge about childhood mental disorders, the diagnostic process, evidence-based treatments and service options. During 10 weekly 4-hour training sessions consisting of didactics, practice exercises, and group discussion, the PEP trainers cover six modules on practices and processes that are designed to give trainees the skills to develop effective working relationships with the families with whom they work: 1) Essential engagement and community skills based upon prior research outlining effective strategies to increase initial involvement and ongoing retention in children’s mental health services (e.g. McKay, McCadam, & Gonzales, 1996; McKay, Stoewe, McCadam, & Gonzales, 1998), 2) priority setting and problem solving skills, which teaches advocates how to systematically identify potential obstacles and problem solve barriers to increase parent success towards goals and objectives, 3) group management skills to provide basic information on group development, management, and facilitation skills, 4) understanding psychiatric disorders, the diagnostic process and treatments for children, which covers common child mental health problems, the diagnostic process, evidence-based treatments, and the implications of diagnostic labels for the child and family, 5) the mental health system, an overview of the array of services available in children’s mental health (e.g., outpatient and inpatient care, emergency services, and community supports), and practical tips to prepare parents to navigate this complex system, and lastly, 6) service options through the education system, including basic information about parents’ rights and responsibilities within the educational system, tips on how to partner effectively with teachers and other school staff, and ways to access necessary resources or services to support the child’s academic achievement. After the core 40-hour training, the group continued to meet monthly for 6 months to follow up on special topics of interest.

This brief report describes a first prospective evaluation of the early PEP training model. We hypothesized that PEP would improve advocates’ (a) knowledge of skills for working with caregivers, common child disorders and their treatments, and services options; (b) self-assessment of professional skills related to working with caregivers; and (c) mental health services self-efficacy.

2. Methods

2.1 Participants

All professional family peer advocates working in Family Support Programs in New York City were invited to participate in this IRB approved pilot evaluation. The advocates were introduced to the study during a monthly meeting held by the Office of Mental Health. Family peer advocates who were recruited worked in Family Support Programs within different program/agency sites, including free-standing clinics, hospital-based programs, and school programs. Seventeen family peer advocates (all female) consented to be part of this study, which was conducted during 2006. The mean age of this sample was .42.8 years (SD=10.3). The sample was largely Latino (47%) and African-American (35%); one was Caucasian, and two identified themselves as “other.” More than half (59%) of the sample was employed full-time as an advocate, and this group varied in experience (range 0 to 25 years), with an average of 5.1 years (SD = 6.5 years) of working as an advocate. These advocates worked in a variety of settings, including schools, parent resource centers, hospital-based programs, and outpatient clinics. Advocates varied widely in the number of families they worked with (range 0 to 60), with an average of 13.4 different parents during the last four months (SD=15). They reported diverse job functions; the most commonly reported included providing support and education to parents, service coordination, liaison/linkage services, direct service to children (i.e., tutoring, recreational services), and supervisory functions (e.g., of youth advocates, other staff). Table 1 presents the demographic characteristics of the sample.

Table 1. Demographic Characteristics (n=17).

	n	%	cum %
Race
Latino	8	47%	47%
African American	6	35%	82%
Caucasian	1	6%	88%
Asian/Pacific Islander	0	0%	88%
Other	2	12%	100%
Educational Background
< Bachelor’s Degree	10	63%	63%
Bachelor’s Degree or some graduate	5	37%	100%
Missing	1
Employment
Full time	10	59%	59%
Part time	7	41%	100%
Job Function*
Providing support/education	11	65%
Service coordination	7	41%
Liaison	6	35%
Supervisory functions	5	29%
Direct services to children	4	24%

^aParticipants could endorse more than one job responsibility

2.2. Measures

Advocates were assessed at baseline and post-training. Participants completed paper-pencil measures on three measures:

(1) A general knowledge test was developed by the authors to measure objective knowledge of content keyed to the six PEP training modules. This measure consists of 30 multiple choice, true/false, and fill-in-the-blank questions. Knowledge is calculated as the percent of correct responses, ranging from 0-1, with 1 meaning 100% correct.

(2) Self-assessment of skills and competencies was a measure developed by the authors for training participants to self-rate their sense of competency around key knowledge and skills related to material in the six PEP modules. Each section of the questionnaire corresponds to a separate module and contains six or seven questions anchored along a four-point scale ranging from 0 (disagree) to 3 (agree). Items include “I know how to encourage problem solving and critical thinking in a group” (building teaching and group management skills support) and “I am familiar with the various medications that are used to treat ADHD” (specific disorders and their treatment). Professional skills is computed as the average score of items across all the modules.

(3) A modified Vanderbilt Mental Health Services Efficacy Questionnaire (Bickman, Earls, & Klindworth, 1991) assessed the degree to which family peer advocates feel effective in helping parents access child mental health services. Twenty-five items are scored on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree). Typical items include, “I believe that I can help parents access and use effectively mental health services for their children” and “My skills in dealing with mental health services will help me to empower parents to change things that might be wrong with their child’s treatment.” Mental health self-efficacy is the mean score of summed items.

Paired t-tests were conducted to compare baseline and post training assessments.

3. Results

Fifteen of the 17 advocates completed the full 40-hour core training. Two advocates dropped out after the first few training sessions (one moved away and the other passed away) and hence are not included in these analyses. Table 2 presents baseline and post-training data for the three areas of interest: knowledge, self-assessed skill/competencies, and mental health services self-efficacy. Training did not appear to significantly change the knowledge level among this group of experienced family peer advocates (t=−1.62, p=0.127); however, the training reportedly impacted advocates’ perceptions of their overall professional skills (t=−4.32; p <0.001). Among the set of professional skills examined, this experienced group of advocates did not perceive the training to influence basic advocacy skills involving engagement, listening and boundary setting (t=−2.24, p=0.42); rather they reported significant impact on more complex skills involving priority setting and problem solving (t=−3.15, p=0.007), group management (t=−4.15, p<0.001), application of knowledge about child mental health disorders and treatment (t=−3.31, p=0.005), the mental health care system of care (t=−2.61, p=0.020) and service options through the education system (t=−4.86, p<0.001). The training significantly improved advocates’ sense of mental health service efficacy (t=−2.64; p=0.019).

Table 2. Scores on Knowledge, Skills and Mental Health Services Self-Efficacy Scales for 15 family peer advocates trained in the Parent Empowerment Program.

	Baseline Score		Post-training Score			Paired T-test
		Mean	SD	Mean	SD		T-value p
Knowledge (% correct) a Overall	0.51	0.16	0.58	0.10		−1.62	0.127
Professional Skills b Overall	2.15	0.51	2.71	0.18		−4.32	<0.001 *
Engagement, Listening, and Boundary Setting	2.61	0.32	2.84	0.26		-2.24	0.42
Teaching and Group Management Skills	2.22	0.58	2.78	0.24		−4.15	0.001*
Priority Setting Skills	2.24	0.52	2.69	0.29		−3.16	0.007*
Specific Disorders and Their Treatment	1.92	0.72	2.60	0.35		−3.31	0.005*
Mental Health System of Care	2.16	0.59	2.61	0.33		−2.61	0.020*
School System Services and Options	1.89	0.71	2.80	0.29		−4.86	<0.001*
Mental Health Services Efficacy c Overall	4.12	0.57	4.41	0.30		−2.64	0.019 *

^*p<0.05

^aKnowledge is percent correct range possible range 0-1 (1 would be 100% correct)

Baseline min=0.10 max=0.82

Post min=0.32 max=0.77

^bProfessional Skills is average score for each scale with a range of 0-3; 0=disagree and 3=agree

^cMHSE is the mean score,with a range of

1=Strongly Disagree

2=Disagree

3=Uncertain

4=Agree

5=Strongly Agree

DF for all t-tests is 14 (15-1)

4. Discussion and Conclusion

This study is the first prospective study to examine the impact of a parent empowerment training program co-developed and co-delivered by advocates to improve the effectiveness of family peer advocates working with caregivers of children with mental health needs. While this training was developed specifically to target new advocates, this evaluation primarily involved advocates who had an average of over 5 years of advocacy experience. Thus, while the training did not affect their actual level of knowledge about basic skills for working with families, childhood mental health treatments and service options among this experienced group of advocates, these advocates perceived the training to have influenced their skill/competency level in these areas, as well as their sense of self-efficacy with respect to mental health services. In particular, this group of experienced advocates felt that the training increased their sense of competency around Group Management, Specific Disorders and the Service Options through the School System, topics not typically well trained in practice facilities. It is difficult to meaningfully interpret advocates’ improved sense of competency and mental health self-efficacy in light of the lack of significant improvement on the only objective measure of knowledge in this study. It is possible that this group of experienced and well-educated advocates came to the training with a significant amount of knowledge; alternatively, it is possible that the knowledge test was not well constructed to tap content from the PEP training; this knowledge measure has been revised subsequent to the study.

Overall, this finding highlights the value of the PEP training for helping knowledgeable advocates feel competent about applying what they know in their work with caregivers. Feedback from our participants corroborates this interpretation; responses from a focus group conducted post-training suggested that participants viewed the training favorably and felt that the training enhanced their skills for working with families. This first training evaluation has led to the revision of the PEP model to include a theory-driven framework that guides family peer advocate skill application (Olin et al., in press). To our knowledge, this is the first training model developed specifically for professionally-employed family peer advocates.

4.1 Limitations

This pilot study has several limitations. First, the study included a small sample of well-educated advocates who had considerable experience. Thus, although the PEP training was originally targeted at training new advocates in the field, our recruited sample did not reflect this group. Given the variability in experience of advocates in the field, we have since revised the PEP training model to accommodate the range of advocate experience and education (see Olin et al., in press).

Second, the reliance on self-report of changes as it relates to skill is a serious limitation of this study. While attempts were made to follow-up these advocates one year post training, the high turnover rate in this workforce precluded the collection of adequate data for reporting. Future studies based on a rigorous experimental design is needed; in particularly, follow-up post-training will be important to provide valuable knowledge about the implementation of the training and sustainability of the training effects over time.

Third, this study did not collect information about advocate characteristics that might influence their work with caregivers or their response to training. For example, examining advocate motivation for becoming a family peer advocate, types of problems they experienced with their own children, and other personal attributes could help better identify the training needs of this emerging workforce and direct program planning.

Despite the limitations of this initial pilot study, the results are nevertheless encouraging and point to the need for replication and extension using an experimental design, larger sample, and more rigorous measures. As New York State moves to incorporate family-to-family services within the array of mental health services for children and their families, future evaluations of this training model to assess its impact on family outcomes will be valuable.