Community health advisors’ participation in a dissemination and implementation study of an evidence-based physical activity and healthy eating program in a faith-based setting

Abstract

Community Health Advisors (CHAs) have been widely involved in health promotion, but few details on role expectations, retention, and evaluation have been reported. In a Dissemination and Implementation (D&I) study of an evidence-based healthy eating and physical activity program, 59 churches were randomized to an intervention (n=39) or control (delayed intervention) (n=20) condition. In a novel approach, CHAs worked with church committees rather than congregants by providing training (n=59) and technical assistance (n=54) to the committees to implement a program focused on structural and policy-level changes to support congregants’ behavioral changes. CHA training comprised self-study via electronic training modules, in-person training, and telephone-based training. Evaluation methods were pilot test participants’ and CHAs’ ratings of their training; observers’ ratings of CHAs’ church training delivery; church committee members’ ratings of the training experience, including CHAs’ performance; and data from the TA database to assess CHAs’ adherence to the protocol. The main challenge was the early dropout of one CHA and the reduced role of another. CHAs trained 142 intervention and 60 control church committee members in 9 sessions; they covered 99% (intervention) and 90% (control) of training content, indicating high fidelity. Observers’ scored CHAs’ teaching and facilitation skills at 96.7% (intervention) and 80% (control) of the possible score. CHAs completed 92% of intervention and 93% of control TA calls. The great majority of church participants’ comments regarding CHAs were positive. This study demonstrates that with training and support, CHAs demonstrate high levels of intervention fidelity, confidence, and competence.

Background

Dissemination and implementation of an evidence-based intervention

Dissemination and implementation (D&I) research has grown in prominence in recent years to aid the translation of research to practice in health promotion and disease prevention [1–3]. Dissemination is “an active approach of spreading evidence-based interventions to the target audience via determined channels using planned strategies” and implementation is “the process of putting to use or integrating evidence-based interventions within a setting” [4]. Its purpose is to understand how evidence-based programs can be systematically adopted and scaled up [5]. A D&I study of an evidence-based program shown to increase physical activity (PA) and fruit and vegetable intake in African American churches [6,7] is being conducted in a southeastern state. The original, evidence-based program [program name] was first studied through a randomized effectiveness trial and used a community-based participatory research approach. It focused on environmental, systems and policy-level changes in churches to support individual-level behavior changes.

Consistent with the idea of designing for dissemination [8], the evidence-based PA and healthy eating (HE) program was developed with and for churches, targets organizational change, is delivered by church leaders, allows adaptations to meet local church needs, and was tested in southeastern African American churches. The next step in preparation for wide-scale implementation was a D&I study. While faith-based communities have long been research partners for interventions to improve health and eliminate health disparities, large D&I studies in this setting are lacking, so little is known about factors that influence dissemination and implementation of evidence-based interventions in faith-based settings.

To increase the likelihood of successful dissemination and sustainability of an evidence-based PA and HE program, the D&I study involved Community Health Advisors (CHAs), which represented a novel approach in this context. Rather than the usual approach in which trained lay persons interact directly with program participants (congregants in this case) as lay health advisors, the CHAs provided training and technical assistance to church program committees, who were embedded in their church environment and culture, to prepare and support them in implementing changes at the organizational (church) level that would support congregants in PA and HE behaviors. In Phase 1 of the D&I study, the focus of this paper, the CHA training was developed, pilot-tested, implemented and evaluated, and the CHAs provided the training and TA to churches from multiple denominations in one county. This formative work will set the stage for the Phase II, large-scale dissemination and implementation through CHAs in 12 regions of a single religious denomination across the state.

Community Health Advisors

Community health advisors, similar to lay health advisors, are trained lay persons who participate in the implementation of community-based health promotion programs and research by providing information, group and individual education, support and motivation for program participation. They may also participate in recruitment, assessment, and problem-solving [9]. The feasibility of involving CHAs in program delivery, particularly in underserved and minority communities, has been demonstrated in programs focused on diverse health outcomes, including cardiovascular disease (CVD) risk reduction behaviors [9–14]; however, a review of 20 studies focused on CVD-related behaviors in underserved and minority communities found that specific details on role expectations, attrition and retention, and evaluation of CHA activities was sparse [9].

Objectives

The objectives of this paper are to 1) describe the participation of CHAs in a D&I study of an evidence-based PA and HE intervention in church congregations using a two-level approach in which CHAs trained church program committees and provided technical assistance during implementation; 2) provide details of the CHAs’ recruitment, roles and responsibilities; 3) provide an overview of the CHA training experiences; 4) present evaluation results of the CHAs’ delivery of training to church committees and technical assistance to church program coordinators and pastors; and 5) describe successes and challenges of the CHA approach to program dissemination.

Methods

Design and setting

A group-randomized controlled trial [15] was implemented in SC county, a health professions shortage area [16] with a majority of African American residents (58%) [17], high poverty (23%) [18] and high prevalence of obesity (41%) [19]. Churches were randomly assigned to the intervention (n=39) or control (delayed intervention) (n=20) condition. Control churches received the entire program 12 months after the intervention condition. This study received exempt status from the University’s Institutional Review Board, therefore written informed consent was not required.

Community Health Advisor recruitment

As part of the community partnership, the university contracted with an agency in the community agency to hire a part-time Community Outreach Coordinator, who would assume multiple responsibilities, including those of a CHA, as needed. With the agency’s assistance, county-wide recruitment began in March, 2015 to identify two lay members of local churches for the CHA positions. Recruitment occurred in churches within the participating county beginning in March, 2015. The study’s Community Advisory Committee provided input on the position description and support in identifying applicants by sharing the position description throughout their organizations and networks. Research staff persons distributed the position description during orientation meetings throughout the county and included it in recruitment mailings to churches. The position description included the program overview, the CHA’s essential functions, job requirements, skills and abilities, and compensation. See Table 1 for details.

Table 1.

Position description for community health advisors

Heading	Description
Overview	Two Community Health Advisors (CHA) will be selected in Fairfield County to train local churches and deliver technical assistance phone calls to these churches in a program called [Program name]. The program is part of a research project through a grant from the Centers for Disease Control and Prevention. [Organization name] is a key partner on this project. [Program name] is designed to help churches develop an environment that is supportive of healthy eating and physical activity for its members. It is estimated that 40 churches will be trained in October of 2015, and 20 additional churches in October of 2016. The CHAs will deliver technical assistance calls for 12 months after churches are trained.
Essential functions	1. Successfully complete a full-day training and demonstrate competence in key aspects of the intervention and how to deliver church trainings.
	2. Train local churches how to implement the intervention and tailor it to their congregations through approximately 2-4 full-day trainings.
	3. Assist churches in developing an action plan and budget.
	4. Provide brief technical assistance (TA) telephone calls to key church leaders on a monthly basis, document all TA provided as well as requests for assistance and closely communicate with the Recruitment and Outreach Coordinator.
	5. Actively participate in booster trainings (approximately one hour in length).
	6. Provide other project-related support to the Recruitment and Outreach Coordinator, as requested (e.g., making contact with church leaders that the study staff is unable to reach).
Requirements	Must have a strong interest in health promotion, physical activity and/or nutrition and have experience working in church settings. Previous experience delivering trainings or facilitating groups is highly desired. Strong communication skills, comfort addressing groups and good time management are essential to this position. Community health advisors must be professional and personable with strong organizational skills. As part of a research team, careful attention to details, timelines and excellent documentation are needed. Must possess a valid driver’s license and have reliable transportation, as travel to churches for the trainings will be required. Weekend and some evening hours are required.
Skills and abilities	Community Health Advisors must be at ease communicating and working with groups of church leaders and members from diverse racial and ethnic backgrounds. Strong presentation and group facilitation skills are essential. The CHA must be thoroughly prepared and organized. All training components must be delivered and documented consistently, in a professional manner, across all churches. CHAs are expected to remain engaged with churches and to contact them at scheduled times over the course of the intervention. Close communication with project staff and excellent record-keeping are required to document and problem-solve any challenges that may arise, as well as, to share successes. Flexibility is needed when work schedule must vary.
Compensation	A stipend of up to $1,000 per Community Health Advisor can be earned based on the completion of required documents and activities.

Seven people applied, two withdrew from consideration, and five participated in telephone-based interviews. See Table 2 for details. Two applicants also completed in-person interviews to help the interviewers reach agreement. Staff persons trained the two CHAs started by July, 2015; however, one resigned because of unforeseen schedule conflicts after attending CHA training but before church committee training occurred. Subsequently, the Community Outreach Coordinator took over this CHA’s responsibilities; therefore, references to CHAs below include the Community Outreach Coordinator/CHA and one CHA but omit the CHA who resigned. The Community Outreach Coordinator/CHA was a member of a church in the county and had previous experience working with churches in the county on a previous initiative.

Table 2.

Community Health Advisor Position Screener Questions

Candidate’s Background and Interests

•  Please tell us a little about yourself and why you are interested in being a Community Health Advisor for the program.

•  Please tell me about your interest in physical activity and healthy eating (if not mentioned in answer to question above).

Desired Qualifications

Working in Churches and/or Physical Activity and Nutrition Programs

•  What kind of experience do you have working in churches or with church leaders?

•  Did you participate in the Healthy Eating/Active Living Program? If so, please describe how you were involved in the program at your church.

•  What other types of programs related to healthy eating or physical activity have you participated in?

Public Speaking and Teaching Experience

•  How comfortable do you feel about speaking in front of crowds or groups of people?

•  Have you led any groups or led any group trainings before? If so, tell me about it.

•  How comfortable do you feel addressing a wide range of groups of church leaders and members?

•  Please describe any other types of teaching and/or public speaking experiences that you have participated in.

Skills and Abilities

Strengths and Skills

•  Please discuss a few of your strengths and how you would use your strengths in this position.

Candidate must understand that as part of a research team, careful attention to details, timelines and excellent documentation are needed.

Looking for a candidate with strong organizational, time management, and communication skills.

•  Would you be able to remain engaged with churches and to contact them at scheduled times over the course of the intervention (about 2 years)? Close communication with project staff and excellent record-keeping are required to document and problem-solve any challenges that may arise, as well as, to share successes.

Computer Access and Abilities

•  Do you have reliable access to a computer?

•  How comfortable do you feel using computer programs such as Microsoft Word?

•  Do you regularly use email?

Travel and Hours

•  Possess a valid driver’s license and have reliable transportation?

Travel to churches for the trainings will be required.

•  Available to work hours on some weekends and evenings?

Flexibility is needed when work schedule must vary.

•  How long do you think you will be able to commit to this position?

Interviewers considered the following characteristics in assessing the applicants.

•  Candidate must understand that as part of a research team, careful attention to details, timelines and excellent documentation are needed.

•  Candidate needs strong organizational, time management, and communication skills.

•  Close communication with project staff and excellent record-keeping are required to document and problem-solve any challenges that may arise, as well as sharing successes.

Community Health Advisors’ role and responsibilities

CHAs’ duties were to participate in all CHA training; provide day-long, in-person training to church committees; help trained churches develop and submit an action plan for implementation and a budget; provide monthly technical assistance (TA) to church program coordinators and pastors to check in on progress, provide encouragement, and problem-solve; document these calls and the TA provided in a database; support the research staff as needed; and participate in “booster sessions” with the research team by conference call.

Two CHAs led each day-long training session for church committees assigned to the intervention group (and later to the control churches). After the church committees were trained, the CHAs assisted them in preparing their action plans and budget for program implementation.

CHAs then provided 12 months of telephone-based TA to the church program coordinators and pastors using a rotational schedule, with eight calls to program coordinators and four to pastors. The topics for calls were structured to match the four primary program activities covered in training and in each church’s 12-month program plan (i.e., creating opportunities, messages, leader support, and policies/guidelines for supporting PA and HE, as guided by the program’s theoretical model [20]. Using a semi-structured script, the two CHAs made monthly calls to the church leaders in the 36 intervention group churches beginning in November of 2015 (month 1) and continuing through October of 2016 (month 12). One CHA provided TA calls to control churches on the same schedule beginning in November of 2016 and continuing through October of 2017.

Community Health Advisor training

Overview

Training for CHAs comprised introductory material with audio narration; a full day of in-person training focused on the content and delivery of church committee training and how to be a CHA; an in-person session to practice the church committee training, with observers to provide feedback; and training with audio narration and an in-person segment for the delivery and tracking of monthly TA calls.

The program’s training curriculum for church committees was developed as part of the original program effectiveness study [6] and designed for university staff to deliver t church committees. In the current D&I study, the curriculum was adapted for the CHAs’ use in preparing church committee members for program planning and implementation (i.e., a “train-the-trainer” approach). This approach moved training and ongoing technical assistance further into the community setting, a critical consideration for practical and affordable, large-scale dissemination.

Introductory CHA training

The CHA introductory material was provided as an Adobe Presenter file in a slide show format with audio narration. Organized into five parts, the introductory training material included 67 slides and was 46 minutes in length. This material introduced CHAs to the content and materials they would cover in the church committee training, including basic background information about PA and HE, church committee roles and responsibilities, keys to program success, an overview of the churches’ Assessment and Planning Guide, CHA roles, and tips for being an effective trainer. It included was a 19-item multiple choice self-test and an answer key.

In-person CHA training

The day-long training curriculum addressed CHA roles and responsibilities, the program’s goals, and the training curriculum that CHAs would provide to church committees. Training materials included a 138-slide Power Point presentation with talking points (i.e., use of notes section) for the trainer’s use. Included topics were a program overview, a review of CHA roles and responsibilities with timeline of tasks, an overview of the church committee training session and materials, the rationale for the focus on PA and HE, program goals and components, keys to program success, detailed content on implementing PA and HE policies and activities in churches, and discussion of the entire church committee training sessions and related materials and learning activities

The university project coordinator and the study’s principal investigator delivered the in-person CHA training in August of 2015. CHAs received a scripted version of the Power Point presentation, showing suggested time allotments for each section and a detailed CHA Manual that included training schedules, task checklists, and instructions for leading learning activities. In September of 2015 two CHAs conducted a practice session of the church committee training, with each person taking the lead on a portion of the presentation. Three members of the research team observed the practice session and provided verbal feedback to the CHAs.

CHA technical assistance training

CHAs participated in training for the delivery and tracking of monthly TA calls. They reviewed an Adobe Presenter file with a slide show and audio narration, organized into four parts totaling 62 slides of 55 minutes’ duration. This material focused on the following topics: the CHA’s role in providing technical assistance; CHA qualities and attitudes for successful TA; the difference between facilitation of solutions and giving direct advice; communication skills, with role play examples and activities; the content of a TA call (encouragement and support, reviewing progress and barriers, problem-solving, and goal-setting), how to document the call using a “Call Notes” form; tips for reaching people; and the online database for entering the documentation of the call. Training materials included semi-structured scripts for the TA calls and protocols for the timing of communication attempts to schedule the calls with the church program coordinators and pastors. The CHAs received a phone-based orientation and attended a training session focused on delivering and tracking the calls. See Table 3 for the items in the database. Training for data entry of the Call Notes into the TA tracking database occurred in early November of 2015, during which they practiced data entry. An in-person refresher training on the database tracking system took place in January of 2016.

Table 3.

Community Health Advisors’ Technical Assistance Call Tracking Notes from calls with church coordinators and pastors

Church Coordinator and Pastor Technical Assistance Calls Tracking Items

Church name

Who initiated the contact?

CHA

Church member

Name of the church member reached or attempted to reach

Were you able to reach this church this month and deliver a call? Yes No

Date of completed call

Start time of completed call

End time of completed call

Log of contact attempts

Attempt number

Mode of contact

Attempt date

Attempt time

Attempt notes

Log of follow up activities

Activity number

Mode of contact

Date of activity

Description of activity

Planned date and time of next call

Sample items for tracking program implementation

Sample items for calls with Church Coordinators

Did the Church Coordinator give the Pastor last month’s Pastor Activity from the binder of monthly activities?

Yes No Unknown

Did the Church Coordinator share last month’s cook information from the binder of monthly activities with the person(s) in the church who makes decisions about snacks and meals?

Yes No Unknown

Did the Church Coordinator review and use last month’s materials in the binder of monthly activities?

Yes No Unknown

In the past month, has this church conducted activities to engage and support the Pastor in the program?

Yes No Unknown

In the past month, has this church provided opportunities for members to be physically active before, during, or after church events or meetings, or has the church provided any physical activity programs?

Yes, opportunity or program(s) are in place

No, but a plan is in place

No, and no plan is in place

Unknown

In the past month, has this church provided opportunities for members to eat healthy food options during church events that include snacks or meals (fruits, vegetables, whole grain, low fat, low sodium)?

Yes, healthy food options are provided in snacks and/or meals

No, but a plan is in place

No, and no plan is in place

Unknown

Sample items for calls with Pastors

Has the Pastor been receiving and using the Pastor Activities from the binder of monthly materials?

Pastor received and has tried (1, 2 or 3) Pastor Activities this quarter

Pastor received at least one Pastor Activity but has not tried any this quarter

Pastor has not received any Pastor Activities this quarter

Were bulletin inserts or other handouts about physical activity and healthy eating shared with members in the past month?

Yes No Unknown

Has the Church Coordinator (or other program champion) asked the Pastor to have time to talk about physical activity and healthy eating during worship services and church meetings in the past month?

Yes No Unknown

Has the Church Coordinator given the Pastor messages about physical activity and healthy eating that he/she could share during worship services and church meetings in the past month?

Yes No Unknown

Did the Pastor share messages about physical activity and healthy eating during worship services in the past month?

Yes No Unknown

Does the Pastor wear a pedometer and talk to members about it?

The Pastor wears a pedometer and talks to members about it

The Pastor wears a pedometer but does not talk to members about it

The Pastor does not wear a pedometer

Unknown

CHA booster sessions

The university staff provided ongoing support to the CHAs in the form of “booster trainings” delivered by conference calls of 1.5 to 2 hours during March, May, and August of 2016. The booster training sessions addressed CHA-identified barriers to program implementation or communication issues in need of attention, the content of CHA semi-structured scripts and protocols, the CHAs’ requests for specific topics. and maintenance of church engagement for the second program year.

Evaluation methods

Pre-test of the CHA introductory training

Four people lay church leaders from outside the intervention community, similar to the CHAs on age, gender, and race, were reviewed and critiqued the introductory training materials. They made notes on each section, citing any content that needed clarification or revision, as well as strengths.

Pilot test of the in-person training for CHAs on delivery of the church committee training

The in-person CHA training session was pilot-tested in July of 2015 with four lay leaders and a pastor from three churches not located in the intervention county who were actively involved with their churches’ health ministries. They were lay leaders and a pastor who were interested in implementing the program or something similar with their congregations. They received a small financial incentive and a copy of the program’s Assessment and Planning Guide for church committees. The study’s principal investigator and a co-investigator observed the pilot training session and made notes regarding the presentation’s strengths and limitations.

In-person training for CHAs on delivery of the church committee training

Based on observations from the pilot test, the principal investigator revised the CHA in-person training presentation. In August of 2015, the university project coordinator and the principal investigator provided the CHA training. The principal investigator completed an observation checklist form, which served as a pre-test of the form that would be used to observe CHA delivery of training to churches. The CHAs completed a training evaluation form. They responded using a four-point scale from “strongly disagree” to “strongly agree” regarding ease of CHA training to church committee training, clarity of role and tasks, confidence in skills to do church training, comfort level in contacting the university project staff, and intention to use the materials/resources provided. They rated the CHA training’s pace as “too slow, just right, or too fast.” Open-ended questions addressed what they liked most and least in the training and suggestions for improving it, anticipated challenges in working with church committees, and anticipated assistance needed from the university project staff.

CHA delivery of training to church committees

A research staff member attended each church committee training session and completed an observation checklist form to rate whether the CHAs’ coverage of 60 training content areas and activities in the church training was “complete,” “partial,” or “not delivered.” In addition, staff persons rated the CHAs on their ability to stick to time allotments, the extent to which the church committee members seemed engaged in the training, the church committee members’ positive or negative response to training and level of excitement about the content covered, and the church committee members’ level of boredom during the training session. CHAs responded to open-ended questions regarding topics that may have caused difficulty; topics that needed more time than was allotted; training needs church committee members identified that were not addressed; late arrivals and early departures, and any effects this caused; what went well and what should be done differently next time.

The church committee members evaluated the CHA-delivered training indicating agreement on a four-point scale of “strongly disagree” to “strongly agree” immediately after the training. Items address the following: training easily applied, clarity of role and tasks, confidence to implement the PA component, confidence to implement the HE component, and confidence to work with the pastor and church leaders to implement the program.

CHA completion of technical assistance calls and tracking database

CHAs used “Call Notes” form that corresponded to their TA call scripts to track and describe the TA calls (see Table 3). CHAs entered the information recorded on the form in a secure, online data entry and storage system and also recorded call attempts and any follow-up actions taken after a call. This database provided tracking of the CHAs’ adherence to the study’s protocol as well as a record of implementation fidelity for TA delivery.

Results

Pre-test of the CHA introductory training

The study’s program manager summarized the four introductory training reviewers’ comments. Regarding specific sections of the presentation, the reviewers’ positive comments addressed the faith-based emphasis as affirming; the clear, informative content and good examples; the narrator’s pleasant voice; the appealing program options for churches to implement; and attention to involving the pastor. Comments regarding improvements to the material pointed out minor inconsistencies within the presentation or between the presentation and the accompanying Assessment and Planning Guide; suggestions for more or less emphasis on, clarification of, or re-ordering of a few specific content areas; and notes on minor audio problems. Topics mentioned by one or more persons in response to an open-ended request for comments were the following: script size too small; listening to the script and reading the slides was sometimes hard to coordinate; the slides often advanced before being read completely; overall, training was easy to understand, very informative and simple to follow for a ‘non-science’ person; overall good training, informative material, colorful pictures, and slides are not boring.

Pilot test of the in-person training for CHAs on delivery of the church committee training

The main areas for improvement of the CHA training presentation that the two observers noted during the pilot training were the following: a need for balance between attention given to the training content and attention to the process of being an effective CHA trainer (more guidance on “how to” conduct the church committee training); provide more in-depth orientation to handouts, checklists and facilitation tips for CHAs; improve integration of the churches’ Assessment and Planning Guide into the training session; expand opportunities for CHAs’ role play of facilitation skills; generate more discussion among participants; provide more information on the benefits of PA and HE; add transitional statements when the topic changes; and provide more specific examples on healthy cooking and PA options.

In-person training for CHAs on delivery of the church committee training

The three CHAs completed an anonymous, seven-item questionnaire to evaluate their training session. For six items, they responded “strongly agree, disagree, agree, or strongly agree”: ease of applying the training; role and tasks are clear; confidence in skills to conduct training; comfort in contacting the university project staff plan to use the materials/resources provided. For the first item, two CHAs responded with “strongly agree,” while one CHA responded with “agree,” while all three CHAs responded with “strongly agree” to the other five items. All three CHAs described the pace of the information provided in the training as “just right” rather than “too slow, or “too fast.”

CHA delivery of training to intervention and control church committees

Two CHAs provided the training session to 142 church committee members from 36 intervention churches (36 church program coordinators, 29 pastors, and 77 additional program committee members) on six dates between October 10 and November 21 of 2015 and 60 participants from 18 control (delayed intervention) churches (18 church program coordinators, 9 pastors, and 33 additional committee members) on 3 dates between October 15 and November 28 of 2016.. Each church sent attendees, although in some cases not all church members could attend the same session together. An average of 4 people (minimum=3, maximum=6) from each church attended the intervention training session and an average of 3 people (minimum=1, maximum=5) from each church attended the control training session.

The training evaluation form was completed by 137/142 intervention church committee members and 54/60 control church committee members trained. Across five evaluation items, mean ratings ranged from 3.53 to 3.66 for intervention trainings and 3.45 to 3.67 for control trainings, indicating a high level of confidence in readiness to implement the program in their congregations (see Table 4). Several responses to open-ended questions were related to the CHAs. For the intervention trainings, church committee members spoke positively about the presentation, which was described as “easy to understand, well-planned, and organized” (n=24). They described the community health advisors as “helpful, nice, easy to understand, and fun” (n=13). For the control trainings, church committee members’ responses that mentioned the CHAs (n=7) were that they were great, enthusiastic, well prepared and organized, friendly and pleasant, professional and skilled, made examples easy to understand, and stayed on schedule. There was a critique of CHAs’ reading too much directly from the Power Point slides..

Table 4.

Church committee participants’ evaluation of CHA-provided training

	Intervention n = 137			Control n=54
Item	n	mean (SD)	min., max.	n	mean (SD)	min., max.
What I learned in today’s training can easily be applied to my church.	135	3.63 (.48)	3,4	49	3.51 (.54)	2,4
My role and the tasks needed to carry out the program in my church are clear.	134	3.64 (.48)	3,4	49	3.53 (.54)	2,4
I am confident that I have the skills to put the physical activity part of the program in place in my church.	135	3.56 (.53)	2,4	49	3.51 (.53)	2,4
I am confident that I have the skills to put the healthy eating part of the program in place in my church.	134	3.53 (.52)	2,4	49	3.45 (.68)	1,4
I am confident that I can work with the pastor and other church leaders to put the program in place in my church.	131	3.66 (.49)	2,4	27	3.67 (.48)	3,4

Note: 1=strongly disagree; 2=disagree; 3=agree; 4=strongly agree.

The last item was inadvertently omitted from some control participants’ questionnaires, so the n is small.

The university’s project coordinator (n=4 forms) and a graduate student assistant (n=2 forms) completed observation forms at the 6 intervention church training sessions showed that, across the 60 content and activity areas to be covered at each of six sessions (i.e., 360 in total), there were only three instances in which the CHAs were rated as partially covering the content (<1%), and no instances in which the content was not covered at all. Observation forms completed by two graduate student assistants (not the same observers from the intervention trainings) at 2 of the 3 control church training sessions showed that. across the 60 content and activity areas (i.e., 120 across 2 sessions), CHAs partially covered the content 12 times (10%), with no instances of content not covered at all. Observers did not rate the third training session because it was a small make-up training (1 church, 2 committee members). For all instances of partially covered content (intervention and control trainings), a very small portion of the section was omitted.

The observers rated the CHAs’ teaching and facilitation skills across five other aspects of the training sessions on four-point response scales. These were staying on schedule, participants’ engagement, participants’ response to the session, participants’ level of excitement, and participants’ level of boredom. The total possible summed score for a session was 20, and across all six intervention training sessions, the possible summed score was 120. The CHAs attained a summed score of 116 (96.7%). The total possible summed score was 40 for the two control trainings observed. The CHAs attained a summed score of 32 (80.0%).

The observers’ responses to the seven open-ended questions at the intervention trainings indicated that the sessions had few problems, and those mentioned were minor, such as a brief delay. There were some late arrivals and early departures, but these were not disruptive. Observers gave specific suggestions for minor improvements (e.g., have each presenter wear a name tag). Observers described the CHAs as engaging the participants and setting the right tone for the day. The CHAs generally adhered to the schedule. The observers’ responses to the seven open-ended questions at control trainings indicated good participant engagement, excitement, and idea sharing. An additional short break after lunch was suggested to help maintain attention. There were some late arrivals and early departures, though none disrupted the session significantly. The observers noted that some sections seemed rushed, and there was some confusion among participants concerning the contents of the church program plan and budget. Such issues may be solved by closer adherence to the recommended time schedule.

CHA delivery and tracking of technical assistance to the intervention and the control (delayed intervention) churches

CHAs were responsible for making eight TA calls to church program coordinators and four TA calls to pastors for intervention and control churches. They used a communication protocol to guide the timing and frequency for contacting pastors and church committee coordinators through telephone, texts and emails. They used emails and texts to schedule TA calls and send reminders of upcoming calls. While CHAs provided technical assistance mainly by telephone, they occasionally provided TA in person when they had difficulty reaching the pastor or program coordinator.

Between November of 2015 and October of 2016 for a total of 288 calls to intervention program coordinators and 144 calls to intervention pastors (n=432 calls). One church dropped out of the study after three months, therefore the total TA calls to be delivered was reduced by 9, for a revised total of 423 assigned calls (mean=35.25 per month). The TA tracking data revealed that, of this revised total of 423, 390 calls (92%) were completed (mean=32.67 per month). Among the 33 calls that were not completed, the CHAs did not attempt 4.7% of these calls, while the remainder of church program coordinators and pastors could not be reached after two to five attempts. TA calls to pastors and church program coordinators ranged in length from 3 minutes to 32 minutes, with an average of 9 minutes.

Among the 18 control churches, CHAs provided TA calls between November of 2016 and October of 2017, for a total of 144 calls to church program coordinators and 72 calls to pastors (n=216 calls). The TA tracking data revealed that, of these 216 calls, 200 calls (93%) were completed (mean=16.67 per month). Among the 16 calls that were not completed, the CHAs did not attempt four because the pastor’s position was empty after his death, while the remainder of church program coordinators and pastors could not be reached after two to five attempts. TA calls to pastors and church program coordinators ranged in length from 2 minutes to 20 minutes, with an average of 7 minutes.

CHA participation and attrition

Both CHAs completed the Introductory and in-person CHA training, but one decided to take part in a leadership training opportunity and left before providing any church training sessions or technical assistance. The Community Outreach Coordinator assumed this CHA’s responsibilities. The second CHA participated in all CHA training, communicated with churches in their program planning stage, and provided technical assistance calls for the first 5 1/2 months. She experienced competing demands on her time from family responsibilities and challenges with data entry and reporting in the TA tracking database. She requested a reduced role of delivering the delayed intervention church training sessions only, which she did. Beginning in the eighth month, the Community Outreach Coordinator assumed all her other CHA responsibilities.

Discussion

Allowing sufficient time for development and pilot testing of CHA training materials and time for CHAs to practice and receive feedback on their church training presentation were key components in preparing them for effective delivery of training and technical assistance. Staff observations and participants’ feedback from our formative work revealed a few areas for improvement to the CHA training before delivery to the CHAs.

The CHAs indicated a high level of confidence in their skills and readiness on their evaluation forms. University staff observers noted that the CHAs completely covered 99% of the training content for intervention training and 90% for control training, indicating a high degree of fidelity to the training curriculum. These observers also rated the CHAs’ abilities highly across six church training sessions, suggesting good training skills and positive interaction with church participants. They noted few problems during the sessions. Church committee members gave positive qualitative reviews of the CHAs’ delivery of the sessions and of their characteristics as trainers, and they gave high ratings on their own level of readiness to implement the program after the training.

The monthly TA call notes allowed the staff to identify common barriers and successes among church coordinators and pastors during program implementation, as well as revealing CHA skills and knowledge areas in which the CHAs would benefit from content and skills review and practice. The booster sessions provided time for problem-solving, enhancement of CHAs’ content mastery, and reinforcement of CHAs’ communication and problem-solving skills through review, role play and feedback.

With the departure of one CHA and the reduced role of the other, the Community Outreach Coordinator was able to fill this void. She was a part-time, community-based employee, which allowed her to devote time to CHA responsibilities and thereby maintain program implementation fidelity (92% of TA calls were delivered), which highlighted the value of this part-time staff position. There we several issues that may have influenced attrition and role reduction. First, for the CHA who left after training, we have no reason to believe that this choice was anything other than a personal decision to pursue a different opportunity. For the CHA who reduced her role to training only, we perceived that computer access difficulties combined with increased family responsibilities made the time required for TA calls and data entry unmanageable. The number of repeated attempts to reach church coordinators and pastors was variable, depending on the specific churches assigned to each CHA, and the time to complete a TA call ranged from 3 to 32 minutes, with an average of 9 minutes. The time to complete the calls would average 2.6 hours per month, not including the time to make repeated attempts to reach the participants. Added to this was time for data entry, which would vary based on the CHAs’ comfort with the database and access to a computer and about 2 hours for each of three booster calls over the year. It was not possible to know in advance the amount of time that TA calls would require, and we have no way of knowing if the time required did or not match the CHAs’ expectations of their role. In Phase 2 of the study, a larger number of CHAs will allow us to explore these issues in more detail and assess contributors to CHAs’ success, expectations, and satisfaction with the CHA role.

While there is sufficient literature on the feasibility of CHAs as participants in community-based health programs, and some evidence of their effectiveness, less is known about what CHAs’ actual activities are during their work and the extent to which they carry out those activities. This paper provides details of recruitment, role expectations, specific responsibilities, training, and level of attainment in providing training and technical assistance to church representatives in the context of a novel application of CHAs, that being the widescale dissemination and implementation of an evidence-based health promotion program focused on the organizational rather than the individual or patient level.