Achieving Broad Participation in Congregational Health Surveys at African American and Latino Churches

Abstract

Congregation-based health program evaluations often rely on surveys, but little documentation is available regarding specific methods and challenges. Here we describe methods used to achieve acceptable response rates (73–79%) to a survey of HIV-related attitudes and behaviors in 2 African American and 3 Latino churches in high HIV prevalence communities in Los Angeles County. Survey participation was enhanced by: conducting survey sessions at church-based meetings (e.g., women’s Bible study) and after worship services; employing diverse survey staff; providing participation incentives for pastors, church coordinators, and survey participants; and working collaboratively and respectfully with congregational leaders. Achieving broad participation in church-based surveys on sensitive health topics is feasible when done collaboratively with congregational leaders and with a flexible protocol, which permits tailoring survey approaches to cultural and organizational contexts and leverages available resources appropriately.

A sizeable research literature describes how church-based health programming is important for reaching underserved, minority populations to reduce health disparities. Since the 1990s, over 400 articles have been published about congregation-based health programs and the actual or potential health effects of such activities (Campbell, Hudson, Resnicow et al. 2007, DeHaven, Hunter, Wilder et al. 2004). Further, there is increasing interest in how church-level interventions may be particularly effective because they operate at multiple levels of influence (Campbell et al. 2007, Winett, Anderson, Whiteley et al. 1999). However, little has been published about the methodological challenges involved in evaluating church-level programs, including lessons that could guide future studies.

Most outcome evaluations of church-based health interventions have relied on data collected from congregants, usually through church-based surveys that were self-administered (Berkley-Patton, Bowe-Thompson, Bradley-Ewing et al. 2010, Berkley-Patton, Moore, Berman et al. 2013, Davis, Bustamante, Brown et al. 1994, Dehaven, Ramos-Roman, Lee et al. 2011, Frank and Grubbs 2008, Lindley, Coleman, Gaddist et al. 2010, Resnicow, Jackson, Wang et al. 2001, Resnicow, Campbell, Carr et al. 2004). A smaller number of studies have used telephone interviews (Campbell, Demark-Wahnefried, Symons et al. 1999, Campbell, James, Hudson et al. 2004, Duan, Fox, Derose et al. 2000, Resnicow et al. 2004, Young and Stewart 2006) or in person interviews (Erwin, Spatz, Stotts et al. 1999).

Yet despite this fairly extensive literature on church-based surveys, there are surprisingly few implementation details such as sampling, recruitment and survey procedures, and response rates. For example, while articles routinely include the number of churches and total number of survey participants, few clearly indicate congregation type and the total number of congregants in each church (Berkley-Patton et al. 2010, Campbell et al. 1999, Derose, Hawes-Dawson, Fox et al. 2000, Duan et al. 2000), making it difficult to ascertain representativeness. Few report having selected participants randomly (Campbell et al. 1999); more commonly, studies rely on church liaisons to recruit and survey a convenience sample at each church (Dehaven et al. 2011, Resnicow et al. 2004). Moreover, the literature typically provides little information about survey design and implementation (e.g., exactly how and where self-administered surveys were done), or the number and spacing of survey waves. Thus, there is little information on how to achieve high response rates in church-based surveys of congregants.

In addition, few church-based studies have been conducted across congregations of diverse denominations, racial and ethnic makeups, and sizes; the church-based literature has been focused almost exclusively on African American churches (Campbell et al. 2007). The Los Angeles Mammography Promotion Program recruited 1,967 women ages 50–80 from 18 predominantly African American, 15 predominantly white, and 12 predominantly Latino churches through self-administered screener surveys. However, the proportion of age eligible women recruited per church ranged from 10% of age-eligible women in very large Latino churches (400+ age eligible women) to 75% of age-eligible women in small white churches (<100 age eligible) (Derose et al. 2000). Another multi-race/ethnicity study in Los Angeles (Davis et al. 1994) surveyed 943 women ages 21–89 from 23 churches participating in a cervical cancer education program, but no response rates were provided, nor were any details on how the surveys were conducted.

Finally, there is little information about surveying congregants on sensitive health topics (e.g., HIV, drug use, and sexual behavior). There is a growing literature about church-based HIV interventions with African-American churches (Berkley-Patton et al. 2010, Berkley-Patton, Moore, Hawes et al. 2012a, Berkley-Patton, Thompson, Martinez et al. 2012b, Coleman, Lindley, Annang et al. 2012, Francis, Lam, Cance et al. 2009, Griffith, Campbell, Allen et al. 2010, Hawes and Berkley-Patton 2012, Lindley et al. 2010, Moore, Onsomu, Timmons et al. 2012), all of which relied on interviews, focus groups, or self-administered questionnaires with limited numbers of clergy and/or individual congregants to inform and assess intervention design, acceptability, and feasibility. Data collection efforts focused primarily on a small subset of congregants and have tended to measure HIV-related knowledge and stigmatizing attitudes, not personal HIV risk factors (e.g., drug use and sexual behavior) and HIV status. Berkley-Patton and colleagues included an extensive HIV Behavioral Risk Assessment in their congregant surveys (Berkley-Patton et al. 2012a, Hawes and Berkley-Patton 2012); however, participants numbered 211 across 8 churches with reported average sizes of 200 members each, and thus represented a small minority of congregational members at each church.

In sum, a better understanding is needed of the most effective methods for facilitating broad survey participation within and across diverse congregations. In an era of growing public resistance to participation in surveys because of concerns about privacy (Morton, Cahill and Hartge 2006), researchers conducting church-based projects on sensitive health topics are likely to encounter challenges in surveying diverse congregants. Congregations tap into important social networks and thus offer great potential for health programming among populations less connected to healthcare services. However, since most previous church-based studies have used self-administered surveys, it is important to understand how feasible this approach is among populations of diverse literacy levels. Further, it is critical to identify the most promising solutions for maximizing response rates in church-based surveys to evaluate effectiveness of health interventions designed to reach the underserved.

This paper describes the procedures used to achieve high response rates in congregational surveys at African American and Latino churches regarding HIV-related attitudes and behaviors. Because of the gaps in the literature on congregational health surveys noted above, an in-depth discussion of our procedures and lessons learned can inform future evaluations of church-based programming.

METHODS

Study Overview

The purpose of the Facilitating Awareness to Increase Testing for HIV (FAITH) project is to raise awareness and reduce stigma about HIV, encourage congregants to get regular HIV testing, and develop strategies congregations and their leaders can adopt to assist in the community response to HIV. We pilot-tested a multi-faceted, congregation-based intervention to reduce HIV stigma and promote HIV testing among African Americans and Latinos that was developed using community-based participatory research (CBPR) and extensive qualitative and case study research (Bluthenthal, Palar, Mendel et al. 2012, Derose, Mendel, Kanouse et al. 2010, Derose, Mendel, Palar et al. 2011, Derose, Bogart, Kanouse et al. 2014). The multi-component intervention was implemented over approximately 3–4 months in each church and involved congregant HIV education workshops, peer leader workshops, an HIV sermon and imagined contact scenario, and church-based HIV testing events (see Derose et al. 2014 for a full description). All congregants 18 years and older were eligible to participate in intervention activities, but some activities reached a broader proportion of the congregation than did others. As a church-level intervention, baseline and follow-up surveys approximately 6 months apart of all adult congregants were used to assess preliminary effectiveness in reducing HIV-related stigma and increasing HIV testing.

Community Partners

A Community Advisory Board (CAB) composed of local leaders from religious and public health communities and chaired by 2 clergy consultants (CWO and MAM) guided all phases of the study. Pastors of each participating church designated a church coordinator to serve as a liaison between the project team and the congregation to plan, schedule, and implement all intervention and evaluation activities. The project survey team, which included several project outreach workers and a number of part-time survey assistants, was racially-ethnically diverse and possessed the language and cultural skills to work well with study congregations.

Survey Development and Piloting

We conducted church-wide, self-administered baseline and follow-up health surveys that focused on participants’ attitudes and behaviors related to health and HIV, including HIV risk factors (e.g., drug use, sexual behavior) and attitudes toward homosexuality, drug addiction and people with HIV. Prior to implementing the pilot intervention trial, we pilot-tested the survey questionnaire and procedures in English and Spanish with 68 congregants from 2 congregations similar to those in the pilot intervention, and feedback was incorporated into the survey development and implementation (Table 1). (Intervention activities were also piloted at these 2 churches prior to implementation). Survey content was vetted and approved by the CAB and the pastors and church coordinators of the participating congregations.

Table 1.

Exemplary Feedback from Baseline Survey Pilot Participants (n=68)

THEME	PARTICIPANT COMMENTS
Overall reactions to the survey	“Great survey.” “Don’t change a thing. It’s okay to ask those questions.” “Thanks for the reminders of how dangerous and important this topic is.”
Why a church-based survey is preferred	“The church feels safer and nobody would judge you.” “It’s easier to fill out information at church because many people go to church and many would not send it through the mail.” “You get more participation at church.” “I would enjoy doing the survey with my church family.” “For personal reasons, the church is the best place….I think it’s easier to do the survey at church, since it tends to be a peaceful and comfortable environment.” “I like the way the survey was done today at church because otherwise congregants wouldn’t do it.”
Why a mail or telephone survey would not work	“They [congregants] will forget about it and it will end up in the trash. They will not mail it back.” “They [congregants] might feel that it would be more confidential. But…they wouldn’t mail it back and wouldn’t answer it by phone.”
Why a few preferred a mail or telephone survey	“It’s more private by mail.” “By mail, because it’s private, it can be done in the comfort of your home without anyone looking at them.” “Some people might feel more comfortable answering a survey on their own time at home.” “It’s not too embarrassing or uncomfortable if you are talking to a complete stranger on the phone.”

Survey Participants and Recruitment Strategies

Over 1,300 adult participants were surveyed from 5 churches: 1 large (1200+ adult attendees) Latino Catholic, 2 small (<100 adult attendees) Latino Pentecostal, and 2 medium-sized (125–250 adult attendees) African American Baptist in high-HIV-prevalence communities. All adults (18+ years) who attended participating churches were invited to participate.

Based on feedback from the survey pilot and consultation with the CAB, the survey approach and methodology utilized the following strategies to encourage survey participation: (1) the project outreach team worked closely with the designated church coordinator to establish rapport and engage key clergy and lay leaders; 2) surveys were integrated into each church’s regular schedule of services, meetings, and events; (3) survey participants were not asked to provide their names or contact information (instead they were asked 4 questions that generated a unique survey code to link baseline and follow-up surveys); and 4) participation incentives were offered to the church, pastor, coordinator, and congregants. The church, pastor and coordinator each received honoraria ($500 each) for their assistance; congregants received a $20 Target gift card and a light meal for completing a survey.

Survey promotion period

During a minimum 2–4 week survey promotion period, advance information about the purpose and content of the surveys was disseminated broadly via church channels to encourage congregants to participate. Church staff and volunteers distributed recruitment flyers after worship services and during ministry meetings, placed flyers in the church foyer, put notices in church bulletins, and made announcements at church meetings.

Church-based survey administration

Survey sessions were conducted at times when congregants normally came to church, such as weekday evenings and weekends, commonly after worship services and/or during regularly scheduled ministry meetings or church events. Surveys were administered in group settings in classrooms, meeting rooms, or, in the smaller churches, the main sanctuary. Participants received an introductory letter and a project brochure that covered all the required elements of informed consent, which were then reviewed orally at the beginning of the survey session by the project survey administrators. Survey sessions were generally scheduled for 60 minutes each, although the survey took, on average, 30 minutes to complete. All survey materials were available in Spanish and English and trained survey staff provided one-on-one assistance to those who needed help completing the survey.

Attendance at the survey sessions varied greatly and some adaptations were made to accommodate specific congregation concerns. One African-American church was reluctant to host multiple survey sessions after Sunday services, and asked us to modify our procedures so that surveys and consent materials were distributed after Sunday worship services and individuals wishing to participate filled them out at home and returned them to survey administrators at the church the following 2 weeks. To accommodate a large Catholic church (1,200 adult congregants) with limited availability of meeting rooms, concurrent survey sessions were conducted in adjoining classrooms, and a queue was set up to organize congregants.

Survey sessions were repeated multiple times over a minimum of 3 weeks, but the number of sessions and the length of the field period varied based on the size of the congregation. At the large Catholic church, we conducted 4 baseline survey sessions during established ministry or group meeting times during weekday evenings and multiple survey sessions open to all adults congregants on 3 different weekend days; these sessions were spread over 2 months. At the small and medium-sized congregations, we had fewer sessions (usually 3 per church) and the timeframe was 3 to 4 weeks.

Coordinated management of the survey sessions

The survey sessions were managed by trained project staff; because of the sensitivity of some questions (e.g., drug use, sexual behavior, HIV status), church staff were not involved in the distribution and collection of the surveys and project staff placed completed surveys in lock boxes immediately after each survey session and delivered the surveys to the research office at the end of each day. For each survey session, a designated project staff member served as the lead survey administrator who ran the session, gave oral instructions, reviewed the informed consent materials, and answered participants’ questions before they began the survey. Survey assistants helped with logistics, such as greeting participants as they arrived, handing out consent materials and surveys, suggesting seating arrangements to minimize the likelihood that survey responses would be seen by others, answering questions about the survey, distributing incentives, and collecting completed survey materials.

The survey team brought child activity kits to each session with age-appropriate books and games and asked each church to provide some space within or near the survey room where children could use these items while their parents filled out the survey. We also provided a light meal to accompanying children (in addition to parent survey participants). During the baseline survey, project staff purchased most of the food from an outside vendor, but 2 churches catered their own survey sessions and were paid for each meal provided. Participating churches viewed the catering experience as a welcome fundraising opportunity, and church personnel involved in catering often became unofficial “recruiters” for the survey. Given this mutually beneficial experience, each church catered remaining survey sessions.

Ongoing survey feedback to the church

We provided ongoing (e.g., weekly) feedback about our progress during the survey period to church coordinators and sought their advice when we encountered any challenges such as low turnout at a survey session. After the survey was completed, we provided each church with a customized report, which had easy-to-read bar charts showing the church’s aggregate results on each survey item, and offered to meet to answer questions.

Survey Response Rates

The number of completed surveys collected at baseline vs. follow-up was almost identical: 1319 compared to 1320. However, as shown in Table 2, the overall estimated response rates increased from 73% at baseline to 79% at follow-up due to an observed drop in church attendance between baseline and follow-up at 3 churches (estimates provided by church leaders who track attendance). Church-specific survey response rates varied from 68% to 99% at baseline to 75% to 99% at follow-up. At a small Latino church and an African-American church, we surveyed nearly all adult attendees. At the large Latino Catholic church, it appeared that a large number of non-congregant community members (who were demographically different from the predominately Latino parish and indicated on their surveys that they had not attended the church in the past year) heard about the baseline survey and attended one afternoon survey session. Thus, a more conservative estimate of the baseline response rate in this parish would be 59% instead of 68%. We did not experience an influx of non-congregant community members at the follow-up survey sessions in this parish.

Table 2.

FAITH Project congregant health survey response rates at baseline and follow-up by church

Church	Predominant Ethnicity – Denomination	Baseline Survey Response Rate	Follow-up Survey Response Rate
A	African American – Baptist	71% (213/300)	86% (171/200)
B	African American – Baptist	99% (133/135)	99% (113/115)
C	Latino – Pentecostal	86% (77/90)	84% (59/70)
D	Latino – Pentecostal	99% (84/85)	98% (83/85)
E	Latino – Catholic	68% (812/1200)	75% (894/1200)
TOTAL		73% (1319/1810)	79% (1320/1670)

Response rates = number of completed surveys divided by estimated number of adult congregants (age 18 or older) that were attending services on average (per church leaders) at the time the baseline vs. follow-up health surveys were conducted.

Participant Reactions to the Survey

The survey team received feedback about the survey, which was documented in field notes. Participants commented that they thought the survey was good because the questions made them think about HIV and how it affects their community. Only 1 survey respondent provided negative feedback – an older congregant indicated that she was offended by some of the questions, though she opted to complete the survey. While other adults may have had concerns about surveys, none approached our staff nor the church coordinators to express them.

Survey Data Quality

The overall item-level nonresponse in the pre- and post- questionnaires was fairly low (under 2% per item) and appeared to be well within the typical range for self-administered surveys. There were a relatively small number of returned questionnaires (2 at baseline and 4 at follow-up) that were entirely blank or had only a couple of questions filled out, signaling that the respondent was unwilling or unable to participate. About 30 (2%) participants at baseline and 50 (4%) at follow-up required substantial assistance from the survey administrators to complete the questionnaire due to literacy issues.

DISCUSSION

Our results suggest that acceptable survey response rates (over 70%) in church-wide surveys can be achieved across diverse churches. Together with our CAB, we considered but ultimately ruled out alternative methods such as telephone surveys and in-person interviews, both because of the cost and because we anticipated resistance to providing personal information. These privacy concerns were validated by congregational participants in our baseline survey pretests and leaders at participating churches.

Instead, we opted for survey sessions with established ministry group meetings and after religious services. The costs involved participation incentives and survey staff labor; however, the costs would likely have been much higher had we attempted telephone or personal interview surveys. A mail survey would have been very difficult since many congregations do not have updated membership lists with mailing addresses; even if such lists exist, intensive follow-up with non-responders would have been required.

Many survey design and implementation challenges must be addressed to maximize congregants’ willingness and ability to participate. Below we discuss what worked well in our survey and describe remaining challenges and limitations.

A Collaborative Approach

Close collaboration between study staff and congregation leaders, similar to Berkeley-Patton’s TIPS study (Berkley-Patton et al. 2010), helped establish a constructive staff-congregation working relationship. Having the same, primarily bilingual, survey administrators and project team members involved over the course of the study helped establish rapport with clergy, lay leaders, and individual congregants, and allowed our team to become familiar with the culture and traditions of each congregation. This relationship-building helped congregants feel comfortable in asking for help and improved the survey team’s ability to handle unexpected problems that required on-the-spot problem-solving, such as a reserved meeting room being mistakenly assigned for another concurrent use, or a suspected minor attempting to take the survey. Further, the survey procedures were acceptable and successful because of the CBPR approach, which, as others have advocated, used culturally-matched research personnel, linguistically-appropriate materials, and a personalized recruitment process with endorsement from trusted congregational leaders (George, Duran and Norris 2014).

Although the support of congregational leaders is vital to success, appropriate roles will vary. Previous studies that relied on church liaisons to administer congregant surveys (Dehaven et al. 2011, Resnicow et al. 2004) explored dietary behavior and physical activity. Similar procedures for our study, which collected data on drug use, sexual behavior, and HIV status, would likely not been approved by our internal review board, and would have placed too large a burden on our church coordinators. Thus, each study will need to consider appropriate roles for all partners.

Church-Based Survey Development and Administration

We worked hard to make the survey instrument and administration as easy as possible for the congregations and survey participants, and assistance was provided to those who had trouble completing the survey on their own. Similar to other church-based studies (Berkley-Patton et al. 2010, Davis et al. 1994, Wiist and Flack 1990), scheduling survey sessions around regularly scheduled meetings proved to be easier for the congregations, and an effective means to recruit large groups of survey participants.

Understanding unique church characteristics and working closely with the pastor and church coordinators were crucial to identifying appropriate survey content and develop a customized survey plan for each church to achieve acceptable response rates. For example, the group survey sessions conducted after services or during ministry meetings worked extremely well at most churches, but not at one of the African-American churches. After brainstorming with church leaders, we implemented an alternative IRB-approved procedure (take-home surveys) that enabled us to boost our initial baseline response rate at this church from 20% to 71%.

Other logistical challenges included scheduling constraints with other church events, a shortage of large meeting spaces in which to conduct the survey sessions, and managing congregants during the survey sessions. On-going communication with the church coordinators and other key congregational leaders like church secretaries was important to secure space and make announcements about the survey sessions well in advance. The child activity kits and meals for children facilitated participation among parents.

Attempting to include all adult participants in the survey sessions regardless of literacy level was also important to achieving high participation rates and representing the congregation as a whole. However, variable literacy levels in the Latino congregations challenged the staffing and management of survey sessions. Our efforts to accommodate low-literate participants were largely successful, but some congregants may have opted not to come to a survey session because they could not read.

Conclusions

As church-based health promotion literature has proliferated, calls have been made to increase evaluation of these efforts and diversify them beyond African American churches and in particular to include Latino churches (Campbell et al. 2007, DeHaven et al. 2004). However, despite the importance of collaborative approaches such as CBPR to program design and evaluation (Campbell et al. 2007), the literature provides little guidance as to how to conduct church wide surveys of congregants in minority communities in ways that maximize response rates without overburdening congregational partners or compromising confidentiality of participants.

This study demonstrates that acceptable response rates can be achieved surveying congregants in churches of various denominations and sizes using self-administered surveys conducted in English and Spanish. An important key to successful implementation in this study was a lengthy collaborative planning process with our CAB and the leaders of participating congregations, who helped design culturally and organizationally appropriate implementation plans and actively promoted the survey. Fielding the survey in established ministry group meetings and after religious services was important in achieving a broad reach. Finally, employing experienced survey field staff – who were racially and ethnically diverse, bilingual, trained in protecting confidentiality, and comfortable with the content of the surveys and the church setting – was important in establishing trust with congregants. Applying these design elements may help future efforts to survey congregants on sensitive health topics, thereby improving the science and execution of faith-based health programs.