Recruitment strategies and challenges: Lessons learned from a coal ash and children’s health study

Abstract

The purpose of this paper is to describe the approaches and recruitment strategies of a study focused on the impact of coal fly ash on neurobehavioral performance among children living in proximity to coal-burning power plants. Challenges encountered with each recruitment approach are highlighted as well as solutions used to overcome those challenges and ultimately enroll children and one of their parents or guardians. To ensure participants were distributed throughout the study area, geographical information systems were used to guide recruitment and achieve the target sample size (N = 300). Several approaches were employed to recruit the number of needed participants, including “shoe leather” or door-to-door recruitment, placement of flyers and brochures in public spaces, mailings to targeted addresses, media announcements, and local government outreach. Since September 2015, 265 participants have been enrolled in the study using a combination of the described recruitment approaches. Even with a well-designed plan, it is important to re-examine strategies at every step to maximize recruitment efforts. Researcher flexibility in adapting to new strategies is vital in facilitating recruitment efforts, and the recruitment of participants in the study remains a dynamic and evolving process.

1 |. INTRODUCTION

Coal is one of the most abundant fossil fuels used for electricity production in the United States (Ambedkar, Nagarajan, & Jayanti, 2011; Yang et al., 2017). Coal-burning power plants account for approximately 27% of the US electricity generation (U.S. Energy Information Administration, 2019). When power plants burn coal, a by-product, referred to commonly as “coal ash,” is generated from the combustion process (Celik et al., 2007). Coal ash is composed of multiple constituents including fly ash, bottom ash, boiler slag, and flue gas desulfurization waste (U.S. Environmental Protection Agency [EPA], 2016). In 2017, approximately 111 million tons of coal ash were produced; of that only 71 million tons were recycled into products like concrete or wallboard (American Coal Ash Association, 2019). Ultimately, nonrecycled coal ash is stored in landfills and surface impoundments, which are often built near low-income residential communities (Earthjustice, n.d; U.S. Commission on Civil Rights, 2016). Coal ash that is stored in landfills or surface impoundments has the potential to escape as fugitive dust emissions and pollute nearby communities. In 2010, the EPA reported that “without fugitive dust controls, levels at nearby locations could exceed 35 μg/m³ established as the level of the 24-hr PM_2.5 National Ambient Air Quality Standard for fine particulate” (EPA, 2010).

1.1 |. Fly ash

The main constituent of coal ash is fly ash, which comprises 40–78% of coal ash (Lauer, Hower, Hsu-Kim, Taggart, & Vengosh, 2015; Liberda & Chen, 2013; Meawad, Bojinova, & Pelovski, 2010). Fly ash is a fine gray to tan silt, consisting of spherical particulate matter with diameters ≤10 μm. The geochemical properties of the coal used and the burning process determine the toxicity of fly ash (Brown, Jones, & BéruBé, 2011; el-Mogazi, Lisk, & Weinstein, 1988; Liu, Niu, Van Niekerk, Xue, & Zheng, 2008; Miller & Schobert, 1993; Spencer & Drake, 1987). Fly ash is composed mainly of silica, aluminum, iron, calcium, and oxygen, but particles also contain trace elements including heavy metals such as mercury, lead, cadmium, and arsenic (Bednar et al., 2013; Brown et al., 2011; Liberda & Chen, 2013; Patra, Rautray, Tripathy, & Nayak, 2012). The concentration of metals in fly ash may be two times greater than the concentration found in coal (Spencer & Drake, 1987).

1.2 |. Health effects from fly ash exposure

1.2.1 |. Occupational studies

Although fly ash contains toxic elements (Bednar et al., 2013; Brown et al., 2011; Fisher, McNeill, Prentice, & McFarland, 1983; Liberda & Chen, 2013; Patra et al., 2012), there is limited research on the health impacts of exposure to fly ash. Most researchers have conducted studies in workers exposed to fly ash. Occupational researchers report relationships between exposure to fly ash and increased arsenic and mercury levels in exposed workers (Zeneli, Sekovanic, Ajvazi, Kurti, & Daci, 2016), increased oxidative stress (Zeneli et al., 2016), increased frequencies of chromosomal aberrations (Bauman, & Horvat, 1981; Leonard, Deknudt, Leonard, & Decat, 1984), respiratory effects (Cho, Cho, Shrivastava, & Kapre, 1994), and earlier mortality from cancer (Bencko, Rames, Fabianova, Pesek & Jakubis, 2009).

1.2.2 |. Children and fly ash

Children are more at risk from the effects of fly ash compared to adults because they are not fully developed. Researchers have demonstrated that brains stop developing between the ages of 20–30 and lungs stop developing in people between 20- and 25- year old (American Lung Association, 2018; Johnson, Blum & Giedd, 2009). Compared with adults, children are more likely to breathe through their mouths and have larger lung surface area per body weight. Younger children are physically closer to ground level and, therefore, may breathe in more settled particles, they are more likely to engage in hand-to-mouth behaviors, and they are less likely to stop physical activity even if they experience distress (Committee on Environmental Health, 2004; Etzel, 1996; Lipsett, 1995; Schwartz, 2004).

Children who are chronically exposed to fly ash, and, therefore, to the neurotoxic metals within, are potentially at risk for developing neurobehavioral symptoms (Committee on Environmental Health, 2004; Lipsett, 1995). Childhood exposure to the metals found in fly ash has been associated with adverse health outcomes, including lung, kidney, and intestinal damage (cadmium) and increased potential of skin and internal cancers (arsenic; Agency for Toxic Substances & Disease Registry, 2008; Pershagen, 1981). Children exposed to lead are known to have impaired cognitive development and reduced intelligence quotient, and to be at increased risk for attention-deficient/hyperactivity disorder (ADHD), conduct disorder, impulsivity, and poor classroom behavior (American Academy of Pediatrics, 2005; Bellinger, 2004; Braun et al., 2008; Jusko et al., 2008; Needleman et al., 1979). Furthermore, mixed exposures to arsenic, cadmium, lead, and mercury may cause effects on renal and dopaminergic systems (de Burbure et al., 2006).

The number of researchers who assess associations between coal-burning power plant pollutants and health effects in community-based studies is small. Researchers have assessed the effects of coal-burning pollutants on children’s development in three studies. Tang et al. (2008) found that prenatal exposure to coal-burning pollutants such as lead was associated with decrements in developmental quotients in motor, language, and social areas (Tang et al., 2008). In a follow-up study, Tang et al. (2014) reported that the closure of the coal-burning plant was associated with higher developmental quotients (Tang et al., 2014). In research conducted by Sears and Zierold (2017), children living in proximity to a coal-burning power plant were more likely to report asthma, ADHD, gastrointestinal problems, and multiple sleep problems compared with children not living near a power plant (Sears & Zierold, 2017).

1.3 |. Recruitment in community studies

Effectively recruiting and retaining adults and children in community-based studies can be challenging (Lane, Armin, & Gordon, 2015). Efforts needed to make initial contact with participants, schedule and confirm appointments, ensure parents/guardians allow in-home visits and follow through with their appointments, and navigate differing busy schedules of participants are critical for successful recruitment. Due to these challenges and more, alternative recruitment strategies that differ from initially proposed methods may be developed when recruitment goals are unmet.

Different recruitment methods have been utilized in studies addressing various research questions, among different populations (Lane et al., 2015; Nolte, Shauver, & Chung, 2015; Williams et al., 2011). However, it is important for researchers to recognize the need for recruitment approaches to be germane to the population of interest, the research design and methodology, the location and demographic characteristics of participants, and the feasibility of designated approaches (Newington & Metcalfe, 2014; Nolte et al., 2015). For example, researchers using flyers and social media have shown that it is useful in reaching older and younger participants respectively, whereas community member inclusion and word of mouth may be particularly effective among African Americans or marginalized groups (Nolte et al., 2015; Whitaker, Stevelink, & Fear, 2017; Williams et al., 2011). Ultimately, recruitment efforts in this study were informed by known methods, community input, experiences from a previous coal ash pilot study, study design, and suitability of recruitment strategies (Whitaker et al., 2017; Williams et al., 2011; Zierold & Sears, 2015).

1.4 |. Setting for the research study

1.4.1 |. Power plants

Jefferson County, Kentucky is home to two power plants located approximately 10 miles apart. The Cane Run Generating Station was built in the 1950s and occupies over 500 acres along the Ohio River (Louisville Gas & Electric, 2013). This plant has five ponds, two of which stored coal ash. The main coal ash pond, which was opened in 1972, had a surface area of approximately 40 acres and sat 1,200 feet east of the Ohio River. This pond stored fly ash, bottom ash, and other materials (Martin, 2009). It received a high hazard rating by the EPA, indicating that collapse of the pond can lead to loss of life or major damage to dwellings, buildings, railroads, or important utilities (EPA, 2009). In 2015 the plant was refitted for natural gas. In 2017, the main ash pond was closed and capped. In addition to the capped pond, Cane Run has a large on-site ash landfill that opened in the early 1980s (Adnams, Stellato, & Harris, 2010) and it is now capped (Louisville Gas, Electric, & Kentucky Utilities, 2016). It was last estimated to have a height of over 500 feet and a surface area of 110 acres (Louisville Gas & Electric, n.d.-a).

The Mill Creek Generating Station sits along the Ohio River downstream from Cane Run. It began operating in the early 1970s, occupies over 500 acres (Louisville Gas & Electric, n.d.-b), and ranks as the third-largest plant in Kentucky. The plant’s main coal ash pond, which opened at the same time as the plant, (Bowers & Cormier, 2009) is in proximity to residential homes. The coal ash pond sits on over 40 acres and stores an estimated 6.4 million cubic yards of material (Bowers & Cormier, 2009; Zimmerman, 2017). It has been given a high hazard rating by the EPA. Mill Creek’s coal ash landfill opened in the 1980s and occupies over 200 acres with a height of almost 600 feet (Holm, 2016).

1.4.2 |. Study community

The study area includes multiple neighborhoods that span more than 12 zip codes within Jefferson County and Bullitt County, Kentucky. According to the 2015 American Community Survey the population of the study zip codes is predominately white (67%) with 27% black, 3% biracial, and 3% American Indian, Alaska Native, Asian, Native Hawaiian, or other race. Approximately 28% of children ages 5–14 years live below the poverty level (U.S. Census American Community Survey Data, 2015). The community is a mix of single standing homes, subdivisions of single-family homes, mobile home parks, and apartment complexes. The community is largely nontransient, and multigenerational homes are common.

1.5 |. Purpose

The purpose of this paper is to describe the overall study and discuss the recruitment approaches and strategies utilized to reach the 300 children needed for this research project. Specifically, we detail (a) the approaches used; (b) the challenges encountered; and (c) the solutions used to successfully recruit a large population of children and their parent or guardian. The described recruitment strategies can be utilized in various multidisciplinary research approaches to facilitate participant recruitment.

2 |. METHODS

2.1 |. Pilot work/context of the study

Before the current study, from 2011 to 2013, a pilot study involving four neighborhoods near one coal-burning power plant was conducted. This pilot study was a mixed-methods community-based study that involved working with community leaders to conduct focus groups and a survey to assess the health and well-being of adults and children (Sears & Zierold, 2017; Zierold & Sears, 2015). The focus groups (N = 26 participants) for the pilot study were conducted to determine the issues that the community felt were the most important regarding their exposure to coal ash and health. Among the participants in this pilot study, 69% (n = 18) had lived near the power plant for more than 20 years. Of the 20 participants with children, 85% (n = 17) reported that their children had distinct health problems (Zierold & Sears, 2015). The themes related to children that were identified from the focus groups included (a) feelings of helplessness and hopelessness about protecting their children; (b) fear of the unknown for what their children are being exposed to; (c) coal ash dust everywhere in their homes and continuous exposure; and (d) increased health problems among children.

Based on the focus group themes and scientific findings regarding the toxins that may be found in coal ash, researchers created a comprehensive questionnaire that was administered to 231 adults living near the coal-burning power plant and 170 nonexposed adults living in a community 60 miles away. Seventy-six percent of participants (n = 176) living near the coal-burning power plant indicated that they believed “coal ash was dangerous to children’s health.” The parents of 61 children aged 6–14-year old reported that 36% (n = 22) had been diagnosed with ADHD, 26% (n = 16) had learning difficulties, 38% (n = 23) had emotional and behavioral disorders, and 8% (n = 5) had developmental delays (Sears & Zierold, 2017).

Logistic regression analysis was used to compare parent-reported health outcomes among children living near the coal-burning power plant and the nonexposed comparison group of children (Sears & Zierold, 2017). Compared to a nonexposed group of similar age and socio-demographics, children living near the power plant were more likely to have ADHD (adjusted odds ratio = 3.61, 95% confidence interval [1.2–10.7]), controlling for age, gender, and second-hand smoke exposure (Sears & Zierold, 2017). These findings provided support for further investigation of the relationship between coal ash exposure and neurobehavioral symptoms in children.

2.2 |. Preparation for study recruitment

To ensure participants were distributed throughout the study area, geographical information systems (GIS) was used to design the recruitment plan. Allpress, Curry, Hanchette, Phillips, and Wilcosky, (2008) demonstrated the successful use of GIS-based methods for participant recruitment in an Illinois pesticide study. Similar, but enhanced, methods were used for this study. Initially, the study area was stratified using a series of five concentric buffer zones divided at 2-mile intervals from the midpoint between the two power plants. Using this midpoint between the two power plants as a reference point enables us to target and recruit participants who live in geographic proximity to both plants. After the buffer zones were created, they were stratified into wedge-shaped quadrants (See Figure 1).

FIGURE 1.

Study zone defined by buffers and quadrants. The buffers and quadrants are on the right side of the figure because this area represents the state of Kentucky, where the study is occurring. Across the Ohio River is the state of Indiana.

Zone 1 consists of the area within a 2-mile radius of the midpoint; Zone 5 represents the 8–10-mile buffer. Quadrants are labeled A–D. The combined buffer zone/quadrant designation is referred to as a sampling unit (SU). Thus, A2 refers to the SU in the northeast quadrant that lies within 2–4 miles of the midpoint. All the SUs are on the right half of Figure 1 because this area represents the study area, which is in the state of Kentucky. On the left side of the figure is the Ohio River and the state of Indiana. The use of both buffer zones and quadrants allows for stratification of our analysis results to optimally assess the effects of distance, wind patterns and exposure to multiple power plants.

2.3 |. Power calculation to determine the study sample size

Statistical power was calculated based on the exact version of the Cochran-Mantel-Haenszel test using 1,000 simulations. Prevalence data for the power calculations came from the described pilot work. Since the pilot study may have over-estimated the true percent in the population, we used more conservative estimates of between 20% and 30% for the prevalence of symptoms among exposed children. For the prevalence of symptoms in the nonexposed children, we assumed two prevalence values of 5% and 10%, which represent a range for neurobehavioral conditions in the United States, such as ADHD (7%) and behavioral conduct problems (4%; Perou et al., 2013).

Based on the assumed prevalence of exposure in each buffer zone and the assumed prevalence of neurobehavioral symptoms in exposed and nonexposed children, a total minimum sample of 60 children from each buffer zone is needed to achieve adequate (≥80%) power in the majority of scenarios. Therefore, a total sample of 300 children, with 60 in each buffer zone; comprising 15 in each of four SUs per buffer zone, is needed. US Census data indicate that there are 11,568 children aged 5–16 in the recruitment area, confirming an adequate population of children in the target area (U.S. Census Bureau, 2012). Because indoor exposure may differ by season related to participant behaviors such as opening windows and doors, weather patterns, and increased/decreased use of heating and cooling, we are assessing potential differences in the season by recruiting equal numbers of children during the four seasons.

2.4 |. Outcomes being assessed

For this study, multiple outcomes related to neurobehavioral performance and neurobehavioral symptoms are being assessed. Memory, concentration, and fine motor skills are measured utilizing a computer assessment system. Symptoms such as ADHD and depression are assessed by the Child Behavioral Checklist. The methods of exposure assessment, the collection of the health outcomes data, and the planned statistical analyses are being reported in a separate paper.

2.5 |. Inclusion and exclusion criteria of participants

For this study, one child and one parent or guardian per household are being recruited. To be able to participate, the family must have lived within one or more of the study zip codes for at least 2 years and have a child between the ages of 6 and 14 years. Parents have to provide consent for their child to participate in the study; allow the research team to conduct the home exposure assessment, health outcomes assessment, and pediatric health history; complete three questionnaires; and help collect the child’s fingernails and toenails. In addition, parents who smoke are required to smoke outdoors during the week the air samplers for particulate matter ≤10 μm are running inside the home.

For the child to participate in the study, he/she must assent to participate in the study, take a series of computer tests that measure neurobehavioral performance, allow researchers in his/her room to take dust samples, and agree to the collection of his/her toenails and fingernails. Children are excluded if they have a genetic disorder known to cause neurobehavioral problems, such as Fragile X syndrome or Down syndrome. If an adult participant agrees to participate, but the child does not assent, we do not enroll the child into the study.

Participants complete the study when a minimum of 150 mg of combined toenails and fingernails have been collected. Collection of the nails can take as little as 2 weeks to as long as several months. At the initial consent, the first sample of nails is collected from the child. This sample is placed into a plastic container labeled with the child’s participant ID. The research team takes the nail sample back to the laboratory, where the sample is weighed and the information is entered into a database. Parents or guardians are called every 2 weeks to check if nails have been clipped. If nails have been clipped, the research team picks up the nails from the participant’s mailbox and leaves an empty plastic container in the mailbox for future clippings. After each collection, the nails are weighed and the database is updated. This process is continued until the participant reaches a minimum of 150 mg of nails. The parent or guardian receives a $100 VISA gift card incentive for participating and the child receives a $25 VISA gift card.

2.6 |. Human subjects protection

All study protocols and documents have been approved by University of Louisville Human Subjects Protection Program (IRB#:14.1069) and the University of Alabama at Birmingham (IRB#:300003807), where the principal investigator (PI) of the grant is currently employed. IRB amendments were approved before each recruitment strategy or materials change.

2.7 |. Procedures

2.7.1 |. Recruitment, flyer development, and participant contact

Changes in the information included on recruitment flyers and methods of participant contact have occurred throughout the study. Figure 2 displays the changes in recruitment methods overtime, as well as the number of participants enrolled by season.

FIGURE 2.

Participant recruitment strategies overtime with enrollment per season. Gray shading indicates results from spring 2017 media campaign and new flyer

Before recruiting, the first flyers were created in the summer of 2015 (Figure 2). These original flyers only included the phone number for the PI’s office. Calls were responded to within 24 hr. Participants were more likely to agree to take part in the study if their call was answered immediately or returned within 4 hr. If the PI waited longer, participants were less likely to answer the phone or return the phone call.

In spring 2016, prepaid cellphones were purchased, and graduate student members of the research team began taking participant calls. New flyers were created with the same information as the original flyers as well as phone numbers for the PI’s prepaid cellphone and the study prepaid cellphone. The PI and the students maintained a daily call log to track when new calls were received and when participants were added to the schedule. Having more contact information became confusing to potential participants who were not sure which number to call or placed multiple calls to different phones. In addition, voicemails were not always received in a timely fashion when using the prepaid cellphones.

In spring 2017, Office of Communication and Marketing staff at the University of Louisville helped to strategize ways to improve communication with participants. Flyers were changed again to account for alternative methods of contact. The study prepaid cellphones were retired, and the PI’s personal cellphone was made the primary method of contact for participants. In addition, options for email contact and texting were added to the flyer. Furthermore, the flyer color was updated to bright neon green, and pictures of children were added. The neon green was selected so the flyers would stand out from other papers that were placed in doorways, under doormats, and in fences. Most participants contacted the PI by phone, followed by texting, and then email.

The recruitment items that are used in this study (flyers and letters) do not contain information about the study’s outcome (e.g., neurobehavioral performance and symptoms). The title on the recruitment materials is “Coal Ash and Children’s Health.” Potential participants calling to sign up for the study are not aware that this study is designed to assess neurobehavioral outcomes. However, when the PI describes the study, potential participants are informed that the study is assessing fly ash exposure and neurobehavioral outcomes. By not using the outcome measures in the title, we hope to reduce potential selection bias (discussed below).

2.7.2 |. Recruitment strategies

Various recruitment strategies have been implemented since September 2015 when the study began (Figure 3).

FIGURE 3.

Changes in recruitment flyer overtime

Method 1 - “Door–to–Door”

The first strategy utilized for recruitment involved pairs of research team members going door-to-door in a targeted SU to enroll participants. Using the US Census Data and GIS we determined subdivisions and streets in the SUs with the highest concentration of children. Flyers and brochures were distributed in these locations by walking door-to-door in neighborhoods and talking with community members who were outside. Initially, two community leaders, who were long-term residents and active in environmental issues in their neighborhoods and the community in general, assisted with recruitment and talking with potential participants. Research has shown that involving community members in community-based research to administer data collection instruments increases the rate of recruitment because the community members have standing in the community and are trusted (Schultz et al., 2005; Schulz et al., 1998; Tajik & Minkler, 2006; Thompson, 1999). Community-based research that involves active community members has lowered attrition, increased compliance, improved accuracy of reported information, enhanced study relevance, and greater applicability and usability in the settings where community research occurs (Corburn, 2002; Green & Mercer, 2001; Jordan, Lee, & Shapiro, 2000; Lynn, 2000; Zierold & Sears, 2015).

Flyers were typically positioned in between doors or under doormats. Attempts were made to place flyers in newspaper boxes, located under the mailbox, but the wind would frequently blow them out of the box and down the street. Flyers were not distributed to homes that had “no trespassing,” “no soliciting,” or “beware of dog” signs. If fences were present that prevented access to the front door, flyers were rolled up and placed in the gate of the fence, to ensure visibility to the resident. Flyers were not distributed to homes that did not have a clearly visible front door and lacked other practical means for flyer placement.

Initially, the research team distributed flyers three to four times a week, including Sundays. However, after several months of this “shoe-leather” method, a pattern emerged suggesting that participants were more likely to respond to the flyers if they received them early in the week, so foot recruitment was limited to Mondays and Tuesdays. From September 1, 2015 to January 31, 2016, 28 participants were recruited from this door-to-door recruitment method. It is unknown how many participants were recruited from this method after January 2016, because new methods of recruitment were introduced.

Method 2 – “Public Spaces”

The second strategy that the research team used to recruit participants involved distributing flyers in public spaces. The first public places targeted for recruiting were schools located in the neighborhoods of our study area. At four different schools, flyers were placed on the windshields of parked cars in the faculty/staff parking lots during multiple occasions over the course of several weeks. We were surprised that no participants were recruited through this method. However, we surmised that since Jefferson County, Kentucky does not have “neighborhood schools,” children whose parents were teachers or staff may not have lived in the study area.

In addition to school parking lots, flyers were distributed in parking lots of major brand-named stores located in the area. Main grocery stores and department stores were chosen based on their location within the study area. At the large chain neighborhood stores, flyers were placed on windshields of parked cars. In addition, research team members distributed flyers and talked with people coming out of the stores or getting into their cars. Stores were visited two to three times on the weekends when stores were most crowded. Again, we were surprised that no participants were recruited through this method.

Method 3 – “Mailings”

In February 2016, we added a third method of recruitment. Since the “shoe-leather methods” were extremely time intensive and affected by season, we began mailing letters and flyers to potential participants within the SUs. A list of addresses was purchased from two separate companies: LeadsPlease.com and InfoUSA Both companies sell mailing lists that can be customized. In our case, study zip codes and the ages of children in our study were used to customize the address lists before purchase. In total, 4,088 addresses were purchased from LeadsPlease.-com, and 12,511 were purchased from InfoUSA. Once the database of lists was created, we used GIS to determine the location of addresses in SUs. Mailings were targeted based on where participants were needed in our SUs. This recruitment method is the prime method currently being used. Potential participants in the SU are mailed a letter on university letterhead providing an overview of the study and the study flyer. Mailings are completed 3–6 times a year. The timing and number of mailings are dependent on the success of the prior mailing. For example, if one mailing fills our intake schedule for 3 months, a second mailing would be conducted 2 months later. We have found that at times, residents respond after receiving multiple mailings. On average, we recruit 10–20 participants per one thousand letters mailed.

One of the major challenges posed by the mailing recruitment method was lack of interest after flyers were mailed. On two occasions, flyers and letters were mailed, but no participants were recruited. This could have been partly due to residents receiving information on Mother’s Day weekend, which was also the weekend of graduations from several local universities. To avoid such situations from recurring, subsequent mail dispatch did not occur during holidays or celebratory periods. In addition, the study team recognized that responses from mailings were improved if residents received the flyers and letters in the beginning of the week. Hence, mailings were always scheduled to go out on Fridays, so that potential participants received the information (Monday or Tuesday).

There are two plausible explanations for the response to mailings being low. The first is that the letters are sent “bulk mail” and not first-class, so our letters may be thought of as “junk mail.” A second is that several other studies are ongoing in sections of Jefferson County, and residents may receive numerous opportunities to participate in various research studies and limit how many they become involved in.

Method 4 – “Media”

In spring 2017, recruitment via a local media campaign and Facebook ads were added. The Office of Communication and Marketing at the University of Louisville created a media campaign and a corresponding website. The campaign led to a press release that resulted in several newspapers, a national radio station, and several local TV stations highlighting the study. This media campaign led to an influx of participants. The PI received so many calls, voicemails, and text messages, that it became too overwhelming to rapidly enroll interested participants in the study. Though this influx could be viewed as an advantage, it required additional personnel, resources, and time. To meet this challenge, an additional staff member was hired and more study equipment was ordered. This campaign resulted in 55 recruited participants.

In addition to the radio, TV, and newspaper spots, we utilized a Facebook ad designed to be displayed in the geographical area related to our study. This ad was developed in consultation with the University’s Office of Communication and Marketing. The Facebook ad ran for 2 weeks in April at a cost of $50. After 1 week, we had 61 clicks on the ad (35 women, 26 men). An example of the ad is shown in Figure 4.

FIGURE 4.

Facebook Ad used for recruitment.

In addition to the media campaign, unbeknownst to the research team, several active participants posted advertisements for the study on Facebook pages of local community groups. The research team did not have access to these Facebook pages, as they were closed groups, but two participants informed us about these pages. We were unable to track the number of participants enrolled from these ads, but several potential participants who contacted the PI indicated they had seen the Facebook pages.

Method 5 – “Local Government Outreach”

At the same time the local media campaign was introduced, emails were sent to city council members throughout the study area. Council members were asked to place information about the study in their weekly or bimonthly e-newsletters. Four council members obliged, yielding four participants.

Method 6 – “Snowballing”

Snowballing has been used throughout the study. Snowball sampling is a nonprobability sampling method used in survey research that relies on participants to notify other potential participants of the study (Blair & Blair, 2015; Fink, 2003). This method is commonly used to target hard-to-reach populations. In our study, we used snowballing two ways. First, the PI asked current participants if they knew anyone who might be interested in participating in the study. Participants positively responding to this inquiry were provided with additional flyers and were encouraged to distribute the flyers to anyone they knew who might be interested in the study. Second, current participants asked if we needed more participants or if their family members or friends could participate. If the family members did not live in the home of the current participant, we offered to provide the participant with extra flyers to distribute. Overall, snowball sampling was not as successful as we had hoped. Although participants initially seemed excited at the opportunity to recruit, we recruited only a few participants with this method, most of whom were participant family members.

2.7.3 |. Costs associated with recruitment methods

Costs of the recruitment methods varied. In fall of 2015, when we were only doing door-to-door recruitment, our monetary expenses were approximately $158 for 300 printed colored flyers on white, low-cost paper. However, due to the long hours the research team spent recruiting in neighborhoods, the cost of time and effort were high. The PI would canvas the neighborhoods with two to four graduate assistants. The time spent in the neighborhoods ranged from two to 4 hours in the area selected for recruitment. At times, the graduate assistants would go alone. If graduate assistants went for 4 hours twice a week, this left 12 hr of their time for other study assignments (graduate students work 20 hr per week for stipend and payment of tuition). It should be noted that we spent several days a week setting up equipment for the study, taking down equipment for the study, consenting participants, and conducting the neurobehavioral testing that was associated with this study; hence graduate assistant time was exhausted. However, many of the students were interested in the study, completed their thesis or dissertation on the study, and hence worked more than their allotted graduate assistant time.

Our overall monetary expenses increased when we added mailing information as a recruitment method. In addition to the cost of flyers, there was the cost of printed university envelopes and the cost of printing letters to the community on university letterhead as required. Our costs for 1,000 letters and 1,000 envelopes were approximately $400. To reduce costs associated with the flyers, we stopped printing the flyers in color. Instead, we copied the flyers on bright neon green paper using the only black font. For 1,000 such flyers, the cost was approximately $142. In the beginning of the mailing effort, the research team folded the materials and stuffed the envelopes, which was a time commitment for the PI and graduate assistants. After a year, this process was outsourced. Using bulk mail rates, the cost of the postage was approximately $104 for 1,000 pieces of mail. The cost of folding the materials, stuffing the envelops, and mail processing (data import, the process through the National Change of Address) was approximately $210.

The other methods of recruitment (local government outreach and Facebook) cost less than mailings and door-to-door recruitment and required less personnel time. With the Facebook ads, we were able to set the amount of money we wished to spend initially at $50.

Media coverage (such as TV spots and newspaper stories) cost the research team nothing in terms of money. The PI and graduate students did spend time being interviewed and showing equipment. However, this method provided the most participants, so the payoff exceeded the expense of time.

3 |. DISCUSSION

This ongoing study is the first carried out to investigate the impact of fly ash on a community of chronically exposed children. The health impact of fly ash is unknown, but the potential threat is immense. Our success in recruiting 265 participants to date, representing 88% of the targeted sample size, suggests that implementing multiple recruitment strategies facilitates continual participation from the community.

As of February 2019, the study has entered its fifth year and with the continued adaptation of its recruitment methods, we are poised to obtain the required number of participants. As times change, so too must recruitment methods; what may have worked in the past may not be as effective now. Traditional approaches such as door-to-door and telephone recruiting are no longer adequate in the face of changing social and technological structures. Physical mail has maintained its relevance thus far, but electronic media is growing increasingly prominent as a recruitment source. Researchers conducting community studies might benefit from employing media campaigns early on and intermittently throughout the study period.

This study has presented some unique challenges to recruitment. First, many rumors about the study have surfaced. Examples of common rumors include the study involving taking blood specimens from children, giving injections to children, or equipment that puts “something” into the air. These rumors may have prevented some people from calling or being willing to participate.

In addition to the rumors, as previously noted there are multiple research studies occurring in some of our target zip codes. In some cases, potential participants may be overwhelmed with recruitment materials and ignore them altogether. Past research has also resulted in a lack of trust of the University. Several community members in our study indicated that they had participated in studies but never heard the outcome and/or did not receive their results. For our study, we try to return neurobehavioral results within 3 months. Results from the exposure assessment will be provided at the end of the study in the form of community meetings and mailed reports to participants.

Our recruitment strategies have several limitations and barriers that need to be considered by others planning a similar dynamic approach. First, employing multiple recruitment strategies post hoc required continuously updating our proposal and getting IRB approval for each amendment. To prevent a delay in recruitment, researchers should consider getting approval for multiple recruitment methods in the initial IRB proposal.

Second, our study covers a large geographical area, which includes more than one neighborhood, each presenting unique challenges. In the pilot phase of our study we developed relationships with well-known community leaders and relied on these relationships to help with study recruitment. When the larger study expanded to other neighborhoods, we had to adapt our community-based approach because the community leaders did not have the same level of rapport with other neighborhoods. Identifying other gatekeepers (i.e., local politicians, church leaders) or communication channels (i.e., community social media groups) in each unique community while developing the study protocol could help identify the most effective recruitment strategies from the study onset.

Third, the mailing addresses were obtained from several vendors. The methods these vendors use to obtain resident information may result in a nonrandom sample from each neighborhood. However, by using GIS and door-to-door methods we were able to monitor where the mailing addresses were located and were able to access residents in the neighborhoods through other recruitment methods.

Fourth, we were unable to track the number of participants recruited by each method. Recruitment methods overlapped and potential participants were not asked how they heard about the study. The observations described in this manuscript, such as best days of the week to mail out information, were based on our experiences during the study. A fifth limitation is that during the study, we did not record the number of contacts a potential participant received before contacting us about the study. In many cases, flyers were mailed and door-to-door contact occurred where additional participants were needed in a particular SU.

A final limitation of our study is the potential for selection bias. Parents or guardians who may have had a child with health conditions, or who may have been more knowledgeable about coal ash, may have been more likely to want to participate in the study. In an attempt to reduce selection bias, our recruitment materials did not provide information about the health outcomes we were studying.

Our recruitment strategies had several strengths. By using door-to-door recruitment and mailings we were able to target specific geographical areas, which will allow us to evaluate proximity to power plants with coal-combustion storage facilities and the impact on exposure and children’s health. It would be difficult to ensure that this geospatial analysis had sufficient statistical power if recruitment was conducted from a specific site, such as a health clinic. In addition, using multiple communication channels (i.e., print, radio, social media) increased the likelihood that residents were aware of the study and trusted the information source. By providing multiple communication channels (i.e., email, phone, and text) for residents to seek out more study information, we also allowed them to respond in the modality in which they felt most comfortable.

The recruitment challenges we encountered have been overcome by a continuous re-evaluation of the challenges inherent within each recruitment strategy and finding solutions to those challenges. Activities were modified accordingly to address the situations presented at the time. Researcher flexibility in adapting to new strategies is vital in facilitating recruitment efforts, and as such, the recruitment of participants in this study remains a dynamic and evolving process.