Feasibility of Recruiting in Prisons During a Randomized Controlled Trial with People with Serious Mental Illness

Abstract

Background:

Successful participant recruitment is vital to the feasibly of intervention research. In the behavioral and social sciences, intervention researchers face a myriad of recruitment barriers, many of which stem from working in real-world settings and among hard-to-access populations. Optimizing recruitment efforts requires being intentional about study planning and resource allocation, carefully documenting the outcomes of recruitment efforts, and developing and implementing procedures and strategies to overcome anticipated recruitment barriers.

Methods:

The current paper presents recruitment flowcharts to illustrate (a) the multistep recruitment process and (b) the points of potential participant attrition during recruitment from a two-phase group-based intervention study conducted among individuals with serious mental illness incarcerated in a state prison system in the U.S. Additionally, qualitative methods are used to examine strategies employed during the study to support recruitment efforts.

Results:

Despite challenges, this study was able to achieve recruitment goals. Analyses found the majority of potential participant attrition occurred prior to informed consent, highlighting the need for studies to track recruitment efforts in more detail than is currently recommended by commonly used guidelines. Strategies to optimize recruitment efforts included maximizing recruiter availability, developing a responsive communication approach, demonstrating respect for facility procedures and operations, and ensuring peak preparedness.

Conclusion:

Careful documentation of recruitment efforts and the early deployment of recruitment strategies is vital to the feasibility of intervention studies conducted in real-world settings with hard-to-access populations. The publication of recruitment procedures and outcomes can help future researchers anticipate recruitment challenges and inform recruitment goals, timelines, and strategies.

Background

Individuals with serious mental illness (SMI) are overrepresented in the criminal justice system. It is estimated that up to 38% of individuals in jails and prisons have an SMI^1,2 and that once involved in the criminal justice system, these individuals recidivate more often and more quickly than individuals in the general population.^3,4 Thus, programming to improve mental health and criminal justice outcomes among this population is sorely needed.^5–7 Rigorous research methods, such as randomized controlled trials (RCTs), are critical to developing such programming. Unfortunately, RCTs conducted among justice-involved individuals with SMI have historically struggled to achieve recruitment goals.^8,9

Previous literature has documented the general challenges of recruiting participants into clinical trials, including misconceptions of randomization and treatment allocation, distrust of researchers, the time burden of participating in research, and structural hinderances such as inadequate transportation.^10,11 Given these challenges, it is not surprising that nearly 1 in 5 clinical trials are terminated early due to recruitment failure¹² and that, even among trials that do meet recruitment goals, over 50% fail to do so within originally proposed timeframes.¹⁰

Intervention research conducted among individuals with SMI in prison settings face an even greater number barriers to successful recruitment. These include the mental health symptomology and stigma associated with mental illness^13–17 as well as rigid facility protocols and high staff turnover that make accessing potential participants difficult.^9,18–23 This confluence of recruitment challenges threatens the feasibility of conducting urgently needed research that could improve health and criminal justice outcomes among individuals with SMI who are incarcerated.

General strategies to overcome common recruitment barriers have been identified in the clinical trials literature.^11,12 While some of these are applicable to the behavioral and social sciences, others may be difficult or impossible to implement in these research settings. For example, in the context of criminal justice settings, extensive site screening prior to site selection is not practical when potential sites are limited in number, spread far apart geographically, or when research site protocols dictate permissible types of study operations and materials.²¹ Additionally, early participant outreach efforts may not be possible when working with populations that are difficult to locate, transient, or in the case of incarcerated individuals, whose accessibility is heavily restricted.²²

Therefore, the documentation and publication of procedures and strategies to support recruitment efforts is needed to help future behavioral and social science researchers (a) determine the number of potential participants and study sites required to obtain study samples, (b) plan realistic and achievable recruitment timelines, (c) adequately allocate study resources to support recruitment efforts, and (d) anticipate study-specific recruitment barriers for clinical trials conducted in real-world practice settings.^11,24,25

The current paper fills a gap in the literature by examining the recruitment efforts during a study of a behavioral health intervention conducted in a prison setting among individuals with SMI. The primary goals of our analyses were to (a) examine the attrition of potential participants at each step of the recruitment process, (b) identify the potential participant pool size and timeframe required to achieve recruitment goals, and (c) explore the procedures and strategies used to support successful recruitment efforts.

Methods

Study design

This paper draws from a study of a 14-week, group-based, cognitive behavioral therapy intervention (i.e., Thinking for a Change²⁶) designed to target risk factors for recidivism among justice-involved individuals. This intervention was delivered in a closed-group format using a novel targeted service delivery approach that encompasses a set of service delivery strategies which address the specific treatment needs of individuals with SMI.²⁷ The two phases of this study included (a) an open trial phase where three cycles of the intervention were delivered to finalize intervention materials and protocols and (b) a small-scale RCT that involved four intervention cycles of the newly developed intervention. The closed-group format of the intervention meant that all participants for each cycle had to be recruited before each of the seven intervention cycles could begin.

Study setting

Participants were recruited from three state prison facilities (two men’s and one women’s prison) in a state prison system located in the southeastern United States between October 2017 and March 2020. The two prison facilities that incarcerated men included: one maximum security facility that incarcerated over 1,000 individuals, located within an hour’s drive of the university; and one medium security facility which incarcerated over 600 individuals located one and a half hour’s drive from the university. The women’s prison was a multilevel custody facility that incarcerated over 1,700 individuals and was located within an hour’s drive from the university.

Sample

In order to be eligible for participation in this study, individuals needed to (a) be aged 18 years or older; (b) have a mental health diagnosis of schizophrenia, schizoaffective disorder, psychotic disorder, bipolar disorder, or major depressive disorder; (c) had moderate to high criminogenic risk levels as determined by the Level of Service Inventory;²⁸ and (d) had at least one year remaining on their prison sentence at the time of the screening interview. Study exclusion criteria included (a) an intellectual or developmental disability; (b) assault precautions or other restrictions that would preclude the person from being in group gathering spaces; and (c) participation in Thinking for a Change in the previous year during the open trial phase and the last six months during the RCT phase.

Each recruitment cycle began a month prior to the initiation of a new intervention cycle and continued until the necessary number of individuals were recruited. The study team ultimately enrolled 32 participants over the three cycles of the open trial phase. During the open trial, every participant was offered the intervention. Over the four cycles of the RCT phase, 100 participants were enrolled and randomly assigned to receive either the study intervention or standard prison programming.

The study recruitment process

The study team developed a participant tracking form that was used to monitor participant attrition and retention through each of the seven steps of the recruitment and enrollment process. A description of each of these seven recruitment steps can be found in Table 1. Data from the participant tracking forms were used to create the recruitment flow charts presented in Figure 1.

Table 1.

The Seven Steps of the Study’s Recruitment Process

Step	Description
1. Identify potential participants	• Correctional staff at the facility where the study intervention is being delivered provided the research team with a list of all potentially eligible participants at the facility. After receiving the list of all potentially eligible participants, study staff completed all recruitment activities.
2. Invite potential participants	• Study staff invited potentially eligible individuals to meet with study staff in a private setting in the prison to learn more about the study.
3. Initial approach of potential participants	• For participants who accepted the invitation to learn about the study, study staff met with them in private locations in the prison and provided an institutional review board approved study flyer. During this meeting the study staff reviewed the information in the study flyer and emphasized that study participation was strictly voluntary, that potential participants’ decisions about participation had no bearing on their legal system involvement, and that they could end their participation at any time without negative consequences.
4. Obtain informed consent	• Study staff completed informed consent immediately after completing recruitment activities with all potential participants who expressed interest in learning more about the study.
5. Conduct the screening interview	• Study staff completed a screening interview immediately following informed consent with all potential participants who consented to participation in order to determine who was eligible for participation.
6. Determination of study eligibility	• The screening interview determined whether participants were eligible to participate. Study staff notified potential participants of their eligibility at the end of screening interviews.
7. Enroll eligible participants	• Participants who were found eligible for the study and agreed to participate were enrolled into the study. In the open trial phase, all eligible individuals received the research intervention. In the RCT phase of the study, eligible individuals were randomized into either the (study intervention) or control (standard prison treatment and programming) condition.

Figure 1.

Recruitment Flow Charts for the Open Trial and RCT Phases of the Study.

Note. T4C = Thinking for Change Experimental Group. TAU = Treatment as Usual Control Group

*Details not reported due to small cell size

Data collection

The study team developed the recruitment tracking form through an iterative process in the open trial phase. This recruitment tracking form incorporated the guidelines established by the Consolidated Standards of Reporting Trials, which recommends tracking participant flow during enrollment, randomization, treatment allocations, follow-up, and analysis.²⁹ To more fully understand the flow of potential participants during recruitment, the study team included addition steps on the recruitment tracking form that were derived from the study’s research protocols. The inclusion of these additional steps provided a more detailed picture of the recruitment process and helped the research team identify points in the recruitment process where potential participant attrition occurred.

Data analysis

The recruitment tracking forms were used to construct recruitment flow charts for both phases of the study (Figure 1). While our participant tracking form documented attrition at each step of the recruitment and enrollment process, data was not collected on the specific reasons for attrition. Rather, in order to protect potential participants’ privacy and autonomy, study staff classified attrition into broad categories as illustrated in Figure 1.

In addition to tracking participant flow, inductive qualitative research methods were used to identify and categorize the strategies that study staff used to optimize recruitment efforts. The lead author conducted this analysis under the supervision of the study PI, who has extensive experience and expertise conducting qualitative research. For the purposes of this paper, we define the optimization of recruitment efforts as both (a) successfully recruiting the study sample within prespecified timeframes and (b) minimizing barriers in the recruitment process that might impede potentially interested participants from participating in the study. This analysis triangulated data drawn from a number of different sources including project meeting notes, study memos, site correspondences, the manual of operating procedures, and recruitment tracking forms.

The first step of the qualitative analysis engaged inductive line-by-line coding of all the project materials that contained details related to study staff recruitment activities and procedures. The coded text was then compiled by the lead author into a single document and a second round of inductive descriptive line-by-line coding was conducted. The goal of this second round of coding was to identify and develop descriptive labels for all study activities and procedures related to recruitment efforts. Then, a third round of focused coding was conducted. First, relational coding techniques were used during focused coding to organize the data into an initial list of recruitment strategies. Then, discriminant coding techniques were used to triangulate the emerging recruitment strategies identified in this analysis with those in the publish literature.^{e.g.,11,13–17,24,25} After focused coding was complete, member checking was engaged with the study team in order to seek feedback on the accuracy and completeness of the analysis to date. The results of the member checking were integrated into a final round of analytic coding, where the emerging categories where organized into the final set of recruitment strategies described in the results.

Results

Potential participation attrition and refusal rates

The recruitment flow charts in Figure 1 show that the study achieved its recruitment goals, enrolling 132 individuals in 33 recruitment days across seven intervention cycles.

Figure 1 also shows that 33.3% of all invited potential participants were enrolled during open trial phase, and 54.1% of invited potential participants were enrolled in the RCT phase. Therefore, 47.0% of invited potential participants were enrolled across both study phases. The overall refusal rate for this study, calculated as the number of invited individuals who did not enroll in the study across both study phases, was 53.0% (i.e., Out of 281 total invited potential participants, 149 potential participants were not enrolled in the study).

Additionally, Figure 1 illustrates that most of the attrition in recruitment occurred when potential participants were initially invited to consider participating in this study and following the initial approach. Only 9 individuals out of the 141 who completed informed consent (6.4%) failed to be enrolled in the study. This means that while only 47.0% of potentially eligible participants were enrolled in the study, 93.6% of individuals who completed informed consent were enrolled.

Strategies to support participant recruitment

Qualitative analysis of the study procedures and activities identified a number of strategies that the study team used to support recruitment efforts. These recruitment strategies centered on being responsive to a prison environment governed by restrictive protocols and limited in the time and space available to accommodate study recruitment. Specifically, these strategies fall under the categories of maximizing recruiter availability, developing a responsive communication approach, demonstrating respect for facility procedures and operations, and ensuring peak preparedness. A description of each strategy, including examples of how each strategy was deployed in this study, is presented below.

Maximizing recruiter availability.

Given the distance that staff had to travel to get to facilities (i.e., up to 3 hours round trip), the short recruitment windows for each intervention cycle (2–3 weeks), and the limited availability of time and space within each prison facility for recruitment activities, the study team found that recruitment efforts were optimized by having as many staff members as possible available to engage in recruitment.

To this end, the study team maximized the number of study staff who were (a) trained to conduct recruitment activities as per study protocols and (b) approved to enter the prisons per facility protocols. Specific strategies used to maximize recruiter availability included mapping out all of the recruitment windows before each study cycle in order to synchronize staff availability with recruitment windows. Additionally, the study team trained every eligible staff member in recruitment activities, regardless of their primary role in the study (i.e., interventionist, project manager, study PI, research assistants). These trainings were held on a rolling basis to account for study staff turnover and other issues that impacted availability during recruitment.

Study staff were also required to complete criminal background checks and prison-specific trainings, which needed to be renewed on a regular basis, before entering a facility. Therefore, all study team members who were trained to engage in recruitment activities were asked to complete all facility-specific clearance requirements, regardless of whether or not their primary study duties required them to enter the facilities. This ensured that the study team had an ample number of staff available to recruit at any given time, including on short notice.

Developing a responsive communication approach.

Gaining permission to conduct research in each prison facility required communicating with multiple stakeholders prior to each intervention cycle, even when multiple cycles occurred consecutively within the same facility. Therefore, the study team employed a number of strategies to promote early and consistent communication with facility staff prior to beginning recruitment and throughout the recruitment process.

For example, the study team identified the appropriate chain of command for communication in each facility and then assigned one study staff person to act as the central point of contact. This streamlined communication and minimized the burden placed on facilities. In addition, the study team worked to identify and establish a “project champion” at each prison facility. This prison staff person would often act as a liaison to the study team, assisting in navigating the prison protocols and procedures necessary to gain entry to, and work within, the facility.

In order to minimize delays in recruitment activities, the study team also found it necessary to begin planning for recruitment three to four months prior to the initiation of recruitment activities for each study cycle. Finally, the prison and facility staff expended time and energy to make accommodations that supported recruitment efforts. Therefore, it was important for study staff to communicate regularly with prison staff about schedules and any anticipated changes or potential delays in planned recruitment activities.

Demonstrating respect for facility procedures and operation.

Prisons employ an array of practices and rules to ensure the safety of everyone in the facility. These rules can vary between facilities within the same prison system due to differences in custody levels, administration, and other factors. The prior work of study staff in correctional settings sensitized the study team to the importance of ensuring that study protocols were concordant with the security protocols at each prison facility. As such, the study team engaged recruitment strategies that focused on minimizing delays in entering and navigating prison facilities by gaining a comprehensive understanding of facility-specific protocols and ensuring that study procedures were responsive and adherent to facility-specific protocols.

During the initial planning meeting for each recruitment cycle with prison administrators, study staff would ascertain facility-specific security requirements. For example, the facilities in this study had different rules regarding study staff appearance as well as differing levels of security screenings that affected which articles of clothing would be able to pass through the gatehouse metal detectors. Study staff found it useful to adopt a “prison outfit,” comprising shirts, pants, and shoes that had previously passed the facility’s screening process. Furthermore, study staff carried backup clothes in the car in the event that they did not successfully pass the screening process the first time. Two of the three study sites had rules that could permanently ban staff from entering the facility if they repeatedly failed to clear security screening. Therefore, failure to pass the screening process could derail recruitment activities for a given day and erode relationships with prison staff.

Facilities also had strict rules around what types of study materials were allowed to enter the facilities. In response to prison facilities’ prohibition of staples, paper clips, and other metal objects, the study team adopted specially formatted paper interview booklets for all interview materials. Additionally, blank copies of all research materials (e.g., interviews, informed consent documents, and study flyers) were reviewed and approved by prison administrators as part of their Institutional Review Board review process, which minimized suspicion or confusion among prison staff as to the types of research activities taking place in their facilities. This review was also necessary in order to obtain a “gate memo,” or list of the approved recruitment materials against which study staff would be checked when entering facilities on each recruitment day. The study team also complied with facility security protocols by developing procedures to systematically organize and account for all study materials at the end of each recruitment day, ensuring that no study materials were left inside the facility.

Additionally, study staff ensured that participant-specific information was placed in sealed and initialed envelopes during transport at a facility. This provided confidentiality of participant information while adhering to prison rules that required staff and visitors to carry all possessions in transparent bags. Finally, the study’s recruiter training protocols included a “shadowing” period for new study staff. This allowed new study staff to learn both the written and unwritten rules and practices within each prison facility and become familiar with prison staff and operating procedures.

Ensuring peak preparedness.

Study staff developed a mantra of peak preparedness in response to a number of challenges they experienced while conducting recruitment activities in the prison facilities. These challenges included the time-intensive nature of travelling to study sites, completing security screenings, and the requirement that study staff be escorted by prison staff to the location within the facility where study recruitment took place. They also included the strict facility schedules that limited the time available for recruitment activities each day. Study staff developed a number of strategies that maximized their access to space and potential participants during each recruitment cycle that, together, constituted “peak preparedness.”

First, the study team learned and carefully documented facility-specific schedules in order to plan recruitment activities at times that minimized disruptions in both individual and institutional schedules. For example, during a typical recruitment day, the study team had to work around prison facility “count” times. These occurred at least 3 times each day, during which all prisoners were required to be in their cells and were therefore inaccessible to study staff. Other schedule related considerations included meal times, additional facility lockdowns related to things like breaches in security and staffing shortages, the work schedules of prison staff and potential participants, staff shift changes, and other prison programming. The study team also avoided scheduling recruitment activities around major holidays due to reduced facility personnel resources during these times.

In addition, the study team developed a number of strategies to maximize their preparedness for recruitment activities once inside a facility. These included things as simple as ensuring extra copies of all recruitment materials were available to study staff during each recruitment day, allowing study staff to be self-sufficient once inside the facility. Furthermore, due to the time spent being processed in and out of the facilities, study staff were always prepared to spend the entire day inside the facility. In order to do this, study staff had to bring their own food and water in prison-approved packaging.

The strict schedules within prisons meant that facility staff were only available to escort study staff at certain times and delays in security screening or arriving late to a facility due to traffic could severely limit or curtail recruitment activities for the day. Therefore, study staff would routinely arrive at facilities 30–45 minutes before schedule. Finally, the study team would always try to send an additional study staff trained in study recruitment activities for each recruitment day. This optimized the team’s ability to take advantage of additional space in the prison that was suitable for study recruitment activities if it became available unexpectedly, even for limit periods of time, during a study recruitment day.

Discussion

The results of this research show that one-third of potentially eligible participants were enrolled in the open trial phase of the study and just over half of potentially participants were enrolled in the RCT phase. These findings suggest that the recruitment strategies that the study staff developed and refined over the course of the study achieved their intended goals.

Given the increased emphasis on recruitment efforts in clinical trials research, this study offers timely and concrete strategies that researchers can use to facilitate recruitment efforts in prisons and other real-world practice settings not optimized to accommodate research. Staff in this study learned that the more they asked facilities to modify their operations and practices, the more difficulties they experienced in recruitment efforts. On the other hand, adopting the strategies described above helped study staff build positive relationships and reputations within the facilities which helped to reduce barriers to successful recruitment. The results of this research demonstrate the importance of developing research procedures that are responsive not only to a research team’s internal needs and deadlines, but also to the needs of the organizations where the research takes place.

This analysis identified several strategies that the study team employed to optimize recruitment efforts in a bureaucratic and hierarchal research environment that adheres to strict schedules and routines of daily life within each facility, and rules regulating who and what were allowed into the facilities. The application of these strategies needed to be tailored to each individual facility. In this regard, the open trial phase of the intervention proved invaluable for gaining an understanding of the rules and procedures of each facility and enabled researchers to develop strategies that could be deployed during the RCT phase.

These recruitment strategies also demonstrate the value in having study staff with prior experience operating in the setting where the research takes place. This is especially important in places like prisons which have a host of rules, some written and others not. Having staff with prior experience conducting research in correctional settings helped sensitize the study team to potential barriers that could impede study operations and provided the team with a “boundary spanner” who could help bridge the gaps in knowledge and practices between study and facility staff.

The inductive analysis used in the current paper did not reveal any specific strategies used by the research team regarding individuals’ mental health symptomatology. One potential explanation for this is that members of the research team already had extensive experience recruiting individuals with SMI into research. Therefore, the study procedures used to support the recruitment of individuals with SMI may have been taken for granted knowledge that was hard for staff to identify or name in discussions. Another possible explanation is that the barriers to study recruitment imposed by the prison setting were more pertinent to recruitment efforts in this study than those imposed by individuals’ mental health symptomatology. Finally, in the prison settings in this study, individuals with SMI had consistent access to mental health services including medication. Therefore, mental health symptoms may be better managed in this setting than in other community settings. Future research should further explore the strategies that support the recruitment of justice-involved individuals with SMI, in both community and corrections settings.

The results of our analysis also demonstrate the importance of engaging a more detailed level of analysis of recruitment activities than is commonly reported in approaches such as the Consolidated Standards of Reporting Trials.²⁹ For example, studies reporting recruitment rates in prison settings have historically tracked participant recruitment starting at the assessment of eligibility.^{e.g., 9,30–33} The results of our study demonstrate that, in order to estimate the number of potential participants needed to meet recruitment goals in prison research, it is important to track potential participant attrition at all steps in the recruitment process.

The results of this analysis show that half of individuals invited to participate in the study were lost to attrition before an assessment of eligibility could be made, and nearly a quarter of the potential participant pool was lost before any contact with study staff due to non-response. When we calculate rates of refusal beginning with individual who were approached by study staff, we find that 28.5% of potential participants declined participation. This was higher than the refusal rates reported in other studies conducted in prison research which range from 6.7%³² to 11.8%.⁹ It is worth noting that other studies conducted in prisons either fail to report refusal rates entirely or only report refusal rates among those who meet eligibility criteria.^30,31,33 The relatively high refusal rate found in our study could be explained by the fact that the initial contact with potential participants was made by the research team. This method of recruitment addressed a key ethical concern in this study related to ensuring that participant selection was fair and protected from influence by the prison. But this procedure could also have increased the refusal rate by creating a situation where potential participants were learning about the study for the first time when meeting the study team.

When our study enrollment refusal rate is calculated as the percentage of potential participants who were not enroll in our study after completing informed consent, our refusal rate is quite low (6.4%). While this refusal rate more closely aligns with other published estimates, it points to a disconnect that requires attention in clinical trials research in prisons and other real-world settings. Namely, that many published refusal rates do not provide the information needed to help studies determine the size of the potential participant pool needed to achieve a desired sample size. Our results highlight the need for a relatively large potential participant pool, and the allocation of significant study resources to recruitment activities, for even small-scale intervention studies conducted in prisons. As is the case with any study site that houses a limited number of individuals, the size of the required potential participant pool will determine how many sites will need to be engaged during the site selection process.

The findings from the current study should be interpreted within the context of several limitations. First, due to the relatively small number of participants as well as limited number of study sites included in the current analyses, comparisons of recruitment rates and strategies by facility was not possible. Future research should examine differences in recruitment rates and strategies by both individual- and facility-level characteristics in research with people with serious mental illnesses in prison settings. Additionally, since this study took place in the southeastern United States, future research is needed to examine if these recruitment rates and strategies are generalizable to broader geographic areas, including internationally.

Establishing realistic recruitment metrics and milestones is difficult when working in and with hard-to-access facilities and populations. In these contexts, recruitment barriers and bottlenecks are often frequent and difficult to anticipate. By documenting and analyzing recruitment outcomes and strategies, clinical trials research in the behavioral and social sciences can help mitigate these challenges and promote study feasibility. It is important that the results of these analyses be published to assist future studies in planning timelines, anticipate challenges, and allocate resources accordingly.

Abbreviations:

SMI

serious mental illness

RCT

randomized controlled trial