Developing a Community-Based Participatory Research Model to Engage Transition Age Youth Using Mental Health Service in Research

Abstract

We present a model for the development and conduct of a Community-Based Participatory Research (CBPR) project with transition age youth (TAY) mental health service users. CBPR frameworks can facilitate equitable partnerships and meaningful inclusion, but have not been fully drawn-upon in mental health research. The model included TAY as trained research associates involved in every aspect of the research process. We describe the development of the project, creation of the research team, training, the design and conduct of the study, and challenges faced. The methods developed successfully provided support for the meaningful participation of TAY in the project.

Introduction

Recently, there have been a number of efforts that encourage the inclusion of the voices of community members, including service-users and their families, in the design and conduct of mental health services research.^1–3 Community-based participatory research (CBPR) frameworks provide a model for increased community inclusion in research, but these models have not been fully drawn upon in mental health services research. While the benefits and challenges of the use of CBPR methods and principles have been well documented,⁴ the increased use of CBPR necessitates a critical examination of the ways these paradigms can be incorporated in mental health research as well as strategies to address challenges. Here we describe the development and application of a CBPR model, in studying the role of housing in the lives and well-being of TAY between the ages of 18–25, who have been using mental health services (“service users”) living in Boston, Massachusetts.

Community-Based Participatory Research in Mental Health Services Research

While significant progress has been made in the development of effective treatments and preventive interventions targeting mental illness and substance use disorders, the findings of rigorous research have not been suitably translated into services in non-research settings.^{1, 2}

Mental health services research has typically left out consumer input, resulting in delays in the translation of research into practice.^4,5 To address this gap, the President’s New Freedom Commission on Mental Health report recommends the involvement of consumers and families in the planning and evaluation of services, with a specific reference to consumer led research.^2(p37)

Community-Based Participatory Research (CBPR) is a collaborative approach to scientific investigation that engages community members affected by the issues being studied to fully participate in all aspects of the research process.⁶ Recognizing the community to be a unit of identity fostered by emotional connection with other members, the CBPR approach allows for a research process to unfold that is characterized by shared decision-making and ownership. This is facilitated by a meaningful understanding of a group’s unique life and health circumstances that focuses on the existing strengths and assets in a community and by involving all stakeholders in every phase leading up to and inclusive of the research plan.⁷

While CBPR principles have been drawn upon in many areas of public health and medical research, few mental health studies have drawn upon these principles and methods. Rowe⁸ has described how CBPR and concept mapping were used to develop a multi-dimensional construct of citizenship in relation to people with mental illness. In another study CBPR methods were used to develop coping skills interventions designed to increase the utilization of services by African American adolescents and transition age youth.⁹

Transition Age Youth, Mental Illness, and Housing

The developmental stage between adolescence and adulthood (ages 18–25) is a critical life transition involving physical, contextual, social and emotional changes.¹⁰ While transitioning from adolescence to adulthood is a challenge for all young people in our society, it is a major struggle for young adults who have been diagnosed with a serious mental illness (SMI), many of whom went through adolescence with symptoms that affected their development and, as young adults, face institutional transitions in addition to developmental transitions.^11–13 These young people experience worse outcomes, compared to their non-diagnosed peers, in the domains of: social skills; academic achievement; criminal activity and legal troubles; employment and financial independence; and more limited interpersonal relationships.^14,15 This group, which often has difficulty securing safe and stable housing, has the highest rates of residential change of any age group.¹³ It has been argued that TAY must be safely housed before other critical issues such as mental or physical health, education, and vocational goals can be addressed effectively, yet developmentally appropriate housing supports for them are limited.

Partnering to Study the Mental Health & Housing Needs of TAY

The use of CBPR methods to examine the role of housing in the lives and recovery of TAY provides an exceptional opportunity to increase the inclusion of their voices in mental health research. There are very few models of research in which the involvement of youth with mental illness and substance use disorders occurs at every step of the research process and typically the involvement of young people in the design and conduct of research has been limited to participation in surveys and focus groups.^16,17 In this paper, we describe the development and implementation of a CBPR project in which community members were involved in every step of the research process.

The Boston Community Academic Mental Health Partnership

The Boston Community Academic Mental Health Partnership (B-CAMHP), established in 2006, was designed to bring together academics, mental health service users, and mental health community stakeholders to build an infrastructure to support CBPR methodologies in mental health services research. The B-CAMHP grew out of existing collaborations among several of the partners and included both public and private organizations. The primary B-CAMHP partners are: Northeastern University; Consumer Quality Initiatives (CQI) – a mental health, consumer-directed and staffed research organization; the Massachusetts Department of Mental Health (DMH); the Massachusetts Chapter of National Alliance for the Mentally Ill (NAMI); the Parent/Professional Advocacy League (PAL) – an organization directed by parents of youth who have (or have had) a serious emotional disturbance; and the Transformation Center – a consumer run organization that provides statewide technical assistance and training.

The TAY Research Project Team was established by the B-CAMHP partners and included representatives from each partner, as well as the statewide DMH Youth Coordinator. The TAY Project Team chose to develop a grant proposal to conduct an exploratory study to examine the role of housing in the lives and recovery process of TAY. In 2010, with the receipt of NIMH funding, the “Transition Age Youth and the Boston Community Academic Mental Health Partnership” study began. The project used a multiple Principle Investigator (PI) model with one PI based at a local university (Lincoln – Northeastern) and the other, an executive director of a community-based non-profit (Delman – CQI). While Northeastern served as the lead agency for administrative purposes, the majority of the funds were placed with CQI in an attempt to truly place the financial support in the community. The goal of this study was to better understand the housing support needs of transition age youth living with mental health conditions. In this paper we present the model for developing and maintaining a TAY CBPR partnership.

The B-CAMHP TAY Study

Developing the Partnerships

Our first study aim was to develop a mechanism for young adults in Boston living with mental illness to participate in the design and conduct of mental health research. Young adult participation in research design and implementation allows for them to actively examine issues that affect their lives, make decisions to create meaningful change in their communities with respect to these issues, and contribute to creating health programs and services that can better meet the needs of young people while simultaneously expanding their knowledge and skills.^17,18 Investment in the development of research partnerships with TAY has been shown to improve recruitment¹⁹ and retention,²⁰ and trained community interviewers are in a better position to carry out interviews while improving rapport with respondents. The study design included the creation of two teams: a research team and a project team. The Research Team was responsible for the design, conduct, and dissemination activities of the research study, and included the Principal Investigators (PIs), a Project Coordinator (PC) and six TAY Community Research Assistants (CRAs). The larger Project Team included the original B-CAMHP community partners who provided oversight and guidance.

Hiring and Training Community Research Assistants

Transition age youth mental health service users were recruited from the local community to work as CRAs. Our goals were to recruit young adults who had interest in conducting research, who were at all stages of their illness and recovery, and who represented racial, ethnic, gender and age diversity. The Project Team developed the job description, including that we sought 18–25 year olds, living in Boston, who were using mental health services, for a research assistant paying $13 per hour. This was circulated through partner’s networks in order to reach community members not connected to established organizations or formal groups. We also placed notices on electronic bulletin boards such as Craigslist.

We received a large number of resumes (over 40) from individuals interested in the position over five weeks. Fourteen applicants, all of whom met the job description criteria, were interviewed by the PIs and the PC using a standard set of questions as a guide. Out of these 14 applicants, six CRAs were hired. Our CRAs were disproportionately female, between the ages of 18 and 25, self-identified as African-American, Asian, and White, and all had used mental health services. Their education level ranged from college degree to taking GED classes and they had diverse lived experiences of mental health and recovery.

The Community Research Assistants participated in extensive training with the research team in order to address the imbalance of knowledge and resources between the researchers and community members that can sometimes act as a barrier to active engagement on the part of the community members.^22,23 The training, developed by the PIs and overseen by CQI, was both classroom and field based, and was supplemented by interactive discussions, observed practice, ongoing group discussions and supervision.^21,23,24 The 16-hour didactic portion of training included an introduction to research, an overview of research methodologies, data collection (in this case, qualitative interviewing), research ethics and informed consent, collaborative research models, and current research in mental health services, housing and TAY. While the PIs and the PC lead the sessions, the training emphasized participatory methods, including role-plays and question/answer formats. The training also included workshops led by community partners sharing their perspectives and knowledge on mental illness, transition age youth and housing. These cross-trainings encouraged team-building and addressed the level of mistrust that sometimes exists between the community members and community organizations.²²

Mastery of these concepts and skills was demonstrated during the training sessions, and also through a written exam developed by the PIs that focused on qualitative research methods. Each CRA completed the NIH Office of Extramural Research (OER) on-line tutorial on Protecting Human Research Participants and received the NIH certificate. Lastly, the CRAs moved into the field, where they conducted at least two interviews independently, as well as an observed interview, including assessment of capacity to consent to participate in research, with either the PC or a PI observing.

As the CBPR model capitalizes on the benefits of community members as interviewers, including increased rapport and trust with community members,^22,23 a critical part of the training focused on the interviewer’s role in developing rapport between interviewer and interviewee and establishing relationship boundaries. As a part of the rapport building process, the CRAs elected to share a personal narrative with the interviewee. The CRAs wrote their narratives, including how they chose to introduce themselves, talk about their role as service-users, and explain their involvement in the study, and practiced them during trainings. We also developed scenarios that allowed the CRAs to practice skills related to relationship boundaries while interviewing. When utilizing a CBPR model that trains community members from the group of interest to interact with and/or collect data from their peers, training takes on increased importance as study participants may say or do things with a peer that they would not do with an academic researcher. The hope is that study participants will feel more comfortable conducting an interview with a peer and subsequently share meaningful, rich information, rather than a researcher, who could unintentionally cause the participant to feel an imbalance of knowledge and power, and subsequently “shut-down” and act more guarded. The thorough and rigorous training process we engaged in ensured that the CRAs were capable of conducting effective interviews. The Research Team also created a set of guidelines attending to participant behaviors and emotions during the interview and offering suggested actions if a participant became upset, distressed, agitated or inappropriate. This included practicing how to end an interview early if necessary, knowing when study confidentiality should be broken if a participant threatens to hurt themselves or others, and how and when to call the Boston Emergency Services Team. Finally, to support the CRAs the PC accompanied each CRA to their first interview, and was available by phone during all remaining interviews. After each interview CRAs called the PC to debrief. When the formal training was complete the Research Team continued to meet weekly. During each meeting, the team focused on designing and/or implementing different aspects of the research study.

Literature Review

The first activity of the Research Team was to conduct a literature review on the topics of TAY, mental health services, housing, and homelessness. The purpose was to place individual lived experience of CRAs in the context of the larger young adult population and to identify gaps in knowledge and key questions that remain unanswered in the academic literature. Articles were read and discussed as a group. This process, combined with the CRA training, provided a strong foundation for the team to begin to operate as cohesive group and to define research questions.

Defining the Problem

The literature review process allowed us to shift our focus to refining the research questions. We discussed the importance of language, and invested time to working through the implications of using phrases such as “mental illness” versus “mental health condition” versus “psychiatric disability.” It was important that we used language that reflected the community of interest and represented real-life issues, needs and experiences of our community. The study question chosen, through an iterative process based on the literature review, lived experiences of the RAs, and group dialogues was: What is the role that housing plays in recovery and over all well-being of young adults living with mental health conditions in Boston?

Designing the Research Methodology

We first worked with the Massachusetts Department of Mental Health to understand what types of data existed describing TAY and housing. We then chose a mixed methods approach incorporating existing administrative data with in-depth one-on-one interviews with TAY. This qualitative method provided both the depth of exploratory information and the flexibility required to address our research question. Next, we made decisions about the nature and structure of the interview guide. The CRAs, PIs and PC developed the guide in three stages. First, domains of inquiry, topics deemed most important to cover in the guide were chosen based on the research question, the information learned from the literature review, and the CRA’s lived experience. The CRAs then wrote questions based on the domains of interest. Through an iterative process, questions were examined for areas of overlap and those which best captured the domains of interest were chosen. Questions were then organized and an interview guide was structured in a way that flowed logically. Next, the CRAs conducted practice interviews with each other and again through an iterative process refined and edited the list of questions. A draft of the interview guide was brought back to the Project Team and a similar process of revisions and edits occurred. Finally, we conducted pilot interviews with TAY that lived outside of Boston to check for poorly worded and confusing questions, and to ensure that the interview could be done in approximately one hour. Lastly, a final version of the interview guide was created.

Development of the Human Subjects Protocol

In B-CAMHP’s prior CBPR work, several challenges in developing human subject’s protocols for multiple institutional review boards (IRBs) were identified. Most often the primary concern was the ability of community members to interview their peers. In anticipation of these obstacles the IRBs were provided with information on our rigorous research training and evaluation process. However, in this study IRBs raised another concern regarding the ability of our CRAs to assess the respondents’ capacity to consent to participate in research. While some behavioral health research projects involve clinicians in the assessment of capacity to consent, others rely upon professional lay interviewers. In response, we adapted a strategy used in a previous study conducted by the B-CAMHP. Based on work by Appelbaum and Grisso²⁵ we included a set of three true/false questions in the consent form to facilitate determination of a respondent’s capacity to consent. This strategy successfully addressed the IRBs’ concerns.

The IRBs also raised concerns as to whether our CRAs, TAY themselves, might face their own personal challenges interviewing. We addressed these concerns by expanding our self-care plan. Self-care was built into all aspects of interviewing. For example CRAs could take breaks from the project if they conducted a particularly difficult interview or they could have someone else on the team transcribe a difficult interview so they did not have to “relive” the experience. In addition, CRAs developed a list of “self-care techniques” such as journal writing, listening to music, talking to supports outside of the project, as well as a plan for what they would do if distressed by an interview, and CRAs debriefed after each interview with the PC.

Finally, one IRB raised concerns about how we would handle the situation of when a CRA knew the young adult she was going to interview, given that this was more likely than in standard research projects as our interviewers were peers. We developed a protocol allowing the CRA to proceed with the interview if she felt comfortable doing so, or to notify the PC who would then reschedule the interview with a different CRA.

Study Sample

The group shared the belief that the study sample should include TAY mental health service users, but there were many ideas as to how this should be defined. There was agreement that the sample not be based on any diagnostic categories but include TAY who had a “significant service use” experience, particularly those whose mental health conditions were severe enough that they affected their housing situations. Drawing on the knowledge gained through the literature review and the CRAs lived experience, we considered several indicators of severity: the experience of hospitalization, receiving social security disability, or being a client of the Department of Mental Health. However, the CRAs identified limitations of each of these approaches and pointed out that it is possible someone could have a serious mental health condition but never have been hospitalized for it, may not know about social security, and may not seek out or experience acute mental health services. The team then consulted with several academic researchers and developed the following criteria to identify our study sample: young adults that received specialty mental health treatment (examples include treatment provided by a psychologist, psychiatrist, mental health counselor or social worker) prior to and after their 18^th birthday; and young adults that have used at least one of the following services at some point in their life: in-home visits for mental health, inpatient psychiatric hospitalization, day treatment programs, case management for mental health or residential home living for mental health.

The team proposed to conduct a series of 40 open-ended qualitative interviews, 10 in each of four categories that capture diverse perspectives of housing for TAY in Boston. The final categories chosen were: 1) living independently with no formal mental health services attached to their housing. This did not mean that TAY had to be living on their own but meant they had to be living in an environment that was not attached to formal health services; 2) in supported housing referred to TAY living in an environment that included services meant to help the person maintain her/her housing; 3) in group homes – TAY who were living in a facility, such as a residential home, that was supported by private or state funding and living with other TAY; 4) who were homeless including TAY who had no permanent address, were couch surfing, or living with a friend(s).

Recruitment

Discussions of recruitment focused on how and where we would find and recruit a diverse group of TAY in the housing situations identified. We sought diversity in gender, age, race, ethnicity and experience with mental illness. The CRAs helped to identify potential recruitment sites in the community, however, the recruitment methods chosen were primarily shaped by the experience of the researchers in previous community studies. Each step of the recruitment strategy was discussed with the CRAs and they provided feedback and made suggestions. Participants were recruited using two different strategies. First, a flyer was distributed that provided the PC’s contact information so that interested participants could call the PC directly. Secondly, a permission-to-be-contacted form was created and distributed by staff on-site and then given to the PC who called potential participants. The team also discussed appropriate study incentives including whether to provide cash or gift cards. Ultimately, study participants were compensated $20 in cash for their time and transportation costs.

Qualitative Data Collection

Referrals were screened for study eligibility by the PC and assigned to a CRA for interview. All interviews were conducted by the CRAs ensuring that the interviewers were from the same community as the research participants. The Research Team developed a packet for the CRAs to take to the interview. In addition to the consent forms and interview guide, this included a checklist of the steps needed to complete the interview process, an Interviewer Impression form and a resource guide to be handed out to a participant if he/she expressed need for services in the Boston area. Community RAs also carried an “Interviewer’s Manual” that contained all materials from the CRA training, interview procedures, important phone numbers, an overview of the interviewer’s role, tips for safety and effective interviewing, self-care and wellness plans, techniques and tips for interviewing, the interviewing protocol, audio recorders, and information about housing.

The interview protocol included the personal narrative and introduction followed by the informed consent process. Following the completion of the interview, in addition to the de-briefing, the Interviewer Impression form was completed. This form provided an opportunity for the CRA to document any questions or concerns they may have about the interview, to note any particularly interesting sections of the interview, and to document any concerns as to how well the participant understood the questions or was able to participate.

Data Analysis

A rigorous group coding and analyses process was developed. CRAs received additional training in qualitative analyses including inductive and deductive coding, a demonstration of the group code development process using sample transcripts, and an introduction to HYPERRESEARCH qualitative software, a program for data entry, coding, sorting and retrieval.²⁶ The training prepared the CRAs to be fully engaged in the subsequent steps of the data analysis process.

After completing an interview, the CRAs transcribed the audio-recordings of their interviews following specific guidelines for formatting to facilitate the interface of the transcript with analytic software. After a transcript was completed, it was given to the PC who reviewed it for quality and clarity. The PC also used this as a training opportunity to address question flow and rigor and review these concepts with the interviewer. At Research Team meetings, transcripts were reviewed to provide feedback to the CRA interviewer.

In the second step of the group data analysis process, the Research Team reviewed selected transcripts and underlined important ideas related to the study aims. The team then met and discussed areas of agreement and disagreement. Next, underlined segments of text were tagged with descriptors and through an iterative process these descriptors became a preliminary code list. New selected interviews were then evaluated by this preliminary set of codes. The Research Team reviewed and refined the coding system using three sets of multiple interviews, and finally the PC developed definitions for each code based on group discussion. The team reviewed codes and definitions, and a final set of revisions to the code list was made. Transcripts were then entered into a computer using HYPERRESEARCH. The PC, a CRA and a graduate RA coded the full data set using HYPERREEARCH. After all of the transcripts were coded, code reports were run to reproduce all segments of text to which each code is attached, including reference information about the original location of the segment.

Data analysis was iterative, first reviewing reports for single codes. Code reports were produced by the PC and distributed to the Research Team. Each member of the Research Team read through the code report and completed an “analysis memo.” The analysis memo was used to identify the themes and patterns that were present in the text of each code report. These themes and patterns were analyzed to develop main findings and conclusions. The findings and conclusions will be disseminated through multiple mechanisms as determined by the team.

Results

Twenty-nine transition aged youth participated in an interview with a CRA. The age range of the participants is 19 to 25 years, with a mean age of 22 years. Seventeen males, 10 females and two people that identified as other took part in the study. Thirteen (44.8%) self-identified as African-American or Black, 6 (20.7%) self-identified as Caucasian or White, 6 (20.7%) self-identified as multi-racial, and 5 (17.2%) self-identified as other. Six people (21%) identified as Hispanic/Latino, 21 (72%) identified as not Hispanic/Latino and 2 (7%) identified as other. One fifth of the participants were living independently, 4 (14%) were living in supported housing, 10 (34%) were living in a group home, and 9 (31.0%) were homeless. These and other characteristics of the study sample suggest that we were able to reach and engage TAY from diverse racial/ethnic backgrounds, ages, genders, and a variety of housing situations and with differing levels of housing stability.

Discussion

We successfully created a mechanism for TAY to participate in CBPR through expansion of the B-CAMHP. This is demonstrated by the CRAs successful completion of CRA training, and the design and conduct of a mixed-methods study with full community participation at each stage of the research process. Twenty-nine interviews were conducted at which point saturation was reached in the data, and final dissemination activities are ongoing and include the participation of TAY. Many challenges were identified throughout; some related to the use of CBPR generally and other concerns related to the extension of these methods to mental health services research. We have touched on four of them here: defining the community, issues of disclosure, unique considerations of the nature of mental illness, and human subjects concerns and working with IRBs.

Communities are defined in many ways. Communities in CBPR projects are often defined by geography. They are also frequently defined by a shared experience, problem or health condition. Our team chose to define community as a group of people using a certain type of health services, specifically mental health services, in Boston. Developing consensus on the definition and boundaries of community is a lengthy, challenging, but critical process. It is marked by discussions of “insider” and “outsider” status, and seeks to give voice and empower those without a voice in the traditional research process.

Disclosure of community membership is complicated by the presence of stigma. When conducting CBPR with stigmatized communities, discussion of the cost of disclosing community membership becomes critically important. Our CRAs applied for a position that explicitly required lived-experience with mental illness or use of mental health services. However, a person’s choice to disclose or to be “out” with their community membership may not be static, and may change with context and life circumstances. Thus, throughout the course of the project CRAs often felt different levels of comfort with disclosing their community membership. For example, if a CRA decides to go back to school or apply for a new job, they may not want to be listed on the internet as a CRA on a project with a stigmatized community. As CBPR is built on “community participation” this creates some innate challenges. We learned that open-discussion of this issue, and respect for diverse perspectives on this challenge, was critical to moving the project forward.

Finally, more work is needed to explore whether the unique, often cyclical nature of mental illness plays a role in the design and conduct of CBPR studies with the community of mental health service users. As researchers with training in mental health, we understood that mental illness falls on a continuum of severity and thus our CRAs health might vary over the course of the project. We worked as a team to develop strategies to support the CRAs and adapt the training and study design to fit these needs as much as possible.¹⁹ However, this was a concern for the IRBs that we worked with. There was concern over the need for clearly articulated self-care plans for our CRAs, and a more generalized lack of understanding of the nature of mental illness and the ways in which it may or may not matter in training research associates. This is likely related to stigma as well. We did a great deal of work, partnering with and educating IRBs. For instance, self-care plans are effective supports for all interviewers, particularly those engaged in interviews that elicit discussion of difficult or painful topics. And while it is more likely that a CRA would have a prior relationship with a research participant in a CBPR study, this again, could occur in any research study, and the responsible conduct of research necessitates the development of a plan for this circumstance should it arise. These efforts were time consuming but of great importance, and the impact will likely be felt beyond our project.

Transition age youth have had little voice in the design and delivery of services aimed at supporting them. Our use of CBPR, with TAY who are often difficult to engage in research, provided benefit to each aspect of our study design and protocol. The academic-community partnership and the inclusion of CRAs assured that our study questions were highly relevant and that the knowledge generated would have the potential for impact for efficient change. The development of the interview guide was enhanced by the contribution of the CRAs and contributed to the development of a clear and respectful interview guide. The use of CRAs as interviewers contributed to a strong rapport with study participants and enhanced the types and quality of data we gathered. The involvement of the CRAs in every aspect of the data analyses process challenged the team to question pre-existing assumptions, always a critical aspect of qualitative analyses. Our CRAs viewed the world differently than other members of the Research Team and this allowed for a rich and deep exploration of the meaning of segments of text and codes. As we finalize the development of our dissemination strategy, the use of a CBPR methodology has broadened our scope, beyond the traditional academic papers, to include policy briefs, presentations to stakeholders, newsletters of partners, and the exploration of other innovative ways to share what we have learned.

Finally, the impact of this CBPR project with TAY has already been felt, even prior to the dissemination of the study results. Several of our CRAs have transitioned to other educational and occupational opportunities. We have educated multiple IRBs and are now often consulted when they review CBPR proposals. The state mental health agency and other service providers continue to expand the inclusion of TAY in their ongoing work and the positive experiences of the BCAMHP TAY project provide evidence of the feasibility of these types of efforts. Finally, there has been the undocumented impact on our study participants, who often asked our CRAs, “How did you get this job?” as well as questions about how they got well, and stayed housed. Creating a mechanism for TAY mental health service users to participate in research is highly rewarding and efforts to increase their voices, as well as those of other young people in research should be expanded.