Engagement and Retention of Suicide Attempters in Clinical Research

Abstract

Background

High attrition rates in longitudinal research can limit study generalizability, threaten internal validity, and decrease statistical power. Research has demonstrated that there can be significant differences between participants who complete a research study and those who drop out prematurely, and that treatment outcomes may be dependent on retention in a treatment protocol.

Aims

The current paper describes the challenges encountered when implementing a randomized controlled trial of cognitive therapy for the prevention of suicide attempts and the solutions developed to overcome these problems.

Methods

Problems unique to suicide attempters are discussed, and strategies successfully implemented to boost retention rates are provided.

Results

The methods implemented appeared to increase retention rates in the randomized controlled trial.

Conclusions

Many steps can be taken to work with this difficult population, and researchers are encouraged to be as involved and flexible with participants as possible.

Attrition within a study’s sample population has long been recognized to be an important factor to consider when interpreting research results. Because there is often a qualitative difference between those who finish a research study and those who drop out prematurely (e.g., Issakidis & Andrews, 2004; Pagnin, de Queiroz, & Saggese, 2005), high rates of attrition can compromise the interpretation of results. Incomplete follow-up information limits the generalizability of research findings, can threaten the internal validity of the study, and reduces statistical power. Retention in longitudinal research poses a particular challenge for research with traditionally difficult-to-follow populations, such as individuals who abuse substances, those with severe mental illness, or homeless individuals. Retention in intervention research is particularly challenging during the acute treatment phase because of the importance of frequent patient visits. Research indicates that those who leave treatment early may have worse treatment outcomes than completers (Bleiberg, Devlin, Croan, & Briscoe, 1994; Messina, Wish, & Nemes, 2000).

In recent years, researchers have been increasing their efforts to boost research retention rates. There is evidence that increased system awareness of problematic research issues including recruitment and retention can enhance problem resolution and improve participant involvement (Morse, Simon, Besch, & Walker, 1995). Substance-abuse researchers have paid particular attention to attrition rates, and several reported that some of the strategies employed to increase retention in their studies (e.g., Cottler, Compton, Ben-Abdallah, Horne, & Claverie, 1996; Desmond, Maddux, Johnson, & Confer, 1995). Cottler and colleagues (1996) generated a follow-up rate of 96.6% in a longitudinal study of out-of-treatment drug abusers. Their methods included decreasing the length of follow-up assessments and conducting meetings in locations convenient to participants, such as their homes or jails. According to the authors, successful retention relies on “patience, persistence, enthusiasm, creative team work and money” (p. 215).

Other researchers have highlighted successes in minimizing attrition in other at-risk populations. McFarlane (2007) reported strategies for the retention of abused women in longitudinal research that resulted in retention rates ranging from 89% to 100% in a number of studies with varying methodologies. Cohen and colleagues (Cohen et al., 1993) reported an 86% retention rate of homeless mentally ill (HMI) individuals at 4 months and 83% at 12 months. Belcher and Toomey (1988) reported an 86% retention rate over 3 months with patients discharged from a state mental hospital. These impressive rates may be related to attributes of both the patient population and the systems of support. Of Cohen et al.’s (1993) participants 72% reported no substance abuse at intake, and the investigator’s primary source of leads was from paper trails through mental health and shelter settings. Belcher and Toomey’s (1988) figures were a result of weekly coordination with homeless shelters, jails, mental health clinics, social security records, welfare offices, and the attainment of photographs of participants. Although these methods can greatly increase the likelihood of maintaining contact with transient study participants, such methods may not always be ethically or logistically feasible for researchers. Furthermore, individuals who consent to such coordination of information may not be representative of a broader population of individuals who are homeless or abuse substances.

Retention in research is of the utmost importance when conducting intervention research with suicidal individuals, given the level of risk associated with this patient population. To date, there has been little discussion of the difficulties encountered when conducting efficacy research of interventions for suicide attempts, and there are few suggestions regarding follow-up strategies aimed specifically at individuals who have made attempts. Several randomized controlled trials that have been successful for preventing suicide attempts have involved the use of home visits (Guthrie et al., 2001; Huey et al., 2004; Salkovskis, Atha, & Storer, 1990; Welu, 1977). However, suicidal individuals in urban settings cannot always be followed by social or service structures like home, family, or health providers. High levels of hopelessness and poor overall functioning often lead to an avoidance of regular contact with health providers, unemployment, and homelessness.

Another salient characteristic of the population of people who have attempted suicide is a high incidence of substance abuse and dependence. A co-occurring diagnosis of substance abuse has been shown to increase an individual’s risk of becoming homeless by about three times (Caton, Wyatt, Felix, Grunberg, & Dominguez, 1993). Given the no-tolerance drug policy of shelters, patients with the poorest prognosis are likely to become homeless and remain homeless without access to shelter services. Christensen (2006, p. 447) asks “where is the research on homeless persons and suicide?” highlighting the dearth of research that may be due to the difficulties encountered with this transient population.

This paper describes the challenges encountered in implementing a pilot intervention study for patients in an urban setting who had recently made a suicide attempt and is unique in providing a number of strategies that were successfully employed to address these challenges. The full randomized controlled trial (see Brown et al., 2005; here-after identified as the “clinical trial”) was a study of the efficacy of a 10-session cognitive therapy intervention designed to prevent repeat suicide attempts in adults who had made previous attempts. The results of the study indicated that a brief cognitive therapy intervention (see Wenzel, Brown, & Beck, 2008) was effective in preventing repeat suicide attempts among individuals who had recently made suicide attempts. Participants assigned to the cognitive therapy intervention were approximately 50% less likely to reattempt suicide during the follow-up period than participants assigned to the control condition.

The retention rate in this study was high (85% at 12 months), especially given the severity of psychiatric illness, high rate of substance use disorders, and low SES of the participants. However, during an initial pilot study, retention rates were much lower (50% at 12 months), and the study population proved extremely difficult to follow longitudinally. This paper highlights some of the problems encountered during the course of the two studies and outlines how these problems were overcome in order to attain high rates of retention (see Table 1 for a summary).

Table 1.

Problems encountered and solutions developed by study phase

Phase of study	Problems	Changes implemented
Screening	Difficulty maintaining contact with po- tentially eligible participants	Interviewed potential participants in the hospital as soon as possible after admission
Informed consent	Unfamiliarity with research; suspi- ciousness	Lengthy discussion of consent, administered brief quiz to check un- derstanding before consent
Randomization	Negative reaction to TAU assignment	Assignment of study case managers to provide support and referrals
Completing baseline interview	Poor concentration	Use of open-ended questions; interviews conducted after medica- tions stabilized
Follow-up	Difficulty maintaining contact	Detailed locator interview at each assessment; community voice- mail; study case managers maintained frequent contact; tokens pro- vided for appointments
Therapy attendance	Poor attendance; inability to attend ses- sions on time	First session conducted while still in hospital; one established week- ly time per patient; full-time therapists available for “walk-in” ses- sions

Overview of Research Studies

Participants in both the pilot and the clinical trial were individuals who had attempted suicide and who had received a medical or psychiatric evaluation within 48 h of their attempt. Individuals were identified in the emergency department following a suicide attempt or intentional self-injury (e.g., overdose, laceration, gunshot wound) at the Hospital of the University of Pennsylvania, in Philadelphia, PA, USA. Individuals were not asked or required to discontinue any form of mental-health or substance-abuse treatment prior to entering the study. Baseline evaluations were typically administered within several days but no longer than 3 weeks after the suicide attempt by trained clinicians who held masters or doctoral degrees. Subsequent in-person assessments were conducted independently of study therapists up to 18 months following the baseline interview.

In the pilot-study design, participants were randomly assigned to receive cognitive therapy or not receive cognitive therapy. Participants in the cognitive therapy intervention were scheduled to receive 10 outpatient cognitive therapy sessions specifically developed for preventing suicide attempts. The cognitive therapy sessions were provided on a weekly or biweekly basis or as needed. Participants in both study groups received usual care from clinicians in the community and were offered referrals to community mental-health treatment, addiction treatment, and social services (as needed during the follow-up period). Although participants in both conditions were encouraged to seek additional mental-health and substance-abuse treatment in the community, the study did not cover the costs of these interventions.

Challenges Encountered and Solutions Developed

Challenges with Initial Engagement

When a patient was identified in the hospital as potentially having made a suicide attempt, every effort was made to establish contact while the patient was still in the hospital. After patients had been discharged home, they were difficult to contact, and recruitment efforts often faltered. Medical records were often out of date, displaying incorrect or out-of-service phone numbers and addresses at which the patients had not resided for years. In order to address this problem, study staff completed a location interview that included information on family and friends with whom the participant had contact as well as care providers, parole officers, shelters used by the participant, and other means of tracking the participant. This interview was conducted at each assessment to keep the information as current as possible.

Once patients had been contacted, several issues served as obstacles in obtaining informed consent. For example, finding an appropriate incentive for participation in extensive research interviews was a challenge. Some patients were interested in receiving the free therapy and/or financial compensation offered by the study, but on the other hand hesitant to enter into a research study. Some individuals conveyed an air of suspicion toward research in general, expressing reluctance to, in their words, be “lab rats” in our “laboratory experiment.” Thus, care needed to be taken to minimize the chances of coercion or participation strictly for the initial financial compensation. Study personnel emphasized the commitment required for participation (attending treatment and/or follow-up evaluation sessions regularly). In addition, study personnel read the consent form aloud to the prospective participants and quizzed them on their understanding of what the research participation entailed, their rights as participants, and their chances of being randomized into the therapy condition. In addition, care was taken not to obtain consent from patients if they were heavily medicated or too emotionally distraught to give informed consent. In such cases, study personnel called or returned to the hospital after some time had elapsed and the patient was stabilized.

Some problems were encountered when patients were assigned to the usual care condition. The prospect of nocost mental-health treatment often motivated patients to participate in the study. Although study personnel explained that treatment assignments were completely random and generated by a computer, when some patients found out that they had not been assigned to therapy, they expressed a range of negative emotions, from sadness to frustration and anger. Some indicated that being assigned to the control condition reinforced negative beliefs about themselves and the world around them, and the research team became concerned that this aspect of the research might reinforce feelings of hopelessness among the participants in this condition.

In the clinical trial, the problem was addressed by offering the services of “study-case managers.” Case managers served many functions, as outlined below, but one primary function was to provide support for participants who did not receive active treatment. Case managers worked to help participants get treatment outside of the study and contacted participants throughout the follow-up period by mail and by telephone. Study participants in both conditions of the clinical trial reported a high degree of satisfaction with the case-management component of the study, and our research team became convinced that the case-management component is a crucial element of any effort to intervene with this population.

Other problems were encountered when attempting to complete the baseline assessment. If the evaluation was conducted on an inpatient unit, some patients were too heavily medicated and too somnolent to participate, and interviews had to be postponed until they were more lucid. The intake interview required about 2 hours to complete, and many patients experienced difficulty staying focused for that length of time because of the effects of both depressive symptoms and medications. Another significant problem was some patients’ tendency to disaffirm answers regarding their current suicide ideation or substance use in order to speed up the interview. In order to avoid any type of response bias as well as to counteract the effect of poor concentration, interviewers in the clinical trial were trained to ask open-ended questions as well as multiple follow-up questions. Increasing the amount of participation by the patient helped increase focus and decrease false responses.

Challenges with Tracking and Follow-Up

Difficulty was encountered in the pilot study tracking patients through the follow-up phase of the research as well as maintaining a regularity of sessions with patients in therapy. Locating patients once they had been discharged from the hospital was generally difficult and time-consuming. Many of the depressed patients in the study had failed to maintain contact with friends or relatives, and when they were discharged from the hospital, they often found themselves without a place to live. In general, over the course of the research study, participants tended to be transient, with sporadic contact with friends and family. One of the methods used to target this problem in the final study was collecting a greater number of contacts for the participants. Upon providing informed consent, all participants were asked to provide their contact information as well as information of at least two friends or family members whom they would consent to be called about their whereabouts. These numbers were then verified prior to their entrance into the study, and contacts were asked to give permission to be contacted periodically if the patient could not be reached by other means. Generally, within appropriate bounds of confidentiality, rapport was established between the research team and the participant’s relatives and friends, greatly improving the ability to contact patients. Unless the participant signed a release allowing for more detailed information to be provided, all contacts were simply told that the study personnel were calling from the university to set up or discuss meetings with the patient. Additionally, the contact problem was improved in the final study by the use of a community voicemail system. Many participants in the study did not have their own phone, and messages communicated via friends and family were frequently not passed on. In the final trial, patients without stable contact information were given a voicemail box in which they recorded an outgoing message and could call from any phone to hear incoming messages. This aided in communication with the study as well as family and potential employers, and was extremely popular with the patients.

Case managers also played a large role in the successful follow-up of participants in the final study. Each patient, regardless of treatment condition, was followed throughout the study by his or her case manager, who kept in touch with the patient on a weekly or monthly basis by phone or mail. At each contact with the patient, their location and contact information was verified. Case managers also sent participants birthday and holiday cards. Participants reported that they greatly appreciated this contact and found it to be very beneficial. Many came to view the study as an important aspect of their life, and study staff were considered a trusted resource if they found themselves in crisis.

Because of the high incidence of substance abuse among our participants (68% were diagnosed with substance abuse or dependence), many were discharged from the hospital into rehabilitation programs or entered rehabilitation during the course of follow-up. When this occurred there were obstacles to conducting assessments with patients, because of confidentiality or treatment policies at the treatment programs. Several times study participants contacted us from the rehabilitation site to express willingness to do a follow-up interview over the phone, but it was not allowed by their facility. The same was true for patients who wished to attend sessions or follow-ups with study staff in person.

Similar problems occurred when study participants entered a homeless shelter. Homeless shelters appropriately keep client records confidential, and we were unable to locate patients who entered the system unless they contacted us. This was especially a problem when attempting to locate female study participants with children. When study staff became aware that study participants were planning to enter the shelter system, they provided the participants with multiple business cards with our name and phone number and urged them to call as soon as they were settled. Participants were also provided subway tokens to drop by the office when they were able to do so. This improved attendance, but future suicide research would be enhanced by collaboration with multiple treatment and service facilities.

Challenges with Therapy Attendance

Many patients were reluctant to commit to treatment or to establish treatment goals, a pattern not unusual for depressed individuals. Regular attendance of psychotherapy sessions was often difficult for study participants. At times, their psychopathology was characterized by impulsivity and an inability to use available supports and community resources effectively. Rather than going to outpatient services (or contacting study staff), the hospital emergency department was often a first point of contact when they experienced physical or emotional distress. Avoidance was also an important clinical issue to consider. Many patients reported that they would avoid treatment sessions because they did not want to talk about the problems that had led up to their suicide attempt. Furthermore, re-engagement could be difficult after patients missed several consecutive therapy sessions because of feelings of guilt or awkwardness about having missed appointments with their therapist.

Such information was routinely synthesized into the conceptualization of the patient, but it was also crucial that therapists remained mindful of their own reactions to their patients. It would be easy to dismiss a noncompliant patient as “unmotivated” and to decrease efforts to engage them, but such interpretations of therapy-interfering behaviors did not fully capture the complexity of the patients’ experiences. Many of the patients in our study led very chaotic and unpredictable lives, and they experienced many genuine, practical barriers to regular attendance of therapy sessions. Because of the transience and unpredictability of their lifestyles, patients would often lose our contact information or forget appointments. It was also not uncommon for something unexpected to happen that precluded attendance at the last minute, such as unpredictable work schedules and childcare difficulties. Initially some patients also had difficulty understanding directions to their appointment or navigating public transportation if our center was not in a part of town they generally visited. In addition, because some cultural and familial values do not support help-seeking behaviors, patients could not count on help from their families or friends for childcare or transportation to appointments, and at times they may have been discouraged from seeking treatment at all. Furthermore, individuals who live in poverty often find themselves in the position of prioritizing their most immediate needs. If they were given a subway token to attend therapy, but had a more pressing need to use the token to attend a meeting with a parole officer, housing coordinator, or social services benefits coordinator, it was not uncommon for them to do so and miss the appointment with the therapist.

Based on these lessons learned in the pilot study, several strategies were adopted in the clinical trial to increase therapy attendance. The first treatment session was conducted in the hospital whenever possible in order to establish rapport and maximize the likelihood of patients transitioning to sessions in our office. Patients were given explicit verbal and written directions to our office, including maps and customized public transportation routes. Case managers also made reminder calls both the day before and the day of appointments and worked with patients to address any barriers that might prevent them from attending. In addition, more explicit descriptions were given to patients about their expected role in therapy. Therapists were encouraged to keep one regular weekly time for each patient, so that even if appointment cards were lost, patients would might recall that were due to come in every week at the same time. For patients with inconsistent work schedules, therapists were encouraged to have a set weekly phone conversation and schedule appointments week-by-week during this call. And, patients were not discouraged from bringing their children to the office when necessary.

Nevertheless, despite the implementation of these strategies, additional flexibility was necessary until patients became accustomed to attending therapy regularly and were able to problem solve effectively enough to make their appointments. Attendance was boosted in the final study by having therapists on the full-time staff so that patients could be seen whenever they were came into the center. Mindful of the genuine barriers and difficulties faced by the clients, therapists worked to balance impressing upon patients the importance of attending regularly while validating patient feelings and circumstances. An important part of the treatment protocol was an emphasis anticipating obstacles to attending appointments and problem-solve how to handle them. This aspect of treatment may have served to help patients learn new ways of coping with practical problems and prevent such problems from snowballing into a crisis.

Summary and Integration

The current paper describes problems encountered in implementing a randomized controlled trial with urban suicide attempters. Problems were encountered in many areas, including difficulties with attendance and severe financial and social problems that led to the absence of a fixed residence or social support network. Overall, in terms of treatment implementation, the most successful step taken to increase attendance in the final study was to work with each therapy patient as an individual. Therapists in the clinical trial were encouraged to be flexible and involved with the patient, and most became the patient’s first point of contact when any crisis arose. Such availability is extremely difficult to achieve in community mental health settings, and Ghahramanlou-Holloway, Stirman, Brown, and Salsman (2008) described strategies described to increase attendance in therapy in effectiveness trials based in community-based mental health agencies.

Our experience to date suggests that treatments may be improved if clinicians in community mental health settings were able to implement a more flexible treatment model. For instance, it may be beneficial if, early in treatment, high-risk patients were able to go into the clinic if they were in need of support or (nonemergency) intervention without an appointment and wait for a slot in the therapist’s schedule (either when they are free or when the therapist has a cancellation or missed appointment). Therapists could emphasize that when the patient keeps his or her scheduled appointment time, their waiting time would be decreased, but that that it is better to reach out to the therapist than to not attend treatment at all. Another possible strategy for high-risk patients such as those who recently attempted suicide is implementing a walk-in clinic. Here, a small team of therapists make themselves available during a few hours per week or when they have missed appointments throughout the day. In all cases, one of the goals of treatment is to assist the client to develop the strategies and skills necessary to attend regularly scheduled sessions – which is only possible if the high-risk patient is suitably engaged in the first place. In our experience, flexibility is the key to engagement.

Problems and obstacles are likely to be present when conducting any longitudinal research with this or any similar population, and researchers should be aware of such problems and anticipate them prior to beginning their study in order to maximize patient retention and subsequent validity. Although, when working with high-risk patient populations, many challenges will remain despite the most thoughtful efforts to reduce barriers to attendance, our experience suggests that working to meet the needs and realities of study participants’ lives can result in better retention and better outcomes.

Biographies

Carly J. Gibbons, PhD, is a postdoctoral fellow at the VA CT Mental Illness Research Education and Clinical Center (MIRECC) and Yale University, New Haven, CT, USA. She received her doctorate from the University of Pennsylvania. Her research focuses on the dissemination of empirically supported treatments for mental disorders.

Shannon Wiltsey Stirman, PhD, received her doctorate in psychology from the University of Pennsylvania in 2005. She completed a postdoctoral fellowship in the Department of Psychiatry at the University of Pennsylvania. She is now at the Women’s Health Services Division, National Center for PTSD, and is an Assistant Professor at Boston University. Her research interests include the dissemination and implementation of evidence-based psychotherapies and interventions for depression, suicide prevention, and trauma.

Gregory K. Brown, PhD, is a Research Associate Professor of Clinical Psychology in Psychiatry at the University of Pennsylvania, Philadelphia, PA, USA. He is the Co-Director of the Center for the Treatment and Prevention of Suicide at the University of Pennsylvania. His research has focused on developing, evaluating, and disseminating targeted psychotherapy interventions for individuals indicated as being at high risk for suicide.

Aaron T. Beck, MD, University Professor of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA, is a graduate of Brown University (1942) and Yale Medical School (1946). The recipient of numerous awards and honorary degrees, he is the only psychiatrist to have received research awards from the American Psychological Association and the American Psychiatric Association. He is a senior member of the Institute of Medicine and recipient of its 2003 Sarnat International Award in Mental Health and 2007 Leonhard Award for contributions to Mental Health Services. He has also received the 2007 Lasker Clinical Medical Science Award, often regarded as “America’s Nobel Prize.” He is generally regarded as the “Father of Cognitive Therapy” and is president of The Beck Institute for Cognitive Therapy.