Participation Bias among Suicidal Adults in a Randomized Controlled Trial

Abstract

Although individuals who attempt suicide have poor compliance rates with treatment recommendations, the nature and degree of participation bias in clinical treatment research among these individuals is virtually unknown. The purpose of this study was to examine participation bias by comparing the demographic and diagnostic characteristics of adult suicide attempters who participated in a randomized controlled trial to a sample of nonparticipants. Results indicated that males and individuals with a diagnosis of substance abuse or dependence were more likely to be participants in the randomized controlled trial. The implications of these findings for suicide intervention research are discussed.

The generalizability of findings from clinical trials is based on the assumption that research participants represent the population from which they are sampled. Biases from differential patterns of study enrollment among patient subgroups can lead to overestimates or underestimates of the effectiveness of an intervention. A critical research aim of effectiveness studies, therefore, is to examine the procedures and characteristics associated with participant recruitment. To understand the current limits of our knowledge of what works with whom, it is important to determine differences that exist between participants and nonparticipants in suicide prevention research. The examination of those factors associated with eligible participants’ refusal to participate in clinical trials, especially among individuals who are at risk for suicide, has important public health significance and can lead to more clinically representative research.

The nature and degree of participation bias among individuals who attempt suicide or intentionally injure themselves and who are recruited for clinical outcome studies is virtually unknown (for review, see Arensman et al., 2001). For example, among 32 randomized controlled trials with individuals who attempted suicide or deliberately harmed themselves, only 10 studies reported the proportion of individuals who refused to participate (Allard, Marshall, & Plante 1992; Battaglia et al., 1999; Evans et al., 1999; Guthrie et al., 2001; Hawton et al., 1981; MacLeod et al., 1998; Rotheram-Borus, Piacentini, Cantwell, Belin, & Song, 2000; Verkes et al., 1998; Waterhouse & Platt, 1990; Welu, 1977). For these studies, the proportion of eligible patients who refused to participate in the clinical trials ranged from 0% (Waterhouse & Platt, 1990) to 49% (Allard et al., 1992). Only one study examined the factors that may be associated with participation bias and found that there were no significant differences in demographic variables between the study participants and the study refuses (Welu, 1977). Additionally, to date, diagnostic differences between participants and nonparticipants in suicide research have not been adequately characterized.

Research on treatment compliance in nonresearch settings indicate that the extent of participation bias for individuals who attempt suicide may be substantial. Adults who attempt suicide and who are evaluated in emergency departments or other hospital settings have a very poor compliance rate, with follow-up outpatient appointments ranging from 20% to 40% (Jauregui, Martinez, Rubio, & Santo-Domingo, 1999; Kreitman, 1979; Morgan, Burns-Cox, Pocock, & Pottle, 1975; O’Brien, Holton, Hurren, & Watt, 1987). Low compliance rates have also been observed with adolescents who attempted suicide (King, Hovey, Brand, & Wilson, 1997). Such findings suggest that many individuals who attempt suicide would be unlikely to seek out or participate fully in interventions that are conducted through research.

To date, there is little evidence that specific diagnoses relate to the decision to participate or decline participation in research. However, research involving individuals being treated for psychiatric problems may be particularly vulnerable to participation bias (Patten, 2000; Vanable, Carey, Carey, & Maisto, 2002). Both symptom presentation and distress level may impact the decision to participate in research, particularly after a suicidal crisis. Those who are in greater distress or struggle with a greater number of psychiatric difficulties might feel more inclined to participate in research if they believe that the treatments being examined may reduce their distress. On the other hand, such individuals may feel overwhelmed and may be less inclined to participate in research that they believe involves too much of their time or energy. Research findings to date have been mixed regarding severity and participation in research. Candilis Geppert, Fletcher, Lidz, and Appelbau (2005) found that individuals with schizophrenia who had lower levels of severity were more likely to participate in clinical trials. Individuals with more severe psychosis were less likely to participate in psychiatric research in one study (Haapea et al., 2007). Other research has indicated that individuals with schizophrenia may be more likely to participate in low-risk research (Jaskiw et al., 2003). Individuals with comorbid diagnoses may be more likely to seek treatment (Tomasson & Vaglum, 2007; Wu, Kouzis, & Leaf, 1999), and participation may increase with severity of disorders (Shadish, Matt, Navarro, & Phillips, 2000). In addition, individuals who abuse substances may be more likely to participate in longitudinal research (Vanable et al., 2002). Higher severity was shown to correspond with more full research participation in an effectiveness study for alcohol dependence (Strohmetz, Alterman, & Walter, 1990).

Although data are scarce regarding differential participation in research among individuals who attempt suicide, demographic factors have been associated with lower rates of study participation in other literatures. Some research has suggested that African Americans are less likely to participate in medical and mental health research than individuals of other racial and ethnic backgrounds, possibly due to mistrust of the medical and research community, poor access to care, cultural barriers, and a failure to actively recruit African Americans (Shavers-Hornaday, Lynch, Burmeister, & Torner, 1997). In a randomized controlled trial of cognitive therapy for individuals who had attempted suicide, however, African Americans were less likely to refuse participation than individuals of other races (Brown et al., 2005), but the researchers were not able to determine reasons for the differential participation rates. There is also evidence to suggest that among individuals with serious mental illness, women may be less likely than men to participate in research (Mohr & Czobor, 2000; Robinson, Woerner, Pollack, & Lerner, 1996). Some research has suggested that symptom severity and social circumstances (e.g., homelessness, poverty, etc.) may lead to differential participation among patient subgroups. Individuals with fewer social resources have been found to be less likely to enroll in mental health by some researchers, (Farmer, Jackson, Camacho, & Hall, 2007; Perez, Ezpeleta & Domenech, 2007). Others have observed lower socioeconomic status among participants in substance abuse treatment trials (Rychtarik, McGillicuddy, Connors, & Whitney, 1998; Strohmetz et al., 1990) relative to individuals who decline participation in such research.

The assessment of bias in clinical trial samples through interview or survey research is challenging, as many nonparticipants refuse to speak with researchers or cite reasons for nonparticipation. In the present study, we sought to extend existing research on participation and selection bias by comparing the diagnostic and demographic characteristics of participants in an intervention study with those of nonparticipants who appeared to be eligible for participation, based on de-identified information obtained from medical records from the site at which recruitment took place.

METHOD

With the approval of the University of Pennsylvania Institutional Review Board, a review was conducted on electronic medical records of adult patients who attempted suicide and presented at a University of Pennsylvania emergency department, crisis response center, or inpatient psychiatric unit during the recruitment period for a study of cognitive therapy for suicide attempters (March 2002–December 2007). Data of individuals whose records indicated a probable suicide attempt were compared with the inclusion and exclusion criteria. The inclusion criteria were: (a) attempted suicide within 48 hours of presenting to an emergency department or trauma care unit, (b) age 16 or older, (c) able to speak English, and (d) able to understand the nature of the study and provide written informed consent. Exclusion criteria included: (a) self-mutilating behavior (e.g., burning oneself with a cigarette) without any intent to commit suicide, (b) an incapacity to participate in the study because of an acute, unstable, or severe Axis III disorder (e.g., organic brain damage), (c) a severe Axis I disorder that would prevent participation in outpatient psychotherapy (e.g., acutely delusional), or (d) Axis II disorder of mental retardation that would prevent an understanding of study procedures and participation requirements. Ninety-nine patients who appeared to meet inclusion criteria based on available information were included in the nonparticipant sample. Diagnostic and demographic data from the medical records of 140 patients who were entered into the clinical trial were used for comparison. After participants or eligible nonparticipants were identified in the medical record review, data were de-identified and the following data were obtained from the charts: age, gender, employment status, marital status, type of employment (if applicable), race, DSM–IV diagnoses (Axis I–IV), past suicide attempts, past hospitalization, housing status (homeless vs. not), recruitment site, and the method of suicide attempt.

RESULTS

The percentages of participants and nonparticipants by characteristics that were hypothesized to relate to study participation are presented in Table 1. The variables that were found to be significantly related to participation were gender, substance dependence, and psychiatric comorbidity (other than substance dependence). No other diagnostic or demographic differences were found between participants and nonparticipants. To determine the factors that were most uniquely associated with participation or nonparticipation, the three factors that were independently related to participation were entered into a logistic regression model. As indicated in Table 1, the overall model was significant, and substance use and gender were found to be significantly associated with participation. Odds ratios resulting from the logistic regression indicate that individuals with substance dependence diagnoses were 64% more likely to be study participants than those without a diagnosis. In addition, females were 97% less likely to be study participants than males.

TABLE 1.

Chi-Square and Logistic Regression Results for Study Participants and Nonparticipants (N = 239)

	Study participants (n = 140)		Nonpartici- pants			Logistic regressiona
	n	%	n	%	Wald χ2	Odds ratio	95% CI	Wald χ2
Caucasian	53	38	41	41	1.12	–	–	–
Female	69	49	67	68	7.99**	1.97	1.12– 32.48	5.56*
Homeless	3	4	12	10	3.07	–	–	–
Employed	10	17	11	23	.46	–	–	–
Attempt by overdose	106	76	71	72	4.21	–	–	–
Substance abuse/dependence	104	74	47	47	17.92*	.36	.18–71.71	8.64**
Psychosis	13	9	7	7	.37
Axis I comorbidity	122	87	74	75	6.04*	.87	.37–2.06	.13

For all individual tests, df = 1; CI = confidence interval; for all variables in the model, 0 = variable listed was false, 1 = variable listed was true; due to missing data, n’s for homelessness and employment status are less than 239.

^aOverall model χ² = 23.58, p < .01, df = 3, N = 239.

^*p < .05;

^**p < .01.

DISCUSSION

This study extends the current literature by examining both diagnostic and demographic differences between participants and nonparticipants in a clinical trial among individuals who have attempted suicide, using a method that is intended to draw a representative sample of nonparticipants who presented for emergency care during the recruitment period of the study. Charts of a diverse sample of individuals who presented following a suicide attempt at a large, urban hospital system were evaluated. Consistent with previous research examining participation bias among nonsuicidal individuals (Mohr & Czobor, 2000; Robinson et al., 1996), females were less likely to become research participants than males. Substance dependence was the only diagnostic variable that was found to be significantly associated with participation. These findings suggest that males and individuals with substance abuse or dependence diagnoses may be overrepresented in suicide intervention studies. Reasons for this overrepresentation cannot be discerned from the available data, but potential reasons include a perceived high need for intervention among individuals who use substances, particularly if problems related to substance use precipitated the attempt. A potential alternative explanation that must be considered is that those who misuse substances had an external incentive to pursue financial compensation for time spent participating in study-related assessments, in which case researchers targeting suicidal individuals in future studies should consider appropriate compensation carefully. The provision of compensation and rewards for participation may yield a sample that is more representative of a broader population in terms of motivation (Sharp, Pelletier, & Levesque, 2006), but may lead to an under-representation of the motivation level and different characteristics of individuals who decline treatment or engage in treatment outside the context of a study after a suicide attempt. Further efforts are warranted to examine the characteristics and reasons for refusal among those eligible individuals who decline participation in suicide research. Qualitative and quantitative data obtained through interviews and other research formats would help to provide additional context for the findings reported here.

We note some important limitations to this study. First, the findings are limited to individuals who were evaluated at an emergency department or trauma care unit of a hospital and may not generalize to those who attempt suicide but do not receive medical evaluations, or those who had such acute injuries that they were not medically stable enough to speak with a researcher within the window of time specified for recruitment. Furthermore, as the current study was conducted in an urban setting, it will be important to examine potential differences between urban and nonurban samples in future suicide research. Complete data were not available for two of the variables studied, homelessness and employment status, resulting in a smaller samples size for these analyses. Further research will be needed to determine whether differences in socioeconomic status exist between those who participate in suicide research and those who do not. Another limitation is that associated with the use of medical records rather than standardized diagnostic interviews to obtain diagnostic information. Despite these limitations, administrative data offer rich opportunities to estimate the applicability of the treatment outcome literature to a large sample of patients, and can inform the development of treatments for under-studied populations, particularly when used in conjunction with other studies that estimate the applicability of the literature using other methods.

Our results indicate that caution should be used in considering the effectiveness of suicide intervention research for underrepresented populations. Additional efforts to increase the likelihood of participation are needed to specifically target females and those with diagnoses other than substance disorders who may be more reluctant to enter suicide intervention trials. To increase the extent to which findings can generalize to clinical populations, it will be important to explore potential differences in secondary analyses and target underrepresented individuals in future research. The development of innovative methods for examining sample bias among clinical populations will be central to making further progress in understanding the nature of sample bias in specific clinical populations.