Serious adverse events in randomized psychosocial treatment studies: Safety or Arbitrary Edicts?

Abstract

Human subjects protection policies developed for pharmaceutical trials are now being widely applied to psychosocial intervention studies. This study examined occurrences of serious adverse events (SAEs) reported in multicenter psychosocial trials of the National Institute on Drug Abuse Clinical Trials Network. Substance abusing participants (N=1,687) were randomized to standard care or standard care plus either contingency management or motivational enhancement. Twelve percent of participants experienced one or more SAEs during the 27,198 person-weeks of follow-up. Of the 260 SAEs recorded, none were judged by the Data Safety Monitoring Board to be study related, and there were no significant differences between experimental and control conditions in SAE incidence rates. These data underscore the need to reconsider the rationale behind, and appropriate methods for, monitoring safety during psychosocial therapy trials.

The well-being of participants in clinical trials is of paramount importance, but systems to ensure safety are not well-developed for psychosocial trials. The Code of Federal Regulations (45CFR46) requires research plans make adequate provision for ensuring safety, and the Food and Drug Administration (FDA, 1995) defines adverse events as “any untoward medical occurrence in a patient or clinical investigation subject administered a pharmaceutical product and which does not necessarily have to have a causal relationship with the treatment.” The regulations were initially developed for medication trials (International Conference on Harmonization, 1996), but the National Institute of Health (NIH, 1998) issued a policy requiring all NIH-supported trials to have a Data and Safety Monitoring (DSM) Plan. In the absence of well-defined methods, many psychosocial trials began adopting medically oriented procedures.

While well intentioned, these guidelines were applied without review of appropriateness for psychosocial therapy studies, which have different adverse outcomes than medical trails. While psychotherapies are generally considered safe (Mays & Frank, 1985; Strupp & Hadley, 1977), some treatment-specific serious adverse events (SAEs) occur, such as iatrogenic effects of group therapy among deviant youth (Dishon, McCord & Poulin, 1999). With substance abusers, treatment-related SAEs may involve drug use following discussions about using in motivational enhancement (MET), and gambling problems in contingency management (CM) therapies using chance reinforcement procedures.

Substance abuse treatment trials may be an ideal ground to evaluate the utility of medically-oriented safety monitoring procedures in psychosocial therapy research. The National Institute on Drug Abuse Clinical Trials Network (NIDA CTN) evaluates efficacy of evidenced-based treatments in community clinics. In the first studies (Ball et al., 2007; Carroll et al., 2006; Peirce et al. 2006; Petry et al., 2005), two psychosocial therapies were evaluated: CM and MET. CM provides reinforcers (e.g., retail goods) for evidence of behavior change. MET is a verbal therapy style that bolsters commitment to change. Both of these interventions have applications to other psychiatric populations (Dunn, Neighbors, & Larimer, 2006; Kerwin, 1999).

Evaluation of SAEs in CTN studies could shed light on frequencies, types, and relevance of problems experienced by substance abusers in psychosocial trials. Substance abusers have elevated risks of infectious and heart diseases and cancer, and the impairing, illicit nature of drug abuse results in accidents, overdose, assaults, and violence (Adrian & Barry, 2003). Also, drug abuse often occurs with psychiatric disorders (Regier et al., 1990), which can be exacerbated by drugs and alcohol. When patients enter treatment, medical and psychiatric conditions are often identified, and referrals encouraged (McLellan et al., 1999). Hence, using the FDA definition of SAEs, the likelihood of uncovering medical, psychiatric and drug use hospitalizations is high.

One could argue that monitoring medical events is appropriate when a medication is under investigation that may have unknown side effects, particularly in populations in which the illness may increase vulnerability to adverse reactions. However, one would be hard-pressed to imagine how medical events can be causally related to psychosocial therapies. Data from the CTN studies provide an opportunity to explore the utility of this reporting system for detecting between-group differences in SAEs among participants in psychosocial therapy trials.

This report describes incidence rates and types of SAEs in over 1,600 substance abusers from CTN trials of psychosocial treatments. Incidence rates were compared across experimental and treatment-as-usual conditions to ascertain if interventions increased medical, psychiatric or substance use SAEs. SAEs were also examined in relation to demographics and drug use problems to determine if some subgroups experienced higher rates of SAEs than others.

Methods

Participants

Participants were 803 outpatients from 8 psychosocial and 6 methadone clinics for CM studies, and 884 outpatients from 10 psychosocial clinics for MET studies. Eligibility criteria are detailed in the main trials (Ball et al., 2007; Carroll et al., 2006; Peirce et al. 2006; Petry et al., 2005). Briefly, all participants were ≥18 years old. For CM studies, they self-reported or tested positive for stimulant use, and for MET studies, they were initiating outpatient treatment. Exclusion criteria were in recovery for pathological gambling (CM studies) and psychiatric, medical, or residential instability (MET studies). Participants provided written informed consent.

Study procedures

Participants completed a 1.5 hr interview using an abbreviated form of the Addiction Severity Index (ASI; McLellan et al., 1985). In CM studies, a DSM-IV checklist for drug use diagnoses was used. In MET studies, the Substance Dependence Severity Scale (Miele et al., 2000) ascertained diagnoses for which participants sought treatment.

The ASI was re-administered at months 1, 3, and 6 in CM studies, and months 1 and 3 in MET studies. Regardless of treatment assignment, all participants were assessed at similar intervals and rates in each study, and CONSORT standards were applied, as noted in the main trials. All participants were included in analyses of SAEs, with data presented as person-weeks.

Monitoring of SAEs

Protocols adopted the FDA (1995) definition of SAEs: any event that is life-threatening, causes inpatient hospitalization, creates persistent disability or incapacity, produces congenital anomaly/birth defect, or results in death. SAEs also included any event possibly related to therapies or assessments (e.g., increases in gambling with CM, or drug use after follow-up compensation). At each study contact, research assistants and therapists documented SAEs, recording affirmative responses to items about emergency room visits, inpatient admissions, etc. The SAE form included reasons for hospitalization/event, dates, and relation to study. Within 24 hours, forms were faxed to principal investigators, local monitors, a medical safety officer, and DSM Board, comprised of 7 members (3 MDs, 1 statistician and 3 Ph.Ds with clinical trials experience). Internal quality assurance monitors and external auditors monitored sites regularly. No protocol violations were noted regarding SAE procedures.

Treatments

In CM studies, a computer program randomized participants within sites to: (a) Standard care (SC; counseling and methadone, if applicable, and twice-weekly urine and breath samples), or (b) Prize CM (SC with chances to win prizes ranging from snacks to phones and stereos when testing negative for stimulants and alcohol). The only potential concern about prize CM is its potential similarity to gambling, but no increases in gambling were noted (Petry et al., 2006). In MET studies, participants were randomized to standard care or SC and MET. In the Carroll et al. (2006) study, MET was provided in a single session; in the Ball et al. (2007) study, 3 sessions of MET were compared with 3 sessions of SC. Discussions of advantages of using drugs could potentially increase use, but no documentation of this risk exists.

Data analysis

Frequency tables were created for SAEs, and incidence rates were calculated. Incidence rates are based on ‘person weeks’ of follow-up, defined as total number of weeks from baseline until final study contact. This variable was determined from the maximum of: last follow-up, last study treatment visit, or final contact in which an SAE form was filed, even if no other study forms were completed (e.g., SAE uncovered during attempts to schedule a follow-up).

Incidence density ratios (IDR) were generated, and refer to SAE count per person-weeks of follow-up in one group divided by SAE count per person-weeks of follow-up in the other. For example, if 20 events were noted in 1000 weeks of follow-up (0.02%) for one cohort versus 5 events in a cohort with only 500 weeks of follow-up (0.01%), the IDR would be 2, or a doubling of events. Main analyses compared experimental (CM or MET) and SC groups. The normal approximation to the binomial evaluated significant IDRs (p < .05, two-tailed).

Analyses also ascertained sensitivity of this approach in detecting group differences in IDRs. Comparisons were: women vs men, African Americans vs Caucasian, younger vs older (median split of age), and participants with stimulant, opioid, alcohol and marijuana use diagnoses vs those without. In Carroll et al.'s (2007) study, formal diagnoses were not made, so those seeking treatment for a specific drug class were compared with those seeking treatment for other drugs. Analyses were conducted using Excel and SPSS (Chicago, IL).

Results

In CM studies, 803 participants contributed to 15,574.8 person-weeks of follow-up. In total, 688 persons (85.7%) did not report any SAEs, 86 (10.7%) experienced one SAE, 22 (2.7%) had two, six (0.8%) had three, and one (0.1%) had five SAEs. The incidence rate was 0.01% per person-week. In total, 96 SAEs occurred during the 3-month treatment and 57 at follow-up; the resultant incidence rates were 0.01% during treatment and 0.01% at follow-up.

Table 1 shows baseline variables of participants who experienced one or more SAE, grouped by different characteristics. It also depicts weeks of person-time for each grouping, along with IDRs. An IDR >1.0 represents an excess of SAEs relative to the reference group, after accounting for person-weeks, and IDRs <1.0 show reduced probabilities. There was no association between treatment and SAE incidence rates, with an IDR near equivalence (1.05).

Table 1. Distributions of contingency management study participants and person-time by demographic, substance use and treatment variables (N=803).

Variable	≥1 Serious Adverse Events	Person-weeks of follow-up	Incidence Density Ratio	p value
Treatment assignment, N (%)
Standard care	63 (15.5%)	7537.1
Standard care plus contingency management	52 (13.1%)	8037.7	1.05	0.99
Gender, N (%)
Male	39 (9.7%)	7674.0
Female	76 (19.0%)	7900.8	1.94	<.001
Race, N (%)
European American	33 (12.8%)	4596.0
African American	69 (17.5%)	8098.7	0.87	0.19
Age, N (%)
<=38 years	55 (13.8%)	7295.1
39 years or older	60 (15.0%)	8234.0	0.88	0.16
Stimulant diagnosis, N (%)
No	21 (15.8%)	2789.5
Yes	94 (14.1%)	12785.3	0.97	0.61
Opioid diagnosis, N (%)
No	56 (12.7%)	8063.4
Yes	57 (18.9%)	6524.1	1.33	0.002
Alcohol diagnosis, N (%)
No	93 (16.6%)	11238.3
Yes	22 (9.1%)	4336.5	0.58	<.001
Marijuana diagnosis, N (%)
No	94 (14.0%)	13155.6
Yes	21 (17.8%)	2156.0	1.19	0.99

Notes: In the case of race, other categories were too small for independent analyses of other racial/ethnic groups.

^aIncidence density ratios are calculated by taking the total number of SAEs in one category divided by the person weeks of follow-up in that category, and dividing this number by the total number of SAEs reported for the other category, divided by the person weeks of follow-up in that category. The middle column reports the number of participants who experienced an SAE in each category. Note that some individuals experienced more than one SAE.

Although treatment effects were not noted with respect to SAEs, 3 baseline variables were related to incidence of SAEs. Females had significantly higher rates than males; an opioid diagnosis was positively, and an alcohol use diagnosis was inversely, associated with SAEs.

Most SAEs (Table 3) were for medical hospitalizations (N=79, 51.6%) or for drug treatment (N=58, 37.9%). Psychiatric admissions comprised 16 (10.5%) SAEs, usually for suicidality, and in three cases for psychosis (all one person). IDRs were calculated for the three SAE categories, comparing experimental and standard conditions. None were significant, with IDRs of 0.91, 1.15, and 1.56 for medical, drug and psychiatric SAEs, respectively.

Table 3. Specific types of Serious Adverse Events experienced by participants.

	Standard care	Standard care + CM	Standard care	Standard care + MET
Medical
Accident	0	2	1	1
Skin (abscess, cellulitis)	1	1	0	0
Cardiac (angina, heart failure, defibrillator, hypertension, myocardial infarction)	2	6	6	3
Respiratory (asthma, chronic obstructive airway, lung cancer, pulmonary embolism)	2	5*	2	1
Infection (fever, influenza, pneumonia)	14	7	1	4
Surgery (appendectomy, biopsy, hysterectomy, splenectomy, back)	4	5	3	2
Childbirth (and complications)	4	6	2	1
Diabetes (and complications)	4	0	1	0
Seizure, stroke	2	1	2*	1
Other (biliary colic, bleeding, gastrointestinal, hematemesis, kidney stones, pancreatitis, renal failure)	7	6	4	2
Medical total	40	39	22	15
Substance use
Alcohol poisoning	0	0	0	1
Detoxification	23	30	22	16
Residential treatment	2	2	2	6
Drug overdose	1*	0	1	1
Substance use total	26	32	25	24
Psychiatric
Aggression	1	0	1	0
Depression	1	3	4	1
Psychosis	0	3	0	0
Self injurious behavior	0	0	1	1
Suicidal ideation	0	3	3	2
Suicidal gesture	0	0	0	1
Suicide attempt	2	1	1*	4
Unspecified	2	0	2	0
Psychiatric total	6	10	12	9

^*One participant died from such event.

CM = Contingency management; MET = Motivational Enhancement Therapy

In MET studies, 884 participants contributed to 11,619 person-weeks of follow-up, with 107 SAEs recorded and an incidence rate of 0.01% per patient week. In total, 795 persons (89.9%) reported no SAEs, 75 (8.5%) had one SAE, 10 (1.1%) had two, and 4 (0.5%) had three. Thirty-six SAEs occurred in the first month, and the rest at follow-up, with incidence rates of 0.01 for each time period. Table 2 shows percentages of participants experiencing SAEs and IDRs.

Table 2. Distributions of motivational enhancement/interviewing study participants and person-time by demographic, substance use and treatment variables (N=884).

Variable	≥1 Serious Adverse Events	Person-weeks of follow-up	Incidence Density Ratio	p value
Treatment assignment, N (%)
Standard care	47 (10.2%)	6105.5
Standard care plus motivational enhancement	42 (9.9%)	5513.9	1.11	0.81
Gender, N (%)
Male	52 (9.1%)	7884.4
Female	37 (11.9%)	3735.0	1.53	<.001
Race, N (%)
European American	48 (9.7%)	6110.5
African American	18 (7.7%)	3547.2	0.89	.82
Age, N (%)
<=34 years	41 (8.7%)	5783.3
35 years or older	48 (11.6%)	5836.1	1.26	0.06
Stimulant diagnosis, N (%)
No	57 (9.5%)	7822.5
Yes	32 (11.1%)	3796.9	1.55	0.99
Opioid diagnosis, N (%)
No	68 (8.3%)	10883.8
Yes	21 (33.3%)	735.6	4.28	<.001
Alcohol diagnosis, N (%)
No	59 (11.8%)	6021.3
Yes	30 (7.8%)	5598.1	0.67	0.004
Marijuana diagnosis, N (%)
No	78 (11.0%)	9425.1
Yes	11 (6.4%)	2194.3	0.76	0.99

Notes: In the case of race, other categories were too small for independent analyses of other racial/ethnic groups.

^aIncidence density ratios are calculated by taking the total number of SAEs in one category divided by the person weeks of follow-up in that category, and dividing this number by the total number of SAEs reported for the other category, divided by the person weeks of follow-up in that category. The middle column reports the number of participants who experienced an SAE in each category. Note that some individuals experienced more than one SAE.

No significant differences in SAE rates occurred between treatment conditions. However, females experienced significantly higher rates of SAEs than males. Participants seeking treatment for opioids had higher rates than those not seeking treatment for opioids and those seeking treatment for alcohol had lower SAE rates than those not seeking treatment for alcohol.

The modal type of SAE (Table 3) was for substance use (N=49, 45.8%). Medical (N=37, 34.6%) and psychiatric events (N=21, 19.6%) also occurred. In examining types of SAEs by condition, all IDRs were non-significant; IDRs were 1.06, 0.76, and 0.83 for respective events.

Discussion

SAEs in experimental conditions were not higher than in SC even though there was adequate power to detect small between-group differences. Investigators, DSM Boards, and NIDA medical monitors considered all SAEs to be “unrelated” to study interventions, suggesting these therapies are safe. These data underline high base rates of events considered by regulatory agencies as “serious” among substance abusers. Rates of SAEs in this study were higher than those reported in an Australian study (Digiusto et al., 2004) that did not count “elective” detoxifications as SAEs. If such events were excluded from this trial, overall rates would be similar. In both studies, many SAEs related to medical problems. The average participant age was mid-thirties and pregnancy hospitalizations were common. Others reflected complications of asthma, heart disease, etc., and it is difficult to imagine how these diseases could be causally related to study interventions.

Substance abusers have high rates of relapse and psychiatric disorders, including suicidality (McLellan et al., 2000; Conner, Hesselbrock & Schuckit, 2006). About half the SAEs related to such symptoms, but these events occurred equally among participants assigned to experimental and control interventions. These data are consistent with the only other known report of SAEs in substance abusers. Schroeder, Schmittner, Epstein and Preston (2005) found IDRs for CM, cognitive-behavioral therapy and combined therapies relative to SC were all non-significant.

Differences in SAE incidence rates were noted in some groups based on demographics and primary drug problems. In this study and another (Schroeder et al., 2005), women evidenced more SAEs than men, consistent with higher rates of medical and psychiatric treatment-seeking among women (Middleton & Hing, 2006). Individuals with alcohol use disorders had lower SAE incidence rates than those with other drug problems. Conversely, patients with opioid problems evidenced high rates of SAEs, consistent with medical comorbidities prevalent in opioid-dependent patients (Masson et al., 2002). Although there was adequate power to detect small between-group differences, no such differences were noted between standard and experimental psychosocial therapy conditions, suggesting these therapies are safe.

One could argue that 260 SAEs in 1,687 patients suggests SAEs are fairly infrequent, and hence monitoring SAEs in psychosocial trials should be similar to pharmacological trials. However, each SAE required 1-2 hours to document along with time for the DSM Boards and Institutional Review Boards (IRBs) to review them. If some of these events were no longer mandated for reporting (e.g., clearly unrelated medical hospitalizations), substantial time could be saved without increased risk to participants.

Moreover, individual IRBs create and mandate their own monitoring systems, now including a broader category of Adverse Events (AEs), which may include endorsements of mild or transient thoughts of suicide and “urge to break things.” Instead of a 12% incidence rate, some trials now document hundreds of events, with AEs uncovered at most follow-ups, even though the likely relation to a prior psychosocial treatment is remote. IRBs are being inundated by AE and SAE reports (Califf et al., 2003), many of which simply add noise and detract from the importance of discerning events that may be related to therapies under investigation. Consensus is needed between the FDA, NIH and IRBs to better streamline and standardize this process.

Clearly, studies of psychosocial interventions should collect, report, and monitor study and population-relevant SAEs in a standardized way. The present system evaluates occurrence of problems, but it is unclear that it addresses issues and events related to the safety of interventions. Findings are limited in that only two interventions were evaluated, participation was restricted to substance abusers, treatments were brief, and duration of follow-up was only 3-6 months. These results, albeit not the exact incidence rates, are likely relevant to some other psychosocial therapy trials for psychiatric and medical populations.

Czaja et al. (2006) reported that investigators of psychosocial intervention trials involving Alzheimer patients and their caregivers are likewise unclear about definitions and classifications of adverse events and their “resolutions.” Given the uncertainties of the present system, wide variability exists in practices for reporting adverse events, and their study also called for the need for greater clarity in monitoring procedures for psychosocial treatment trials.

Data from the present report were obtained from large studies representing a diverse group of clinics and patients throughout the country. Local investigators, study monitors, IRBs, NIDA CTN medical safety officers, and independent DSM Boards did not classify a single SAE in this extensive database to be study related. The absence of related SAEs or any differences in SAEs in experimental versus SC conditions confirms the safety of these therapies. These data show that using medical safety procedures in psychosocial research is of limited, if any, value in protecting patient safety. It seems ironic that a system that emphasizes cost-benefit ratios requires monitoring of unrelated medical events that increase costs without benefit. We need a more rational system for reporting adverse events in psychosocial therapy trials.