How Representative are Insomnia Clinical Trials?

Abstract

Study Objectives:

To address the question of how representative subjects studied in hypnotic clinical trials are of the broader insomnia population, this study assessed initial contact rates and reasons for inclusion and exclusion during recruitment to an efficacy trial and to a safety trial of Food & Drug Administration (FDA) approved hypnotics.

Methods:

Otherwise heathy persons meeting Diagnostic Statistical Manual, Fourth Edition, Revised (DSM-IVR) criteria for insomnia were recruited. In one study, persons 32-65 yrs, were invited to a 12 month trial of nightly use of zolpidem or placebo. In the other persons 21-64 yrs with driver’s licenses were recruited to test the effects of a hypnotic on live on-the-road driving ability. In both studies screening was conducted through an initial telephone interview followed by a clinic visit.

Results:

In the United States (US) study 13% (n=410) of 3180 initial contacts and in the Netherlands (NL) study 67% (n=53) of the 79 initial contacts proceeded to the clinic visit. Of those at clinic 25% of US and 37% of NL participants failed to meet additional insomnia criteria. Mental health exclusions accounted for 24% of US and 23% of NL participants and medical problems accounted for 23% of US and 9% NL exclusions. Finally 20% of US and 26% of NL participants were excluded for drug use/abuse histories. After all screening 4% of the initial US contacts and 0% of the NL contacts entered the study.

Conclusions:

These data suggest persons entering insomnia hypnotic clinical trials are a highly selected sample that is unlikely to be representative of the broad insomnia population or the population of potential medication users.

INTRODUCTION

Clinical trials of pharmacological treatments for insomnia are conducted in samples of convenience with selection occurring at both the participant and the investigator levels. At the participant level, studies are advertised and described in initial telephone contacts providing varying levels of detail. Thus, initially, prospective participants make decisions regarding their fit with broadly stated inclusion-exclusion criteria (e.g., disturbed sleep, medical status, allowed medications), the burdens (e.g., time commitment, possible side effects) and rewards (e.g., clinical improvement, financial reimbursement) associated with their further pursuit of study participation. Rarely, in reports of clinical trials are the advertisements described or the content of study details provided to the potential subjects in the initial screening contact. Consequently the number of initial contacts relative to the number that enter the clinic and sign consent forms for additional screening is not reported. Importantly, the reasons for participant loss from initial contact to clinical screen are unknown and hence the full nature of the sample cannot be determined.

At the next decision level, the investigator level, trials have somewhat restrictive inclusion and exclusion criteria. Current studies require that patients fulfill diagnostic criteria for insomnia disorder according to the Diagnostic and Statistical Manual of Mental Disorders, fifth edition, (DSM-5) or the International Classification of Sleep Disorders, third edition, (ICSD3) [1,2]. Studies previous to 2013 used the Diagnostic and Statistical Manual of Mental Disorders, fourth edition text revision (DSM-IVR) or the International Classification of Sleep Disorders, second edition (ICSD2), in which insomnia disorder was previously termed primary insomnia [3–4]. The insomnia disorder/primary insomnia classifications are used to distinguish the insomnia from an insomnia that is comorbid with other disorders. Prevalence estimates of insomnia in the population range from 10-40%, but most prevalence rates do not distinguish comorbid versus primary insomnia [5]. It is estimated that 75% of population-based insomnia is comorbid with medical, psychiatric, or other primary sleep disorders [6]. Thus, a trial excluding comorbid insomnia will be studying only a sub-sample of the insomnia population.

To provide an example of the participant and investigator level selection rates in previous studies, we did a search, not intended to be exhaustive, for papers reporting clinical trials of the FDA-approved hypnotics zolpidem [7], zolpidem XR [8,9], zolpidem sublingual [10], zaleplon [11] eszopiclone [12,13], doxepin [14,15], ramelteon [16,17], and suvorexant [18,19] and several agents investigated as potential hypnotics: gabapentin [20] and esmirtazepine [21,22]. None of these trials described the number of potential participants lost from the initial contact to the clinic screen. Most trials did provide information regarding the number screened versus the number randomized with the percent excluded ranging from 34% to 81%. Some trials made no mention of the screening exclusion rate [8,19].

Additionally, if trials include nocturnal polysomnographic (NPSG) outcomes, they often establish minimal NPSG criteria for minutes to sleep onset, minutes of wake after sleep onset and/or minutes of sleep time per time in bed (i.e. sleep efficiency). Insomnia disorder/primary insomnia is a history-based, not a NPSG derived, diagnosis. NPSG has yielded equivocal results regarding the nature and severity of sleep disturbance in insomnia when compared to subject estimates [23, 24]. Similarly, if self-report outcomes are employed in the trial, the insomnia disorder/primary insomnia inclusion criteria are often coupled with minima and maxima limitations on estimated minutes of sleep latency, wake time after sleep onset, sleep efficiency and regularity and duration of time-in-bed.

Thus, the question arises as to how representative the subjects studied in pharmacological clinical trials are of the broader insomnia population, or of the population which will ultimately be the users of the medication. We systematically counted initial contact rates and reasons for exclusion during recruitment to a five-year NIDA-funded zolpidem efficacy trial in chronic insomnia conducted in the United States (US) and during recruitment to a pharmaceutically-funded insomnia safety trial of ramelteon and zopiclone conducted in the Netherlands (NL).

METHODS

US Study

In the US study, persons (N=116), aged 32-65 yrs, meeting DSM-IVR criteria for insomnia and a PSG sleep efficiency of ≤85%, no other primary sleep disorders, no psychiatric diseases or drug dependency and in good health were recruited to participate in a 12 month clinical trial of nightly use of zolpidem 10 mg or placebo [25–27]. Advertisements in newspapers, hospital intranet news, and hospital clinics solicited individuals with chronic difficulty falling asleep, staying asleep, or awakening too early. The advertisements included the statement that participants would be reimbursed for their time, but no specific dollar amounts were included. The advertisements were approved by the Institutional Review Board (IRB).

Screening was conducted at two levels, an initial telephone interview followed by a clinic visit. The telephone interview was initiated with a description of the study purpose, procedures and duration, and a description of the drug or placebo that may be received. The study required that participants undergo 14 total laboratory nights and days of NPSG and Multiple Sleep Latency Tests (MSLT) distributed across months 1,4, 8, and 12. As well, participants were required to complete weekly sleep questionnaires via an interactive telephone questionnaire system (IVRS) during the 12 months.

If participant interest remained, the phone interview continued, guided by a screening questionnaire that queried regarding current medical/psychiatric conditions, use of prescription and over-the-counter (OTC) medications, sleep medications, and natural substances. Frequency and quantity of recreational drug use including caffeine, alcohol, marijuana and illicit drugs was assessed. Regularity of sleep habits and nightly frequency of sleep problems over the last three months were queried, including time of retiring and arising, total sleep time, difficulties falling asleep, maintaining sleep and awakening too early.

All subjects who passed the initial screening on the phone and were interested in continuing signed, at the beginning of the clinic visit, an informed consent approved by the IRB. The clinic visit included a brief physical, medical and drug use history, laboratory blood/urine testing including a drug screen, psychiatric screen, and a clinical 8-hr NPSG. Psychiatric screening was conducted by a trained research assistant, using the Structured Clinical Interview for DSM-IVR (SCID). The NPSG included the standard Rechtschaffen and Kales electrode montage for the scoring of sleep stages and the American Academy of Sleep Medicine procedures for monitoring and scoring airflow and leg movements [28,29]. NPSGs were scored by certified sleep disorders technicians. The NPSG entry criteria were a sleep efficiency of ≤85% and leg movement and apnea indices of ≤10 events per hour of sleep. The DSM-IVR primary insomnia diagnosis was established in a clinic interview by a sleep medicine specialist aided by the participant’s sleep questionnaire responses.

NL Study

In the NL study a sample of 79 potential participants with insomnia was identified through newspaper advertisements and in consultation with sleep disorders centers. Advertisements asked patients with chronic insomnia (21-64 years old) to participate in a clinical trial to test the effects of a new hypnotic drug on driving ability. The advertisement stated that those who were interested to apply should be healthy, non-drug using persons, who possess a valid driver’s license. It was mentioned that participants would receive a maximum of 1200 euros compensation after completion of the trial. The advertisements were approved by the local Institutional Review Board (IRB).

On the initial telephone contact the basic study entry requirements (i.e. age, possession of valid driving license, etc.) were verified and the study time commitment were described. The study involved sleeping overnight in a study-related residence on an adaptation night and on each of three treatment nights, separated by one week. Post sleep questionnaires had to be completed via an IVRS questionnaire on each of 7 placebo lead-in nights and on the three treatment nights. In the morning following each treatment night a live, on-the-road, 60 minute driving test was to be conducted.

At the clinic screening visit potential subjects first signed the IRB approved informed consent. Study inclusion criteria required a primary insomnia diagnosis based on DSM-IVR criteria with the addition of a reported history of sleep latencies ≥ 60 min and on the morning sleep questionnaire conducted during the 7-night placebo run-in and potential subjects were required to report sleep latencies of ≥45 min on 3 of 5 nights. Subjects had to report a habitual bedtime between 23:00 and 01:00 hrs. and willingness to remain in bed for at least 6.5 hrs during the study. Because the primary safety outcome was absence of morning residual sedation assessed with live on-the-road driving performance, potential subjects were required to have a valid driving license and have driven ≥5000 km per year for at least 3 years. During the study period subjects had to agree to abstain from driving their own vehicles outside of the live on-the-road driving performance assessment for the study.

Study exclusion criteria were subjects that are shift-workers, have flown across > 3 time zones within the past 7 days, and have other primary sleep disorders (i.e. sleep apnea, restless legs/periodic leg syndrome). Also excluded were subjects with clinically important abnormal findings as determined by a medical history, physical examination, ECG, or clinical plasma/urine laboratory tests. Subjects with a past six-month history of psychiatric disorders or a twelve-month history of alcohol or drug abuse, showing a positive urine drug screen or alcohol breath test, or unable to discontinue use of any of the disallowed drugs that might interact with the study drugs were also excluded.

RESULTS

US Study

In the US study , to enroll the N=116 study participants 3180 telephone interviews were conducted with 23% declining after hearing the study specifics. Of those with continued interest after hearing study specifics (n=2,449), 83% (n=2039), were excluded through the telephone screen (see Figure 1). Among those excluded on the phone screen 26% reported present (within past year) mental health problems, 22% nocturnal smoking or past/present drug/alcohol abuse, 19% chronic unstable health problems, 18% sleep disorders/BMI, and 15% did not meet DSM-IVR criteria for insomnia.

Figure 1.

Exclusions based on telephone screen. Mental Health = current-past year psychiatric condition, Chronic Disease = current uncontrolled medical disease, Drug/Alc Abuse = history of drug or alcohol abuse, No DSM Ins = no report of difficulty falling asleep, staying asleep or awakening too early.

At the in-person clinical screen, of the remaining n=410, 294 (72%) were excluded (see Figure 2). Among those excluded 30% did not report for NPSG, 22% failed the NPSG (i.e. Apnea Hypopnea Index (AHI) >10, Periodic Leg Movement Arousal Index (PLMAI> 10, or Sleep Efficiency (SE) >85%), 17% failed the SCID, 15% failed the health screen, and 16% the urine/drug screen or in-person reported drug/alcohol abuse. Thus, the final N=116 study participants represented 4% of the original contacts (n=3180) and 28% of volunteers (n=410) remaining after hearing study specifics, passing the phone screen, and consenting at the clinic visit.

Figure 2.

Exclusions based on clinic screen. No Show = did not report for clinic visit or nocturnal polysomnogram, Drug Scrn = urine drug screen was positive, Health = brief physical examination or laboratory blood/urine testing revealed abnormality, SCID = Structured Clinical Interview for DSM-IVR revealed psychiatric disorder, PSG = nocturnal polysomnogram revealed sleep efficiency of >85% or leg movement and apnea indicies of >10 events per hour of sleep

NL Study

In The NL study 79 persons were telephone screened. During initial screening, n=26 (33%) persons were excluded for practical issues and baseline demographics. Of them, n=9 were excluded because they either did not answer the phone, or had no further interest, or time to participate, n=2 were excluded because they did not speak Dutch, n=6 subjects were either too young or too old, and n=6 did not meet the pre-set driving experience criteria. Another 3 subjects were not willing to stop driving for the duration of the study.

At the clinic screen another n=53 (67%) of patients were excluded for various reasons: n=2 people did not meet the insomnia criteria, n=7 were engaged in shift work, n=2 reported having other sleep disturbances, and n=6 reported a sleep latency of more than 60 minutes, n=17 were excluded because they had comorbid psychiatric disease for which most of them received treatment (n=12) or had other health related issues (n=5). The other n=14 patients were not willing to stop their current treatment with hypnotic drugs. Of the n=5 subjects that were scheduled for the clinical 1-week placebo run-in, n=2 did not show up and were lost to follow up, n=3 subjects entered the single-blind 1-week placebo run-in. These subjects all failed due to a placebo response, (i.e. they had an improvement in subjective sleep latency of greater than 20 minutes). Thus, in the NL study 0% of the 79 potential participants entered the active study treatment phase.

Comparative US and NL Exclusions

Table 1 presents a comparison of the major classes of exclusion after fulfilling interest/feasibility and demographic criteria for the US (n = 2449) and the NL (n = 53) clinical trials, which represented 77% and 67% of the initial contacts for the US and NL trials, respectively. As seen in Table 1 for both studies 71% and 62% of the exclusions were for reasons other than not meeting study insomnia criteria. The most frequent non-insomnia exclusion in both studies was for current or recent mental health reasons (24% and 23%). In the US study 5% of those screened (4% of initial contacts) entered the study and in the NL study zero entered the active study phase; 6% of those consenting showed a placebo response during the placebo run-in.

Table 1.

Comparative Exclusions between the US and NL Studies

	US Study		NL Study
	Number	Percent	Number	Percent
Total Contacts	3180		79
Fill Initial Criteria1	2449	77%	53	67%
Failed Additional Screening
Insomnia2,3	600	25%	20	37%
Mental Health2	596	24%	12	23%
Medical2	554	23%	5	9%
Rx Use/Abuse2	495	20%	14	26%
Lost Phn to Clin2	88	4%	2	4%
Successful Study Enrollment
Entered Study	116	5%	0

¹Fulfill interest, feasibility, or demographic criteria upon initial contact

²Excluded after initial contact; percentages of those excluded among those remaining after initial contact (totals expressed for both phone and clinic level screens) (US n = 2449; NL n=53)

³NL study: includes failures on placebo run-in n=3

DISCUSSION

These data suggest persons entering insomnia clinical trials are a highly selected sample that is unlikely to be representative of the broad insomnia population or the population of potential medication users. They show that, in contrast to population insomnia which is primarily insomnia comorbid with other conditions, clinical trials are being carried out in primary insomnia patients who are not representative of the broader insomnia disorder population. In the US study only 4% of interested participants qualified and entered the study, while in the NL study the investigators were unable to qualify and enter any participants. This raises questions as to how to conduct studies which will produce meaningful clinical results regarding the efficacy and safety of sleep medications without compromising the generalizability of those results.

Unlike many clinical trials, the US and NL studies reported here the percentage of persons from the initial contact that were lost due to various participant level decisions or characteristics. In the US study 77% of initial contacts went on to further screening and in the NL 67% continued their screening. Reasons for the 23% and 33% loss in recruitment prior to the clinical screening is unknown. These potential participants were not re-contacted to query the reason for their loss from further participation.

At the investigator level, there were small differences between the two studies relating to the specifics of the inclusion criteria. Both the US and NL studies required that DSM-IVR clinical criteria for insomnia be met. The US study additionally required that a NPSG show a ≤ 85% sleep efficiency, yielding a 16% inclusion failure rate, while the NL study was more specific requiring subject-reported minutes to sleep onset minima and a placebo run-in to rule out placebo responders, which together yielded a 32% inclusion failure rate. Generally, such additional criteria beyond the DSM-IVR criteria are employed in clinical trials to select subjects whose insomnia is sufficiently severe or specific as to sleep onset versus sleep maintenance problems, so as to enhance the study potential of demonstrating medication efficacy, which is to avoid a floor effect.

The US and NL studies displayed generally comparable percentages of exclusion in the major exclusionary categories. Across these categories about 67% of interested volunteers were excluded and the highest exclusion rate in both studies was for mental health issues. It is well known that insomnia is frequently comorbid with psychiatric disorders and can be a symptom of relapse or the onset of a new disorder [6,24]. Unfortunately, the data in these two studies were not tabulated in such a way as to determine the percentage of those with insomnia complaints that had primary insomnia versus insomnia comorbid with a psychiatric disorder. For example, in the US study if a current or history of mental health problems was identified, further investigation of the nature of the insomnia complaint was not pursued. Overall the weakness in study generalizability for these two trials is quite obvious.

As to enhancing the generalizability of clinical trials of pharmacological treatments for insomnia, one strategy is to systematically conduct comorbid insomnia studies in patients with the various prevalent insomnia comorbidities. That strategy has been taken recently as studies of insomnia comorbid with depression, comorbid with generalized anxiety disorder, and comorbid with schizophrenia have been done [30–33]. The sleeping medication is used in these studies as an adjunct to the appropriate psychiatric disorder treatment. Interestingly, not only is the insomnia improved relative to placebo, but the treatment response for the psychiatric disorder itself is also enhanced. That is improving the insomnia improves the symptoms of the psychiatric disorder.

Another prevalent comorbidity is chronic pain. Studies done in chronic pain populations with insomnia have used benzodiazepine receptor agonists (BzRAs), anticonvulsants (i.e., pregabalin, gabapentin), and antidepressants [34,35]. Given the known bi-directional relation of sleep and pain, these medications are used as the primary medication with the expectation that both sleep and pain will be improved. Without an extensive review a few examples can be discussed. Among the BzRAs, triazolam improved sleep and morning stiffness in patients with rheumatoid arthritis [36], eszopiclone improved sleep and joint pain and pain severity in patients with rheumatoid arthritis [37], and zolpidem improved sleep and pain in patients with fibromyalgia [38]. As to the anticonvulsants, pregabalin improved neuropathic pain and reduced pain related sleep disturbance [39] and in patients with fibromyalgia pregabalin improved pain and the extent to which a patient’s sleep was improved related to their pain improvement [40]. Finally, the effects of antidepressants on sleep are varied with some improving sleep while others can be disruptive of sleep. Their effects on pain in chronic pain patients are equivocal.

Another of the prominent exclusion in both studies was a history of or current substance use disorders (SUD). Disturbed sleep and insomnia complaints are very common in SUD and in some cases are predictive of relapse [41]. As yet, very few clinical trials in SUD with insomnia have been conducted. This is primarily because most all the available medications are GABA-acting drugs, which all have their own inherent abuse liability.

As noted earlier, it is clear from the US and NL studies that the typical insomnia trial does not study representative samples from the insomnia population or from the population to be treated with the medication under study. The shift from studying primary insomnia ( DSM-IVR) to insomnia disorder ( DSM-5) does little to alleviate the generalizability problem as the exclusions of concomitant central nervous system CNS medications, and active medical, psychiatric, or substance abuse disorders results in essentially only primary insomnia being studied.

Aside from studying specific comorbidities as discussed above, an “all-comers” clinical trial should be conducted. Such a trial would have minimal exclusion criteria, or only those exclusions that would preclude a clinician from using the medication in clinical practice (e.g., unstable Chronic Pulmonary Disease (COPD), sedative abuser). Such a trial might not produce optimum efficacy data because of potentially high between-subject variability, but would produce ideal safety data. An “all-comers” trial would ideally be conducted at-home (non-PSG) and would utilize a large sample size. The at-home trial would be less of a deterrent for subjects to participate and hence provide a more representative sample. A large sample size would be required, but would allow subgroup analyses to determine safety and efficacy in the various comorbidities. One could envision people with and without comorbid psychiatric disorder, with or with without comorbid medical disorder, subjects who previously used hypnotics versus those that had not. There are certainly limitations to this approach, but nonetheless the data would be informative, especially as more trials with such a methodology appear in the literature.

Another important take away from these data is the need for authors to routinely provide information as to how subjects were recruited, how they were screened, and the frequency and causes for ALL people who expressed interest in the study, but who eventually did not participate. In that same vein the authors need to describe the study population and how their sample differs from insomnia in the population as described in the literature and from hypnotic users as described in the literature.

Data on sleep medications from double blind randomized trials is of critical importance to understanding the safety and efficacy of a given medication. However, to make these data more clinically useful, it is important to understand the population used in collecting the data and how that sample differs from, or is similar to, those patients who will be receiving that medication in general medical practice.

Abbreviations

AHI:

Apnea Hypopnea Index

BzRA:

Benzodiazepine Receptor Agonist

CNS:

Central Nervous System

COPD:

Chronic Obstructive Pulmonary Disease

DSM5:

Diagnostic Statistical Manual of Mental Disorders, Fifth Edition

DSM-IVR:

Diagnostic Statistical Manual of Mental Disorders, Fourth Edition, Revised

FDA:

Food & Drug Administration

ECG:

Electrocardiogram

ICSD2:

International Classification of Sleep Disorders, Second Edition

ICSD3:

International Classification of Sleep Disorders, Third Edition

IRB:

Institutional Review Board

IVRS:

Interactive Telephone Questionnaire System

OCD:

over-the-counter

PLMAI:

Periodic Leg Movement Arousal Index

MSLT:

Multiple Sleep Latency Test

NIDA:

National Institute of Drug Abuse

NPSG:

Nocturnal Polysomnographic

NL:

The Netherlands

SCID:

Structured Clinical Interview for DSM-IVR

SE:

Sleep Efficiency

SUD:

Substance Use Disorder

US:

United States