Reporting results in U.S. clinical trials for obstructive sleep apnea and insomnia: How transparent are they?

Abstract

Background

Clinical trial transparency is important for scientific research and for the good of the general public. Diversity of study samples by race/ethnicity, gender, and age, is important to ensure that results are generalizable. Moreover, reporting results might also be necessary to engage racial/ethnic minorities in clinical research. The primary objective of this study was to describe the results of clinical studies conducted for obstructive sleep apnea (OSA) and insomnia, two of the most prevalent sleep disorders. The secondary objective was to identify which factors were associated with voluntarily reporting the results.

Methods

We reviewed ClinicalTrials.gov, the public database of biomedical and behavioral research operated by the United States (U.S.) National Library of Medicine at the National Institutes of Health (NIH) to ascertain the reports of demographic variables, including race/ethnicity of the studies conducted for OSA and insomnia. Since reporting race/ethnicity was an optional data feature, we searched for publications in PubMed using the unique national clinical trial identification number (NCTID). The NCITD is assigned as soon as the trial is registered. The article extraction was conducted by graduate students and supervised by NJW.

Results

We identified 427 studies on OSA and 404 studies on insomnia. Results were reported for 122 studies. Based on the 122 studies with results that included studies that were terminated (n=16) and/or completed (n=105), and one study was listed as “active” but not recruiting. 46.7% studies involved drugs, 30.3% studied a medical device, and 8.2% investigated behavioral interventions. The age range of subjects was 2–99 years of age and 16.4% included an age range of 35–50 years. Twenty-nine studies (23.8%) reported race/ethnicity in ClinicalTrials.gov. Of these 74% of subjects were white (n= 2,953); 20% Black (n= 822); 1% Asian American (n=40); 2% Hispanic/Latino (n=77); and 3% of study subjects identified race/ethnicity as “other” (n=118). With the PubMed search, we found an additional 24 studies that reported race/ethnicity. There was no difference in reports of race/ethnicity between studies for insomnia and studies for OSA. The intervention type labeled as “behavioral” was a significant predictor (OR 12.49, p-value=< 0.05, CI: 1.002–155.62) for reporting results.

Conclusion

The NIH has mandated federally funded research include women and minorities and that they are representative of the U.S. population. Though gender was reported, few investigators and study sponsors reported the results of race/ethnicity, which begs the question about trial transparency for the future of sleep research. Presumably, the lack of reporting is related to low enrollment of ethnic/minorities included in these studies. Nonetheless, our key finding warrants increased attention to minority participation in sleep clinical studies and trial transparency.

Introduction

Clinical trial transparency has long been debated. This debate has largely focused on two overarching issues. First, the public has a right to know about clinical research. Second, people who want to participate in clinical research should have access to information that can inform their decisions about whether or not to participate.¹ This debate, in part, has led to the creation of Clinicaltrials.gov,² a web-based registry that includes studies conducted in the U.S. and globally. The registry is managed by the National Library of Medicine of the National Institutes of Health (NIH). The registry became publicly available in 2000 and was established to address issues around drug safety.^{3, 4} Since its inception, trial sponsors and investigators of studies (i.e., drug, device) were required to register studies and submit the study results, including study subject characteristics, within one year of all data collection. Age and gender were the only mandatory subject characteristics to be reported.

Underrepresentation of certain population subgroups in studies may lead to higher occurrences of serious side effects in clinical practice and undermine the effectiveness of treatments.⁵ For example, the U.S. Food and Drug Administration (FDA) recommendation of different doses of drug therapy for insomnia for men and women was due in part, to results from studies.⁶ Racial/ethnic minorities may be at greater risk for obstructive sleep apnea as compared to whites ⁷ and insomnia in racial/ethnic minorities may be more severe compared to whites.⁸ Thus, it is important that racial/ethnic minorities are adequately represented in studies for sleep disorders to better understand the disorder and to explore potential treatment outcomes by race/ethnicity. Beyond this, reporting the participation of racial/ethnic subgroups in studies is a matter of social justice and ethics. Study subjects in studies should reflect the demographic make-up for the U.S. population, which is increasingly becoming a ‘majority-minority’ population. By 2025, race/ethnic groups will make-up over 50% of the population ⁹ illustrating the need to address these important issues in clinical research.

Previous studies have reported on ClinicalTrials.gov,^{10, 11} but none have explored sleep studies, and only one explored the baseline demographic characteristics reported in ClinicalTrials.gov (either those that were mandated and/or voluntarily submitted by study sponsors and investigators). The primary purpose of this study was to describe the types of studies for OSA and insomnia, report the demographic diversity of subjects in studies for OSA and insomnia, and to determine what factors were associated with reporting this information on ClinicalTrials.gov.

Materials and Methods

We downloaded the ClinicalTrials.gov publicly available dataset and searched all registered studies conducted in the U.S. between February 2000 to November 28, 2016. Using the key terms function on the website all studies that were listed as “sleep apnea”, “sleep disordered breathing”, and “insomnia” were included. To ascertain demographic (age, gender, and race/ethnicity) variables, we queried the results database. Since reporting race/ethnicity was an optional data feature, we searched for publications in PubMed using the unique national clinical trial identification number (NCTID). Published manuscripts funded by the NIH must include the NCTID,¹² which is assigned as soon as the trial is registered. The article extraction was conducted by graduate students and supervised by NJW.

Available trial information on the registry includes the following: 1) primary purpose; 2) intervention type; 3) study phase; 4) masking; 5) allocation; 6) study sponsor; 7) study phase; 8) trial status; and 9) subject characteristics. If a study sponsor was annotated as “other” we reviewed the record, and if appropriate, we manually recoded those records to accurately reflect the missing data.

Analysis

Frequency and counts of race/ethnicity data are reported. Binary logistic regression analysis was conducted to determine the factors associated with reporting race/ethnicity. Race/ethnic categories were aligned with Office of Management and Budget categories – American Indian/Alaska Native, Asian, African American/Black, Hispanic/Latino, and Native Hawaiian/Pacific Islander, and non-Hispanic Whites/White. We included the following independent variables in the logistic regression model: study phase (phase 1, phase ½, phase 2, phase 2/3, phase 4); masking (double blind, open label, single blind); primary purpose (basic science, diagnostic, health services, prevention, supportive care, treatment); study sponsor (industry, university, U.S. Fed); intervention type (behavioral, device, dietary supplement, drug, procedure, other); and allocation (non-randomized, randomized). We coded missing values as a new category called “N/A”. We created a third category to capture all possible ways that race could have been reported, that is, reported either in ClinicalTrials.gov or in research papers. We examined the correlation matrix of predictor variables and multicollinearity was not found. With regard to the logistic regression, intervention type labeled as “drug” was set as the reference given the FDA requirements on reporting race/ethnicity ². We also hypothesized that study sponsor labeled “industry” would have reduced odds of reporting race/ethnicity, and “university” was selected as the reference group because these studies are likely representative of federally-funded studies (i.e. National Institutes of Health).

Results

Between January 2000 and November 2016, 173,043 studies were registered in ClinicalTrials.gov. Of those studies, we identified 427 that investigated OSA and 404 that investigated insomnia. Five studies included subjects with OSA and insomnia. As described in Table 1, we identified 122 studies with results. Results are reported after the trial has been completed and/or terminated. Of those studies, 46.7% involved drugs, 30.3% studied a medical device, and 8.2% investigated behavioral interventions. The majority of the studies were completed (86.1%), and less than 1% were active, but not yet recruiting or recruiting by invitation only. Over 70% of the studies were treatment related (72.1 %), and approximately 43% of the study sponsors were reported as “university”. Nearly half of the studies (46%) did not report trial phase.

Table 1.

OSA and Insomnia Trial Studies Reporting Results

Characteristics (n=122)	No. (%)	OSA(65, 53.28%)	Insomnia(57, 46.72%)
Allocation
Non-Randomized	9(7.4)	8(12.3)	1(1.8)
Randomized	86(70.5)	38(58.5)	48(84.2)
N/A	27(22.1)	19(29.2)	8(14.0)
Masking
Double Blind	60(49.2)	20(30.8)	40(70.2)
Open Label	39(32.0)	28(43.1)	11(19.3)
Single Blind	13(10.7)	9(13.9)	4(7.0)
N/A	10(8.2)	8(12.3)	2(3.5)
Primary Purpose
Basic Science	4(3.3)	2(3.1)	2(3.5)
Diagnostic	7(5.7)	7(10.8)	0
Health Services Research	2(1.6)	1(1.5)	1(1.8)
Prevention	4(3.3)	4(6.2)	0
Supportive Care	2(1.6)	1(1.5)	1(1.8)
Treatment	88(72.1)	38(58.5)	50(87.7)
N/A	15(12.3)	12(18.5)	3(5.3)
Study Type
Interventional	112(91.8)	57(87.7)	55(96.5)
Observational	10(8.2)	8(12.3)	2(3.5)
Intervention Model
Crossover Assignment	15(12.3)	7(10.8)	8(14.0)
Factorial Assignment	2(1.6)	1(1.5)	1(1.8)
Parallel Assignment	71(58.2)	32(49.2)	39(68.4)
Single Group Assignment	24(19.7)	17(26.2)	7(12.3)
N/A	10(8.2)	8(12.3)	2(3.5)
Agency Type
Industry	36(29.5)	25(38.5)	11(19.3)
U.S. Fed	14(11.5)	4(6.2)	10(17.5)
University	54(44.3)	28(43.1)	26(45.6)
Other	18(14.8)	7(12.3)	10(17.5)
Intervention Type
Behavioral	10(8.2)	2(3.1)	8(14.0)
Device	37(30.3)	33(50.8)	4(7.0)
Dietary Supplement	2(1.6)	0	2(3.5)
Drug	57(46.7)	18(27.7)	39(68.4)
Procedure	4(3.3)	4(6.2)	0
Other	6(4.9)	4(6.2)	2(3.5)
N/A	6(4.9)	4(6.2)	2(3.5)
Study Phase
Phase 1	2(1.6)	1(1.5)	1(1.8)
Phase 1/2	2(1.6)	2(3.1)	0
Phase 2	12(9.8)	5(7.7)	7(12.3)
Phase 2/3	5(4.1)	2(3.1)	3(5.3)
Phase 3	19(15.6)	9(13.9)	10(17.5)
Phase 4	26(21.3)	10(15.4)	16(28.1)
N/A	56(45.9)	36(55.4)	20(35.1)
Trial Status
Active, but not recruiting	1(0.01)	1(1.5)	0
Completed	105(86.1)	56(86.2)	50(87.7)
Terminated	16(13.1)	8(12.3)	7(12.3)

The number of sleep-related studies steadily increased over the past 17 years, as only 19 studies were registered in 2000 (Fig 1).

Figure 1.

OSA and Insomnia Trials Registered in ClinicalTrials.Gov

A total of 18,476 subjects (including both anticipated and actual enrollment) enrolled in the 122 studies. One hundred sixteen studies included both genders. Five studies included women only and one trial included men only. The age range of subjects was 2–99 years of age and 16.4% included an age range of 35–50 years of age. Out of all 122 studies relevant to our study, only 29 studies (26%) reported race/ethnicity in the results. Of the studies with results for race/ethnicity (n=4010), 74% of subjects were white (n= 2,953). African American/Blacks composed about 20% (n= 822) of the study population. Asian Americans and Hispanic/Latinos represented 1% and 2% of the study subjects, respectively (Asian Americans n=40; 1%, Hispanic/Latinos n=77; 2%) and 3% of study subjects identified race/ethnicity as “other” (n=118).

The first report of race/ethnicity was in 2004. There were no reports in 2005 and for each year from 2006–2014 we found at least 1 trial with the results for race/ethnicity. Additionally, the data revealed that in 2012, only one study reported results and 100% of the subjects were white. In 2007 44% of trial subjects were Black (Fig 2). With respect to the PubMed data, linking the NCTID number to the PubMed database resulted in 24 additional studies, which reported information on race/ethnicity in their respective publications. Of these studies, five only included the results for the white study sample. We did not include additional information beyond race/ethnicity, as this is the main aspect of the study.

Fig 2.

(OSA and insomnia trial results reports of race/ethnicity participation by year)

As shown in Table 2, investigators for behavioral studies were more likely to report race/ethnicity than device studies (OR=12.49, p-value=< 0.05, CI: 1.002–155.62) on either ClinicalTrials.gov or academic papers. There were no differences by study sponsor (See Table 2) in reporting results.

Table 2.

Logistic regression statistical summary

Total sample size n=122	No. (%)	Odds Ratio and 95% Confidence Interval
		Report race on ClinicalTrials.gov		Report race in academic papers		Report race in either way
Study Phase
Phase 1	2(1.6)	>999.999	-	<0.001	-	0.41	0.01–36.95
Phase 1/2	2(1.6)	<0.001	-	2.94	0.09–98.62	1.25	0.05–33.78
Phase 2	12(9.8)	0.79	0.14–4.39	0.40	0.03–5.78	0.59	0.12–2.95
Phase 2/3	5(4.1)	<0.001	-	<0.001	-	<0.001	-
Phase 3	19(15.6)	0.32	0.05–1.89	2.36	0.43–12.89	0.58	0.14–2.46
Phase 4 (reference)	26(21.3)	1	-	1	-	1
N/A	56(45.9)	0.56	0.09–3.57	0.35	0.04–3.05	0.26	0.05–1.34
Study Sponsor
Industry	36(29.5)	1.90	0.47–7.65	1.37	0.28–6.67	1.41	0.42–4.70
U.S. Fed	14(11.5)	1.56	0.22–10.93	1.52	0.17–13.66	1.34	0.22–8.27
University (reference)	54(44.3)	1	-	1	-	1
Other	18(14.8)	1.74	0.35–8.53	1.08	0.15–7.88	1.13	0.28–4.63
Allocation
Non-Randomized	9(7.4)	<0.001	-	<0.001	-	<0.001	-
Randomized (reference)	86(70.5)	1	-	1	-	1	-
N/A	27(22.1)	0.90	0.11–7.49	0.31	0.02–5.50	0.66	0.10–4.29
Masking
Double Blind (reference)	60(49.2)	1	-	1	-	1	-
Open Label	39(32.0)	0.82	0.13–5.14	0.86	0.11–7.06	0.89	0.18–4.51
Single Blind	13(10.7)	0.75	0.05–10.67	<0.001	-	0.52	0.04–6.55
N/A	10(8.2)	<0.001	-	<0.001	-	<0.001	-
Intervention Type
Behavioral	10(8.2)	2.95	0.21–42.13	7.82	0.32–191.08	12.49 *	1.002–155.62
Device	37(30.3)	1.04	0.24–4.52	4.65	0.77–28.02	2.36	0.64–8.69
Dietary Supplement	2(1.6)	<0.001	-	<0.001	-	<0.001	-
Drug (reference)	57(46.7)	1	-	1	-	1	-
Procedure	4(3.3)	>999.999	-	>999.999	-	>999.999	-
Other	6(4.9)	>999.999	-	<0.001	-	>999.999	-
N/A	6(4.9)	>999.999	-	>999.999	-	>999.999	-
Primary Purpose
Basic Science	4(3.3)	<0.001	-	3.58	0.20–65.04	0.70	0.05–9.22
Diagnostic	7(5.7)	1.24	0.08–18.98	<0.001	-	0.80	0.06–11.49
Health Services Research	2(1.6)	<0.001	-	<0.001	-	<0.001	-
Prevention	4(3.3)	<0.001	-	<0.001	-	<0.001	-
Supportive Care	2(1.6)	2.87	0.14–59.13	<0.001	-	1.04	0.05–23.63
Treatment (reference)	88(72.1)	1	-	1	-	1	-
N/A	15(12.3)	<0.001	-	3.44	0.09–137.32	0.50	0.03–9.66

^*means p-value < 0.05

Note: Quasi-complete separation of data detected. This may be because of small sample size for certain categories coded in variables, and it does not provide any evidence for or against a statistical hypothesis. SAS 9.4 was used for analysis.

Discussion

Our findings revealed less than 1% of studies registered in ClinicalTrials.gov are related to OSA or insomnia. Studies covered a full range of intervention types, including drugs, behavioral interventions, and devices. Studies were diverse with respect to age and gender. Most studies were sponsored by universities followed by industry and were located in the northeast region. Study sponsors and investigators of behavioral studies were more likely than drug studies to report race/ethnicity. The factors that contribute to reporting information about race/ethnicity are unknown. One possibility is that study sponsors and investigators of behavioral studies are more likely to interested in examining determinants of disease such as race/ethnicity than investigators of non-behavioral studies. Another possibility is that the increased interest in disparities in sleep¹³ and behavioral sleep medicine, may explain reporting by some investigators. Given this, we think the reports of race/ethnicity of the samples in clinical studies will increase over time.

NIH mandates federally funded clinical research include women and minorities and the number of minorities included in such studies should be representative of the U.S. population.¹⁴ Compared to other racial/ethnic groups enrollment for African American/Blacks seems to be representative of the U.S. population. However, overall reporting of race/ethnicity was incomplete. Therefore, these results should be interpreted with caution. Reporting race/ethnicity across all studies remains important for at least two reasons. First, reporting race/ethnicity is in accordance with the need to enroll minorities in studies, and several studies have documented low minority participation in studies.¹⁵ Second, treatment outcomes may vary by race/ethnicity, as has been reported in previous studies.¹⁶ But, subgroup analyses cannot be performed appropriately if not adequately powered by race/ethnicity. Indeed, some of the studies included small sample sizes, which limits the conduct of detailed analyses by race/ethnicity. Similarly, there was significant missing data in the database, which is another limitation in reporting race/ethnicity. It is also plausible that studies that did not report race/ethnicity data have less diverse samples and are reluctant to report this information. Nonetheless, the results reported here raise important questions about diversity and recruitment and retention of racial/ethnic minorities in sleep studies.

Overall, trial reporting is fairly low across studies. Gopal et al. ¹¹ reported a lack of reporting with results compliance using ClinicalTrials.gov. In that study, 14% of studies reported results, but the authors did not explore other factors related to trial reporting. We found only one other investigation that examined reporting of race/ethnicity using the ClinicalTrials.gov registry. In that study, Charrow ¹⁷ and colleagues reviewed dermatological studies and found that 58 out of 97 studies conducted in the U.S. reported race/ethnicity. Although the findings indicated low reports of race/ethnicity, the number of African American/Blacks enrolled in dermatological studies was comparable to the U.S. African American/Black population. Charrow et al. results are in contrast with sleep studies, suggesting that the field of sleep lags behind compared to other conditions in reporting race/ethnicity.

Recently, the Department of Health and Human Services expanded its policy on reporting subject information, which now includes mandatory reporting of race/ethnicity of study results.¹⁸ Though it is unclear whether or not there is any accountability for compliance with this mandate, it’s unlikely that this mandate can have any impact on reporting race/ethnicity. For example, Corbie-Smith reported that among high impact medical journals, 46% of trials that reported the race/ethnicity of the sample reported only one or two racial/ethnic categories¹⁹. Nonetheless, since we queried the database prior to the new policy, it would be interesting to explore whether or not investigators comply with the new mandate.

The field of sleep medicine has grown significantly over the last decade, and yet, sleep disorders are not studied to the same extent as other health conditions. For example, in an analysis of ClinicalTrials.gov, Califf et al. found from 2007 to 2010 the majority of studies were related to oncology (n=8,992), cardiovascular disease (n=3,437), and mental health (n=3,695).¹ Further, our analysis also revealed that sleep studies were more likely to be funded by universities. This could be a concern to the investigators, in particular, and the field overall, because federal budgets for research and development has decreased over the years and data suggest that this trend will continue in the near future. ²⁰

Limitations

These findings must be discussed in the context of its limitations. First, the Clinicaltrials.gov database may not include the full scope of results because the results feature was launched in 2008 and was optional. Second, some published studies did not include an NCTID, which limited our ability to link the trial with resulting publications. Third, all data is manually entered by the study sponsor and/or investigators, which is likely prone to error. Finally, the odds ratios were very large, which could be related to the small sample size. Notwithstanding, the use of ClinicalTrials.gov is a unique and important registry. The registry provides information, to a wide array of audiences, including current and prospective study subjects, research scientists, clinicians, health policymakers, and other key stakeholders. Given the historical issues around trust in research, especially among minorities, lack of transparency in reports of studies could be a barrier to reach minority populations.²¹ This finding also highlights the need for study sponsors and investigators to forge new partnerships with academic and community institutions who have experience working with minority populations, to address some of these barriers.

Conclusions

The number of OSA and insomnia studies has increased over the past 17 years, but race/ethnicity is rarely reported. To our knowledge, this is the only study that has examined the participation of racial/ethnic minorities enrolled in clinical studies on sleep research. Given the significant and growing empirical evidence of racial/ethnic disparities in sleep health, underreporting of results by race/ethnicity limit our understanding of any potential relationship between racial/ethnic differences in treatment outcomes for these sleep studies. The major sleep journals should consider implementing formal guidelines to influence how race/ethnicity is reported.