Evaluation of Journal Registration Policies and Prospective Registration of Randomized Clinical Trials of Nonregulated Health Care Interventions

Key Points

Question

Have specialty journals, such as in the fields of nursing, rehabilitation, or surgery, that publish randomized clinical trials of health care interventions not subject to regulation by the US Food and Drug Administration implemented and enforced prospective trial registration policies?

Findings

In this study of 953 nonregulated intervention trials published in 254 journals included in the study, only 11% of journals required prospective registration. Only 34% of trials published in journals with a policy were prospectively registered compared with 18% of trials in journals without a policy.

Meaning

Nonregulated health care intervention trials published in specialty journals are often not prospectively registered, and existing registration policies are rarely enforced;

Abstract

Importance

Many interventions that are important to the health care of patients are not subject to regulation by the US Food and Drug Administration (FDA) or comparable regulatory bodies in other nations.

Objective

To determine whether specialty journals that publish trials of primarily nonregulated health care interventions require prospective registration and whether the prospective registration policies are associated with the publication of prospectively registered trials, trials with adequately registered outcomes, and trials with primary outcomes consistent with the registered primary outcomes.

Design and Methods

PubMed was searched daily, from March 18, 2016, to September 17, 2016, for nonregulated intervention randomized clinical trials. The search included all journals in the Clarivate Analytics Science Citation Index Expanded categories of behavioral sciences, nursing, nutrition and dietetics, psychology, rehabilitation, and surgery. Trials of interventions not subject to FDA regulation were included. One investigator extracted journal registration policy and trial registration status. Two investigators independently extracted trial registration and publication characteristics.

Main Outcomes and Measures

For journals, the main outcome was the trial registration policy. For trials, the main outcomes were prospective registration, adequacy of outcome registration, and concordance of registered with published primary outcomes.

Results

In total, 953 nonregulated intervention trials published in 254 journals were identified. Prospective registration was required for publication by 29 (11.4%) of 254 journals, and an additional 12 journals (4.7%) had conditional date-based requirements. Only 189 (19.8%) of the 953 trials were registered prospectively, including 33 of 98 published in journals with prospective registration policies as compared with 156 of 855 in journals without policies (33.7% vs 18.2%; P = .004). Among the 17 journals that required prospective registration and had at least 2 included trials, none had a prospective registration of more than 50%. In journals with policies, only 3 of 98 trials included primary outcomes consistent with prospectively, adequately registered outcomes, as compared with 34 of 852 trials in journals without policies (3.1% vs 4.0%; P = .62).

Conclusions and Relevance

Few journals in behavioral sciences or psychology, nursing, nutrition and dietetics, rehabilitation, and surgery require prospective trial registration, and those with existing registration policies rarely enforce them; this finding suggests that strategies for encouraging prospective registration of clinical trials not subject to FDA regulation should be developed and tested.

This study analyzes the prospective registration policies of specialty journals that publish randomized clinical trials of drugs, biologics, and medical devices not regulated by the US Food and Drug Administration.

Introduction

In the United States, Food and Drug Administration regulations require prospective registration of most phase 2, 3, and 4 trials of drugs, biologics, and medical devices.^{1,2,3,4,5,6,7} The European Union requires prospective registration of trials of drugs and biologics.^8,9 The term nonregulated interventions has been used to refer to health care interventions that are not subject to regulatory oversight and thus not subject to these registration requirements.^10,11,12 Examples of nonregulated health care interventions include surgical procedures, nursing interventions, diets, psychological treatments, and rehabilitation procedures.^10,11,12 Although trials of these interventions may be subject to less scrutiny than trials of drugs and medical devices, some of these interventions pose substantial risks to patients, and some may have important implications for the health care system.

Since 2005, International Committee of Medical Journal Editors (ICMJE) journals have required that all clinical trials of health care interventions considered for publication be registered prior to participant enrollment.^13,14,15 Many non-ICMJE journals have adopted this policy.¹⁶ Prospective registration policies are intended to reduce the risk of bias from nonreporting of trial results or the selective reporting of analyses and outcomes.^{17,18,19,20,21,22,23}

Many nonregulated health care intervention trials are not registered prospectively, even when journal registration requirements are in place.^{10,24,25,26,27,28,29,30,31,32,33} Previous studies on registration policies and practices, however, have focused on retrospective analyses of small groups of journals. One reason is that it is often not possible to determine retrospectively when journal registration policies, if any, were initiated and, thus, to evaluate compliance.

We conducted a daily search for all publications of randomized clinical trials (RCTs) of nonregulated interventions in journals in the categories of behavioral sciences or psychology, nursing, nutrition and dietetics, rehabilitation, and surgery that were initially listed in PubMed during a 6-month period in 2016. By searching daily, we could examine journal registration policies as close as possible to when the RCTs were submitted. We studied journal registration policies, the associations between these policies and the prospective registration of trials, the registration of primary outcomes and whether this registration was adequate, and the consistency of registered and primary outcomes.

Methods

Journals

Ethics approval was not required for this study. We identified all journals listed in the following categories of the 2014 Clarivate Analytics Science Citation Index Expanded: (1) behavioral sciences, (2) nursing, (3) nutrition and dietetics, (4) psychology, (5) rehabilitation, and (6) surgery, that published at least 1 article listed in PubMed in 2014 or 2015. We compared journal names listed in the Science Citation Index Expanded against those in the National Library of Medicine Catalog and used National Library of Medicine names for this search. The search strategy included 50 behavioral sciences, 107 nursing, 68 nutrition and dietetics, 66 psychology, 57 rehabilitation, and 187 surgery journals, totaling 524 journals given that 11 were in more than 1 category. These 524 journals are listed in eTable 1 in the Supplement.

Identification of Nonregulated Health Care Intervention RCTs

We searched PubMed daily, from March 18, 2016, through September 17, 2016, using the names of included journals with the following terms: (random*[Title/Abstract] OR “RCT”[Title/Abstract] NOT (“Drug Therapy”[Mesh] OR “drug therapy”[Subheading] OR “Pharmaceutical Preparations”[Mesh] OR Review[ptyp] OR “meta-analysis”[Title] OR “systematic review”[Title]). More information is provided in eTable 2 in the Supplement.

Eligible RCTs were trials that evaluated the effects of nonregulated health care interventions on health outcomes.^13,14,15 We excluded trials with a comparison to a drug, biologic, or medical device in any arm as well as publications that stated they reported only secondary outcomes. If results from more than 1 RCT were reported in a single publication, we included only the first RCT in the article.

We uploaded search results into DistillerSR (Evidence Partners), which was used for all coding procedures. Given the available resources, one investigator assessed titles and abstracts and full-text articles for eligibility.

Data Extraction

For each eligible RCT, one investigator extracted journal characteristics, including registration policy, and trial characteristics, including registration status. Two investigators independently extracted trial registration information and published outcomes, and discrepancies were resolved through consensus and consultation with a third investigator, as necessary. All extracted data are available on the Open Science Framework at https://osf.io/jk2sq/.

Journal Registration Policy

For each RCT in the study, we reviewed the journal website, including author instructions, within 3 weeks of the daily PubMed search that identified the RCT. Journals were classified as (1) requiring prospective registration; (2) requiring registration with no timing requirement; (3) not requiring registration; or (4) having conditional requirements, typically based on date of trial initiation or journal submission. For RCTs in journals with conditional requirements, we determined whether the requirements applied to the RCT on the basis of dates extracted from the trial publication or, if not specified there, the trial registration.

To evaluate whether journal policies might have changed between the submission date of a trial and its PubMed listing, we used the website Wayback Machine, which stores archived website snapshots.^34,35 For trials with published journal submission dates, if a snapshot was available within 3 months prior to the submission date, we compared the registration policy from the most recent snapshot prior to submission with the policy on the journal website at the time of our PubMed search.

Prospective Registration

To determine trial registration status, we followed a procedure used in previous studies.^29,30,31 We first attempted to retrieve registration data from the publication. If the registration data were not reported, we searched ClinicalTrials.gov; the International Standard Randomized Controlled Trials Number registry (http://www.isrctn.com); and registries, if available, from the first and corresponding authors’ countries. For this search, we used corresponding, first, and last author names and key terms. We attempted to match the principal investigator, funding source, intervention, control group, and trial design of the RCT with registration records. If we did not find a registry entry or if we found ambiguity about the match, we contacted the corresponding author once by email.

To determine if trials were registered prospectively, we compared registration and enrollment dates. We prioritized enrollment dates from publications, but if they were not available, we used dates from trial registrations. We classified trials registered prior to or in the same month enrollment began as registered prospectively.

Adequate Primary Outcome Registration

Among RCTs registered prospectively, we recorded whether registrations included a specific measurement of the primary outcome or outcomes (eg, Beck Depression Inventory score, not simply depression), a specific assessment time point (eg, 12-month postrandomization), a specific metric (eg, change from baseline, final value), and a specific method of aggregation (eg, mean, proportion).^2,36 We classified registrations as adequate if the registration included a single primary outcome or multiple primary outcomes with a specific measurement and time point of assessment.^29,30,31 We did not require specification of analysis metric or aggregation method for classification of adequacy.

Trials were classified as adequately registered—single primary outcome if they adequately registered a single primary outcome variable and assessment time point, or multiple primary outcomes or time points, and there was a plan for statistical adjustment for multiple comparisons included in the registration as well as a subsequent publication that included the same primary outcomes or time points with appropriate statistical adjustment. If registrations listed multiple adequately registered primary outcomes, but there was no statistical adjustment for multiple outcomes, the trial was classified as adequately registered—multiple primary outcomes. If there were changes in trial registration records over time, we used the last update prior to participant enrollment.

Consistency of Registered and Published Outcomes

For each adequately registered trial, we compared the registered primary outcome with published primary outcome. Trials were classified as consistent if registered and published primary outcome analyses were the same and discrepant if they differed.

Statistical Analysis

We present frequencies and percentages of journals with prospective registration requirements; journals with registration requirements with no timing requirement; and journals without a policy, both overall and by journal categories, impact factor, and publishers. For journal policies and prospective registration, we present frequencies and percentages of trials registered prospectively, registered during the trial, registered after the trial, and not registered, both overall and by journal characteristics (registration policy, impact factor, registration policy stratified by impact factor, and publisher) and trial characteristics (category of intervention, country, funding, and sample size). We also present frequencies and percentages of trials with prospectively and adequately registered primary outcomes and with published outcomes consistent with those registered.

We conducted statistical tests to evaluate whether the presence of a journal prospective registration policy was associated with the prospective registration of trials published in the journal, both overall and stratified by impact factor, prospective and adequate registration of primary outcomes in trials published in the journal, and publication of primary outcomes consistent with prospective and adequately registered primary outcomes. In addition, we tested the association of trial registration with journal impact factor. For all of these analyses, we used generalized estimating equations with a binary outcome and an identity link function. We used an exchangeable correlation structure with robust SEs to account for the correlation between trials within the same journal. Analyses were run using the geepack package in R, version 3.2.3 (R Foundation for Statistical Computing). We used α <.05 to test for statistical significance.

Results

Article Selection

The search retrieved 4930 citations, of which 3332 (67.6%) were excluded after title and abstract review and 645 (13.1%) after full-text review. The remaining 953 RCTs (19.3%) were included (Figure).

Figure. Flowchart of Included Articles.

Journal Registration Policy

The 953 RCTs were published in 254 journals, of which 100 (39.4%) published 1 of these trials in the 6-month study period, 58 (22.8%) published 2, 42 (16.5%) published 3 to 4, 34 (13.4%) published 5 to 10, and 20 (7.9%) published 11 to 44. Of the 254 journals, 29 (11.4%) required prospective trial registration and 12 (4.7%) had conditional policies based on the date of trial initiation or journal submission, as shown in Table 1. Table 2 shows the frequencies and percentages of journal trial registration policies by journal category, impact factor, and publisher.

Table 1. Journal Registration Policies of 254 Journals That Published Trials .

Registration Policy	Journals, No. (%)
Prospective registration required	29 (11.4)
Prospective registration required (conditionally)a	12 (4.7)
Registration without timing requirement	31 (12.2)
Registration without timing requirement (conditionally)a	8 (3.1)
No registration required	174 (68.5)

^aRequired registration for a subset of trials based on date of start of participant enrollment or submission to journal.

Table 2. Association Between Journal Registration Policies and Journal Characteristics.

Journal Characteristic	Journal Registration Policy, No. (%)			Total, No.
	Prospective Registration Requirement (n = 41)a	Registration Without Timing Requirement (n = 39)a	No Registration Requirement (n = 174)b
Category
Behavioral sciences or psychology	5 (16.1)	2 (6.5)	24 (77.4)	31
Nursing	3 (5.5)	4 (7.3)	48 (87.3)	55
Nutrition and dietetics	4 (8.7)	15 (32.6)	27 (58.7)	46
Rehabilitation	13 (29.5)	5 (11.4)	26 (59.1)	44
Surgery	16 (19.5)	13 (15.9)	53 (64.6)	82
Impact factor
0.0-1.9	19 (13.9)	13 (9.5)	105 (76.6)	137
2.0-3.9	17 (17.5)	20 (20.6)	60 (61.9)	97
≥4.0	5 (25.0)	6 (30.0)	9 (45.0)	20
Publisher
Elsevier	18 (27.3)	9 (13.6)	39 (59.1)	66
Sage	3 (16.7)	1 (5.6)	14 (77.8)	18
Springer	0	1 (5.3)	18 (94.7)	19
Taylor & Francis	5 (25.0)	0	15 (75.0)	20
Wiley	1 (2.8)	10 (27.8)	25 (69.4)	36
Wolters Kluwer	5 (17.9)	2 (7.1)	21 (75.0)	28
Otherc	9 (13.4)	16 (23.9)	42 (62.7)	67

^aJournals with a conditional policy were classified as requiring registration for both prospective registration and registration without a timing requirement.

^bIncluded 4 journals, all categorized as not requiring registration, that were classified by Clarivate Analytics as belonging to more than 1 category of nonregulated intervention: Appetite (nutrition and dietetics; behavioral sciences or psychology), International Journal of Eating Disorders (nutrition and dietetics; behavioral sciences or psychology), European Journal of Cancer Care (nursing; rehabilitation); and Rehabilitation Nursing (nursing; rehabilitation).

^cIncluded publishers with <10 included journals.

Changes in Journal Registration Requirements

Of the articles and journals, 650 articles (68.2%) from 165 journals (65.0%) included a published submission date. Of these, there was a Wayback Machine archive of author instructions in the 3 months prior to the submission date for 126 articles (13.2%) from 46 journals (18.1%). Author instructions changed between the Wayback Machine archive and our PubMed search date for only 1 article. For that article, the journal Nursing and Health Sciences added a statement encouraging, but not requiring, trial registration. This archived information did not affect the journal’s classification.

For 3 journals (Disability and Rehabilitation, Journal of Head Trauma Rehabilitation, and Obesity), author instructions changed during the 6-month daily search period. Articles published in Disability and Rehabilitation and Obesity included a submission date, and all included RCTs were submitted prior to the change from not requiring to requiring prospective trial registration; we classified both journals as not requiring registration for all included trials. Articles published in the Journal of Head Trauma Rehabilitation did not provide submission dates, but we confirmed through PubMed listing dates and consultation with the journal editor that all 6 included trials were submitted prior to the change in journal policy. We also classified this journal as not requiring prospective trial registration.

Prospective Registration

Of the 953 trials, 189 (19.8%) were registered prospectively, 141 (14.8%) after initiation of participant enrollment and prior to collection of trial outcomes, and 120 (12.6%) after completion of the trial. More than half (496 [52.0%]) were not registered (Table 3; eTable 3 in the Supplement). Of the 141 trials registered after participants were enrolled, 3 (2.1%) were registered within a month and 61 (43.3%) within 6 months.

Table 3. Registration Status of the 953 Trials .

Registration Statusa	Trials, No. (%)
Registered prospectively prior to participant enrollment	189 (19.8)
Registered during trial	141 (14.8)
Registered after trial	120 (12.6)
Registered late (unclear whether during or after trial)b	4 (0.4)
Not registered	496 (52.0)
Trial initiated prior to ICMJE registration policyc	3 (0.3)

Abbreviation: ICMJE, International Committee of Medical Journal Editors.

^aFor ClinicalTrials.gov, the first received date was used as the registration date. For the International Standard Randomised Controlled Trials Number registry, the date applied was used. Trials were classified as registered prior to enrollment if they began participant enrollment in the same month as the initial registration date.

^bTrials were registered after the start of participant enrollment, but it could not be determined if registered prior to collection of trial outcomes or after trial.

^cICMJE policy required prospective registration beginning July 1, 2005.¹⁸

As shown in Table 4, 98 (10.3%) of the 953 trials were published in a journal that required prospective registration. Of these RCTs, 33 were registered prospectively, as compared with 156 of 855 trials published in journals without such a requirement (33.7% vs 18.2%; P = .004). Among journals that required prospective registration unconditionally, 17 journals had at least 2 included trials (range of 2 to 14 trials). The percentage of prospectively registered trials was not greater than 50% in any journals. Table 4 also shows prospective registration by impact factor, journal policy stratified by impact factor, publisher, and trial characteristics.

Table 4. Association Between Trial Registration Status and Journal and Trial Characteristics for the 953 Trials.

Variable	Trial Registration Status, No. (%)						Total, No. (N = 953)
	Prospective Registration (N = 189)	Registration During Trial (N = 141)	Registration After Trial (N = 120)	Late Registration [timing unclear] (N = 4)a	No Registration (N = 496)	Pre-ICMJE Policy (N = 3)
Journal Characteristics
Journal registration policy that applied to the trial
Prospective registration required	33 (33.7)	19 (19.4)	15 (15.3)	0	31 (31.6)	0	98
Registration required without timing	40 (24.5)	30 (18.4)	44 (27.0)	0	48 (29.4)	1 (0.6)	163
No registration required	116 (16.9)	92 (13.4)	61 (8.9)	4 (0.6)	410 (59.9)	2 (0.3)	685
Undeterminedb	0	0	0	0	7 (100)	0	7
Impact factor and registration policy by impact factorc
0.0-1.9	35 (9.8)	38 (10.6)	41 (11.5)	1 (0.3)	243 (67.9)	0	358
Prospective registration required	8 (17.8)	7 (15.6)	9 (20.0)	0	21 (46.7)	0	45
Prospective registration not requiredd	27 (8.7)	31 (9.9)	32 (10.3)	1 (0.3)	221 (70.8)	0	312
Undeterminedb	0	0	0	0	1 (100)	0	1
2.0-3.9	95 (20.4)	72 (15.5)	60 (12.9)	2 (0.4)	235 (50.4)	2 (0.4)	466
Prospective registration required	11 (42.3)	6 (23.1)	3 (11.5)	0	6 (23.1)	0	26
Prospective registration not requiredd	84 (19.4)	66 (15.2)	57 (13.1)	2 (0.5)	223 (51.4)	2 (0.5)	434
Undeterminedb	0	0	0	0	6 (100)	0	6
≥4.0	59 (45.7)	31 (24.0)	19 (14.7)	1 (0.8)	18 (14.0)	1 (0.8)	129
Prospective registration required	14 (51.9)	6 (22.2)	3 (11.1)	0	4 (14.8)	0	27
Prospective registration not requiredd	45 (44.1)	25 (24.5)	16 (15.7)	1 (1)	14 (13.7)	1 (1)	102
Undeterminedb	0	0	0	0	0	0	0
Publisher
Elsevier	39 (21.2)	26 (14.1)	25 (13.6)	0	94 (51.1)	0	184
Wiley	31 (25.0)	20 (16.1)	13 (10.5)	2 (1.6)	58 (46.8)	0	124
Wolters Kluwer	19 (20.2)	14 (14.9)	9 (9.6)	0	50 (53.2)	2 (2.1)	94
Springer	15 (19.7)	12 (15.8)	6 (7.9)	0	43 (56.6)	0	76
Sage	18 (25.7)	12 (17.1)	10 (14.3)	0	29 (41.4)	1 (1.4)	70
Taylor & Francis	10 (19.2)	8 (15.4)	7 (13.5)	0	27 (51.9)	0	52
IPEC Press	0	0	0	0	44 (100)	0	44
Biomed Central	8 (23.5)	4 (11.8)	15 (44.1)	0	7 (20.6)	0	34
Cambridge University Press	7 (21.2)	6 (18.2)	2 (6.1)	1 (3.0)	17 (51.5)	0	33
NRC Research Press	3 (10.0)	2 (6.7)	3 (10.0)	0	22 (73.3)	0	30
American Society for Nutrition	12 (42.9)	8 (28.6)	7 (25.0)	0	1 (3.6)	0	28
MDPI open access	4 (17.4)	4 (17.4)	6 (26.1)	0	9 (39.1)	0	23
Human Kinetics	0	2 (10.5)	1 (5.3)	0	16 (84.2)	0	19
American Psychological Association	1 (6.2)	2 (12.5)	0	0	13 (81.2)	0	16
IOS Press	0	1 (6.2)	1 (6.3)	0	14 (87.5)	0	16
Karger	5 (45.5)	2 (18.2)	3 (27.3)	0	1 (9.1)	0	11
Other	17 (17.2)	18 (18.2)	12 (12.1)	1 (1.0)	51 (51.5)	0	99
Trial Characteristics
Category of nonregulated intervention based on journal where publishede
Behavioral sciences or psychology	35 (27.3)	16 (12.5)	15 (11.7)	1 (0.8)	61 (47.7)	0	128
Nursing	7 (5.7)	17 (13.9)	12 (9.8)	0	86 (70.5)	0	122
Nutrition and dietetics	62 (23.5)	36 (13.6)	49 (18.6)	2 (0.8)	114 (43.2)	1 (0.4)	264
Rehabilitation	49 (17.8)	42 (15.3)	30 (10.9)	1 (0.4)	152 (55.3)	1 (0.4)	275
Surgery	39 (21.4)	30 (16.5)	16 (8.8)	0	96 (52.7)	1 (0.5)	182
Corresponding author country
Australia	22 (44.0)	9 (18.0)	4 (8.0)	1 (2.0)	14 (28.0)	0	50
Brazil	11 (24.4)	8 (17.8)	10 (22.2)	0	16 (35.6)	0	45
Canada	9 (21.4)	3 (7.1)	7 (16.7)	0	23 (54.8)	0	42
China	5 (10.4)	4 (8.3)	5 (10.4)	0	34 (70.8)	0	48
Denmark	7 (46.7)	6 (40.0)	1 (6.7)	0	1 (6.7)	0	15
Egypt	0	0	3 (23.1)	0	10 (76.9)	0	13
Germany	12 (35.3)	1 (2.9)	6 (17.6)	1 (2.9)	14 (41.2)	0	34
Greece	1 (10.0)	1 (10.0)	2 (20.0)	0	6 (60.0)	0	10
Iran	2 (7.7)	6 (23.1)	10 (38.5)	0	8 (30.8)	0	26
Italy	2 (8.0)	2 (8.0)	6 (24.0)	0	15 (60.0)	0	25
Japan	8 (30.8)	5 (19.2)	3 (11.5)	0	9 (34.6)	1 (3.8)	26
Netherlands	12 (42.9)	9 (32.1)	3 (10.7)	0	4 (14.3)	0	28
New Zealand	5 (41.7)	2 (16.7)	0	0	5 (41.7)	0	12
Norway	1 (10.0)	5 (50.0)	3 (30.0)	0	1 (10.0)	0	10
South Korea	6 (8.7)	0	2 (2.9)	0	61 (88.4)	0	69
Spain	4 (11.8)	6 (17.6)	10 (29.4)	0	14 (41.2)	0	34
Sweden	0	8 (50.0)	3 (18.8)	0	5 (31.3)	0	16
Taiwan	1 (5.6)	2 (11.1)	0	0	15 (83.3)	0	18
Turkey	1 (2.3)	2 (4.7)	3 (7.0)	0	37 (86.0)	0	43
United Kingdom	16 (23.5)	16 (23.9)	8 (11.8)	2 (3.0)	26 (38.2)	0	68
United States	48 (21.3)	36 (16.0)	15 (6.7)	0	125 (55.6)	1 (0.4)	225
Other	16 (16.7)	10 (10.4)	16 (16.6)	0	53 (55.2)	1 (1.0)	96
Funding
National/state/provincial	88 (30.3)	58 (20.0)	32 (11.0)	3 (1.0)	107 (36.9)	2 (0.7)	290
Other not-for-profit	34 (24.2)	35 (24.8)	13 (9.2)	0	59 (41.8)	0	141
Industry	36 (19.0)	38 (20.1)	24 (12.7)	2 (1.1)	89 (47.1)	0	189
Internal university or hospital	21 (20.4)	21 (20.4)	21 (20.4)	1 (1.0)	38 (36.9)	1 (1.0)	103
No funding	20 (17.5)	7 (6.1)	24 (21.1)	1 (0.9)	62 (54.4)	0	114
Not reported	27 (10.0)	23 (8.6)	25 (9.3)	0	194 (72.1)	0	269
Sample size
0-49	61 (14.6)	51 (12.2)	49 (11.8)	0	256 (61.4)	0	417
50-99	55 (22.0)	41 (16.4)	31 (12.4)	2 (0.8)	121 (48.4)	0	250
≥100	73 (25.6)	49 (17.2)	40 (14.0)	2 (0.7)	119 (41.6)	3 (1.1)	286

^aTrials were registered after start of participant enrollment, but it could not be determined if registered prior to collection of trial outcomes or after trial.

^bUndetermined applied when the journal had a conditional registration policy, but dates were not provided to determine the policy that applied to the trial. For analysis of registration policy stratified by impact factor, the 7 undetermined trials were classified as not requiring registration.

^cThe percentage of prospectively registered trials was approximately twice as high for journals with a prospective registration policy, compared with journals without a policy (impact factor, <2.0 [P = .10] vs 2.0-3.9 [P < .01]), but was similar across policy status for journals with an impact factor of ≥4 (P = .48). Journal impact factor was correlated with the number of trials published in the 6-mo study period (r = 0.26).

^dThis category included journals with a required registration policy that did not include a prospective registration policy and journals with no registration policy.

^eEighteen trials were published in 4 journals that were listed in more than 1 category of nonregulated interventions.

Adequate Registration of Primary Outcomes

Of the 953 included trials, 3 (0.3%) were completed or still active in July 2005 and met ICMJE requirements to register by September 2005. These RCTs were not required to register before patients were enrolled. Of the remaining 950 trials, based on the criterion of having a specific measurement and time point, only 85 trials (8.9%) prospectively and adequately registered a single primary outcome (60 [6.3%]) or multiple primary outcomes (25 [2.6%]). This group included 15 of 98 trials that were subjected to prospective registration requirements, as compared with 70 of 852 trials in journals without policies (15.3% vs 8.2%; P = .04).

Consistency of Registered and Published Outcomes and Journal Policy

Among the 950 trials that first enrolled participants after July 2005, 37 (3.9%) adequately registered a single primary outcome or multiple primary outcomes and published a consistent outcome or outcomes. This included 35 (58.3%) of 60 trials that registered a single primary outcome and 2 (8.0%) of 25 trials that registered multiple primary outcomes. Among 98 trials subject to a journal policy that required prospective registration, only 3 had adequate registration and a consistent published primary outcome or outcomes, as compared with 34 of 852 not subject to such a policy (3.1% vs 4.0%; P = .62). Of the 48 trials that published primary outcomes discrepant to those registered, 14 (29.2%) registered a single primary outcome but published multiple primary outcomes, 7 (14.6%) registered multiple primary outcomes but published a single primary outcome, 15 (31.3%) registered and published different primary outcomes, and 12 (25.0%) did not identify which outcome or outcomes were primary in the published report.

Discussion

We reviewed 953 RCTs of nonregulated health care interventions with initial PubMed listings in a single 6-month period in 2016 that were published in 254 journals in the categories of behavioral sciences, psychology, nursing, nutrition and dietetics, rehabilitation, and surgery. Prospective registration for all submitted articles was required for 29 journals (11.4%), and an additional 12 journals (4.7%) had conditional prospective registration. Of the trials, 189 (19.8%) were registered prospectively; 85 (8.9%) were registered prospectively with an adequately defined primary outcome or outcomes; and 37 (3.9%) published primary outcomes consistent with prospectively, adequately registered outcomes. Prospective registration was about twice as high in journals that required prospective registration as a condition for publication (33 [33.7%]), as compared with journals without such a policy (156 [18.2%]). Our findings suggest, however, that even when prospective registration policies are in place, they are rarely enforced.

Despite research demonstrating bias and waste owing to misrepresented research results,^37,38,39 our findings suggest that many researchers and journal editors may still be unaware of the importance of trial registration and transparent outcome reporting.⁴⁰ Research funders and ethics committees are well positioned to effectively enforce registration requirements.¹⁰ In the United States, in 2017, the National Institutes of Health began to require preregistration and results reporting of all clinical trials that it funds.⁴¹ Other governmental and nongovernmental funding organizations have made similar commitments.^1,42 However, the degree to which these requirements for preregistration are enforced or whether the quality of trial registrations is reviewed by funders is not known. A publicly visible report card that would grade journals on their degree of compliance with trial registration requirements and enforcement could encourage needed change.⁴³

Limitations

One limitation of this study is that we focused on journals in specialty areas and not other types of journals. We did not collect information on registration of trials of drugs, biologic, and devices that may have been published in the 254 journals in this study. A study that reviewed trials published in 2014 in the 6 highest-impact general medical journals found that 79% of trials of Food and Drug Administration–regulated interventions complied with ICMJE preregistration requirements, as compared with 58% of trials of nonregulated interventions.¹² Another limitation is that we used journal policy at the time of trial listing on PubMed rather than at the date of manuscript submission. Our review of internet archives and tracking of journal policies, however, found that few journals had recently changed registration policies; thus, the main findings would not likely be affected by any undetected changes. We did not examine whether included trials reported summary results in clinical trial registries, which would be an important area for future research.

Conclusions

Few specialty journals that publish nonregulated intervention trial results have adopted comprehensive trial registration policies. Although these policies are in place, they are rarely enforced. Without universal trial registration, it is difficult to interpret the meaning of trial results because of inherent biases, including selective reporting and publication bias. Journals and other entities with a stake in clinical research, including researchers, funding agencies, ethics review boards, and academic medical centers, should be concerned about these findings. They should make a concerted effort to improve trial registration practices.

Supplement.

eTable 1. Journal Inclusion/Exclusion by 2014 Thomson Reuters Science Citation Index—Expanded (SCIE) Categories

eTable 2. PubMed Search Terms

eTable 3. Included Trials and Registration Status Classification