Nonscientific Factors Associated with Acceptance for Publication in The Journal of Bone and Joint Surgery (American Volume)

Abstract

Background: While it is widely accepted that scientific factors may render a study more likely to be accepted for publication, it is less clear whether nonscientific factors may also be associated with publication. The purpose of this study was to identify the nonscientific factors associated with acceptance for publication by The Journal of Bone and Joint Surgery (American Volume).

Methods: A total of 1173 manuscripts submitted to The Journal of Bone and Joint Surgery between January 1, 2004, and June 30, 2005, for publication as scientific articles were analyzed as part of a study on publication bias in the editorial decision-making process. Information was collected on nonscientific factors plausibly associated with acceptance for publication, including study location, conflict-of-interest disclosure, sex of the author, primary language, and the number of prior publications by the corresponding author in frequently cited orthopaedic journals. The final disposition term (acceptance or rejection) was recorded, and logistic regression was used to identify factors associated with acceptance for publication.

Results: Manuscripts from countries other than the United States or Canada were significantly less likely to be accepted (odds ratio, 0.51; 95% confidence interval, 0.28 to 0.92; p = 0.026). Factors positively associated with acceptance for publication were conflict-of-interest disclosure involving a nonprofit entity (odds ratio, 1.92; 95% confidence interval, 1.35 to 2.73; p < 0.001) and ten or more prior publications in frequently cited orthopaedic journals by the corresponding author (odds ratio, 2.01; 95% confidence interval, 1.33 to 3.05; p = 0.001). We did not find a significant association between acceptance and conflict-of-interest disclosure involving a for-profit company, sex of the corresponding author, or primary language.

Conclusions: Manuscripts submitted to The Journal of Bone and Joint Surgery were more likely to be accepted if they were from the United States or Canada, reported a conflict of interest related to a nonprofit entity, or were authored by an individual with ten or more prior publications in frequently cited orthopaedic journals.

Orthopaedic journals are charged with selecting research for publication on the basis of scientific validity and importance to readers¹. It is widely accepted that scientific factors, such as study methodology and level of evidence, may render a study more likely to be accepted for publication by a prestigious orthopaedic journal². However, whether nonscientific factors may also be associated with acceptance for publication remains less clear.

The existing evidence suggests that certain nonscientific factors may render a manuscript more likely to be accepted for publication. Prior studies of the editorial decision-making process at the Journal of the American Medical Association, Lancet, British Medical Journal, and Annals of Internal Medicine have found that these general medicine journals are more likely to accept studies that are conducted in the same country as the journal in question^3,4. Disclosure of a conflict of interest was associated with a higher rate of acceptance in one study³, but this finding has not been consistent⁴. Manuscripts submitted by female first authors were found in two studies to be 17% and 21% less likely to be accepted for publication, although these differences did not achieve significance (p > 0.05)^3,4. Among abstracts submitted to the annual Scientific Sessions research meeting of the American Heart Association, submissions from non-English-speaking countries were significantly less likely to be accepted for presentation (p < 0.001)⁵. Finally, while the influence of an author's prior success in getting research published has not been studied in the context of the editorial decision-making process, the perception certainly exists that prestigious journals are more likely to accept manuscripts submitted by well-published authors.

Recently, Lynch et al. investigated the effects of certain scientific and nonscientific factors on the rates of acceptance by The Journal of Bone and Joint Surgery (American Volume) and found that commercially funded and U.S.-based research was more likely to be accepted for publication⁶. However, that study had a small sample size (209 manuscripts), was limited to manuscripts about hip and knee arthroplasty, and did not control for potentially confounding variables that could bias the association between manuscript characteristics and rates of acceptance. In addition, no author-related factors (such as sex, language, or publication history) were considered.

As such, it remains unclear whether nonscientific factors may be associated with acceptance for publication by a prestigious orthopaedic journal. The purpose of the present study was to identify the nonscientific factors associated with acceptance for publication by The Journal of Bone and Joint Surgery. In particular, we sought to determine the effects of study location and conflict-of-interest disclosure, as well as the author-related factors of sex, primary language, and number of prior publications in frequently cited orthopaedic journals.

Materials and Methods

Eligibility Criteria

All 1181 clinical and basic-science manuscripts submitted to The Journal of Bone and Joint Surgery between January 1, 2004, and June 30, 2005, for publication as scientific articles were analyzed as part of a study on publication bias in the editorial decision-making process. (The nonscientific factors associated with acceptance for publication are analyzed and reported in this article. Scientific factors associated with acceptance have been analyzed and reported in a previous article². The study sample also includes 209 manuscripts on adult reconstruction that were recently studied by Lynch et al.⁶.) Case reports and review articles were not included. Eight manuscripts for which abstracts were unavailable were excluded, leaving 1173 studies available for analysis. Manuscript review was conducted retrospectively.

Ethics

Beginning on January 1, 2004, all authors submitting work to The Journal of Bone and Joint Surgery for publication were informed that “The Journal shall have the right to use (and to permit others to use) the Data in reviewing and/or editing the Work and for any other purpose other than the creation or publication of any other work based exclusively on the Data.” The review and analysis of submitted manuscripts is covered by this statement.

Manuscript Characteristics and Results of the Editorial Review Process

For each submission, The Journal of Bone and Joint Surgery provided unblinded documents containing information on nonscientific factors associated with the manuscript. These documents were reviewed by one investigator (K.O.). On the basis of the address of the corresponding author, the study location was classified into one of five regions: United States; Canada; Europe and Australia; Asia and the Middle East; or Latin America, the Caribbean, and Africa. Conflict-of-interest disclosure forms completed by manuscript authors were reviewed, and self-reported conflicts were classified as being related to a for-profit company (i.e., industry) or a not-for-profit organization (i.e., nonindustry). All reported conflicts, including research support, royalties, stock options, and consultant or employee status, were considered.

The sex of the corresponding author was categorized as male or female on the basis of the author's name, with use of the knowledge that many names are associated with one sex or the other (e.g., “Jennifer” for women and “Dennis” for men). In cases where it was not possible to determine the author's sex on the basis of name, including initials and names not associated with one sex or the other (e.g., “Pat”), sex was recorded as unknown. Primary language was classified as English or non-English, depending on whether English was an official language in the country of the corresponding author. To determine the number of prior publications in frequently cited orthopaedic journals by the corresponding author, PubMed was searched to identify all articles by the corresponding author that had been published at the time of manuscript submission in the five most frequently cited orthopaedic journals (The Journal of Bone and Joint Surgery [American Volume], The Journal of Bone and Joint Surgery [British Volume], Clinical Orthopaedics and Related Research, Spine, and the Journal of Orthopaedic Research)⁷.

In analyzing the effects of nonscientific factors on the rates of acceptance, we controlled for a number of scientific factors, including direction of study findings, study type (clinical or nonclinical), level of evidence, prospectiveness, blinding, controlling, and sample size. Information on these scientific variables was extracted by three individuals with advanced training in clinical epidemiology (K.O., C.T.M., and M.B.) from blinded manuscripts provided by The Journal of Bone and Joint Surgery, as described in a prior publication².

For each manuscript, the final disposition term (acceptance or rejection) was reported by The Journal of Bone and Joint Surgery and was recorded by one investigator (K.O.).

Data Analysis

Analyses of power and reliability were conducted as described in a prior publication². In the multivariate analysis, multiple logistic regression was used to adjust for all variables simultaneously. Associations were estimated by odds ratios and 95% confidence intervals. P values were not adjusted for multiple comparisons, and a p value of <0.05 was considered significant. All tests were two-sided. Statistical analysis was performed with use of SAS (version 9; SAS, Cary, North Carolina), Stata (version 9; StataCorp, College Station, Texas), and SPSS (version 15.0; SPSS, Chicago, Illinois).

Results

Of 1181 manuscripts submitted to The Journal of Bone and Joint Surgery for publication as scientific articles between January 1, 2004, and June 30, 2005, 1173 manuscripts, including 787 clinical studies and 386 nonclinical studies, met the inclusion criteria. Nearly half (49.8%; 584) of all 1173 submitted manuscripts were from the United States. Approximately one-third (34.8%; 348) of the 1001 manuscripts that had provided the information reported a conflict of interest, with disclosures involving nonprofit entities being slightly more common than those involving for-profit companies. Twenty-two percent (255) of the 1173 manuscripts gained acceptance for publication, while 78% (918 manuscripts) were rejected (Table I).

TABLE I.

Characteristics of the 1173 Submitted Manuscripts

Type
Clinical	787 (67.1%)
Nonclinical	386 (32.9%)
Subspecialty field
Adult reconstruction of the hip	151 (12.9%)
Shoulder and elbow	112 (9.6%)
Trauma	102 (8.7%)
Adult reconstruction of the knee	95 (8.1%)
Pediatric orthopaedics	68 (5.8%)
Spine	65 (5.5%)
Tumor and metabolic disease	59 (5.0%)
Foot and ankle	47 (4.0%)
Sports medicine and arthroscopy	43 (3.7%)
Hand and wrist	29 (2.5%)
Practice management	20 (1.7%)
Rehabilitation medicine	6 (0.5%)
Basic science	376 (32.1%)
Study location
United States	584 (49.8%)
Canada	54 (4.6%)
Europe and Australia	321 (27.4%)
Asia and the Middle East	203 (17.3%)
Latin America, the Caribbean, and Africa	11 (0.9%)
Conflict-of-interest disclosure*
None	653 (65.2%)
Involving a for-profit company	172 (17.2%)
Involving a nonprofit entity	219 (21.9%)
Corresponding author sex
Male	1027 (87.6%)
Female	116 (9.9%)
Unknown	30 (2.6%)
Primary language
English	758 (64.6%)
Non-English	415 (35.4%)
Prior publications in frequently cited orthopaedic journals by the corresponding author
0	352 (30.0%)
1-9	496 (42.3%)
≥10	325 (27.7%)
Final disposition term
Acceptance	255 (21.7%)
Rejection	918 (78.3%)
Total	1173 (100.0%)

^*Excludes 172 studies for which conflict-of-interest information was missing. The authors of forty-three studies reported conflicts of interest involving both for-profit and nonprofit companies.

In the multivariate analysis, manuscripts from countries other than the United States or Canada were significantly less likely to be accepted (odds ratio, 0.51; 95% confidence interval, 0.28 to 0.92; p = 0.026). Factors positively associated with acceptance for publication were a conflict-of-interest disclosure involving a nonprofit entity (odds ratio, 1.92; 95% confidence interval, 1.35 to 2.73; p < 0.001) and ten or more prior publications in frequently cited orthopaedic journals by the corresponding author (odds ratio, 2.01; 95% confidence interval, 1.33 to 3.05; p = 0.001). We did not find a significant association between acceptance and conflict-of-interest disclosure involving a for-profit company, sex of the corresponding author, or primary language (Table II).

TABLE II.

Multivariate Analysis

	Acceptance Rate	Crude Odds Ratio (95% Confidence Interval)	Adjusted Odds Ratio (95% Confidence Interval)	P Value
Study location
United States or Canada (n = 638)	28.1% (179)	1.00	1.00	–
Other countries (n = 535)	14.2% (76)	0.43 (0.32-0.57)	0.51 (0.28-0.92)	0.026
Conflict-of-interest disclosure*
None (n = 653)	20.1% (131)	1.00	1.00	–
Involving a for-profit company (n = 172)	32.6% (56)	1.62 (1.13-2.32)	1.28 (0.86-1.89)	0.223
Involving a nonprofit entity (n = 219)	34.3% (75)	1.87 (1.35-2.59)	1.92 (1.35-2.73)	<0.001
Corresponding author sex†
Male (n = 1027)	22.0% (226)	1.00	1.00	–
Female (n = 116)	24.1% (28)	1.13 (0.72-1.77)	1.31 (0.80-2.13)	0.283
Primary language
English (n = 758)	25.7% (195)	1.00	1.00	–
Non-English (n = 415)	14.5% (60)	0.49 (0.35-0.67)	1.06 (0.57-1.96)	0.848
Prior publications in leading orthopaedic journals by corresponding author
0 (n = 352)	15.3% (54)	1.00	1.00	–
1-9 (n = 496)	19.6% (97)	1.34 (0.93-1.93)	1.22 (0.83-1.79)	0.319
≥10 (n = 325)	32.0% (104)	2.60 (1.79-3.77)	2.01 (1.33-3.05)	0.001
Findings‡
Positive (n = 620)	21.3% (132)	1.00	1.00	–
Negative or neutral (n = 235)	23.0% (54)	1.10 (0.77-1.58)	0.92 (0.63-1.36)	0.685
Type
Clinical (n = 787)	23.4% (184)	1.00	1.00	–
Nonclinical (n = 386)	18.4% (71)	0.74 (0.54-1.00)	0.50 (0.31-0.81)	0.005
Level of evidence§
I or II (n = 178)	30.9% (55)	1.00	1.00	–
III or IV (n = 544)	21.5% (117)	0.60 (0.41-0.88)	0.50 (0.31-0.81)	0.005
Prospectiveness
No (n = 516)	23.1% (119)	1.00	1.00	–
Yes (n = 657)	20.7% (136)	0.87 (0.66-1.15)	0.74 (0.50-1.09)	0.123
Blinding
No (n = 1140)	21.6% (246)	1.00	1.00	–
Yes (n = 33)	27.3% (9)	1.36 (0.63-2.97)	1.09 (0.47-2.51)	0.840
Controlling
No (n = 700)	21.4% (150)	1.00	1.00	–
Yes (n = 473)	22.2% (105)	1.05 (0.79-1.39)	0.96 (0.68-1.36)	0.810
Sample size#
Small (1-15) (n = 197)	18.8% (37)	1.00	1.00	–
Medium (16-99) (n = 561)	21.9% (123)	1.31 (0.91-1.89)	1.22 (0.82-1.82)	0.334
Large (≥100) (n = 328)	25.0% (82)	1.56 (1.05-2.31)	1.20 (0.76-1.88)	0.439

^*Excludes 172 manuscripts for which conflict-of-interest information was missing.

^†Excludes thirty manuscripts for which sex of author was unknown.

^‡Excludes 318 manuscripts with findings classified as not applicable (NA).

^§Excludes 386 nonclinical studies and sixty-five clinical studies with level of evidence classified as not applicable (NA).

^#Excludes eighty-seven manuscripts for which sample size was not reported.

Discussion

In this observational study of manuscripts submitted to The Journal of Bone and Joint Surgery (American Volume), nonscientific factors significantly associated with acceptance for publication were study location, conflict-of-interest disclosure involving a nonprofit entity, and the number of prior publications in frequently cited orthopaedic journals by the corresponding author. We did not find significant associations between acceptance and conflict-of-interest disclosure related to a for-profit company, sex of corresponding author, or primary language.

Our finding that studies from the United States and Canada are accepted at higher rates is consistent with a number of prior reports^3,4,6,8. Lynch et al., in their observational study of arthroplasty manuscripts submitted to The Journal of Bone and Joint Surgery, found that studies from the United States were accepted 39% of the time compared with 22% for studies from other countries⁶. Given that The Journal of Bone and Joint Surgery reviewers are blinded to study location in their review of submitted manuscripts, it is unlikely that this finding represents overt bias. While this finding could be related to differential facility with the English language, this possibility is rendered less likely by the fact that the observed differences persisted even after controlling for official language in the country of the corresponding author. As Lynch et al. noted in their Discussion section, this finding could also reflect an unconscious preference for American and/or Canadian research, which tends to be more familiar to U.S.-based reviewers and editors⁶. Studies conducted in fields other than orthopaedics have indicated that reviewers and editors may be more likely to accept studies conducted in their own countries. For example, Lee et al. studied manuscripts submitted to the British journals Lancet and British Medical Journal as well as the American journal Annals of Internal Medicine, and they found that having a corresponding author who lived in the same country as the journal in question was a significant predictor of acceptance (odds ratio, 1.99; 95% confidence interval, 1.14 to 3.46)³. Similarly, a study of manuscripts submitted to Gastroenterology found that U.S. papers were ranked slightly more favorably by non-U.S. reviewers (p = 0.09) but much more favorably by U.S. reviewers (p = 0.001)⁸. On the other hand, manuscripts from the United States and Canada could have been accepted at higher rates in our study because they are somehow better or more appropriate for publication. We controlled for a wide range of quality indicators, including level of evidence, prospectiveness, blinding, controlling, and sample size, but these are only some of the many factors that go into determining the overall quality of a study.

Our detection of an association between manuscript acceptance and conflict-of-interest disclosure involving a nonprofit entity is also in agreement with the findings of Lynch et al., who reported that studies with noncommercial and/or philanthropic funding had an acceptance rate of 55% compared with 25% for nonfunded studies⁶. While there are many possible explanations for this observed association, perhaps most likely is that research grants from nonprofit entities are often awarded on a competitive basis, meaning that studies chosen to receive such funding may be of superior quality and thus more likely to be accepted for publication.

While Lynch et al. also found that commercially funded studies were more likely to be accepted for publication⁶, we did not detect such an association. In particular, we found that studies with conflicts of interest involving for-profit companies were accepted at rates similar to those of studies without a declared conflict of interest. The specific reasons for this discrepancy are unclear, but may be related to the fact that Lynch et al. examined manuscripts from the technologically intensive field of adult reconstruction, in which commercial funding may be especially important⁶. However, when we restricted our analysis to manuscripts about hip and knee arthroplasty, conflict-of-interest disclosure involving a for-profit company remained nonsignificant as a predictor of acceptance (odds ratio, 1.43; 95% confidence interval, 0.65 to 3.12; p = 0.375) in comparison with studies without a declared conflict of interest. It is perhaps more likely that this discrepancy stems from the fact that our study controlled for potential confounders while the Lynch study did not. In preliminary univariate analysis of our data performed without controlling for confounders, conflict-of-interest disclosure involving a for-profit entity was positively associated with acceptance for publication (odds ratio, 1.62; 95% confidence interval, 1.13 to 2.32; p = 0.009) in comparison with studies without a declared conflict of interest. After controlling for potential confounders, however, this association was no longer significant. It would be interesting to see whether the association between commercial funding and manuscript acceptance reported by Lynch et al. remained significant after controlling for potential confounders.

To our knowledge, the association between manuscript acceptance and the number of prior publications in frequently cited journals by the corresponding author has not been examined previously. However, the perception certainly exists that well-published researchers are more likely to have their research accepted by first-tier journals such as The Journal of Bone and Joint Surgery. Given that reviewers are blinded to author identity throughout the editorial review process, it is unlikely that this finding is the result of conscious bias. Instead, it may be related to investigator experience, with well-published investigators having greater research skill or more familiarity with the publication process. As a result, they may conduct studies and prepare manuscripts that are more likely to be published. This is a finding that should be further investigated in future studies.

The results of our investigation must be considered within the context of its study design. As noted previously², our study benefits from the fact that it had a large sample size and controlled for a number of potential confounders. Since the vast majority of manuscripts submitted to The Journal of Bone and Joint Surgery during the study period were included, our results may be generalizable to the publication of orthopaedic research broadly, at least in a general orthopaedics journal.

Our study does have its limitations, however. Given that information on conflict of interest was self-reported, it is possible that some conflicts may have been underreported (either intentionally or unintentionally). In examining the effects of author characteristics on the rates of acceptance, we chose to focus on the corresponding author, who is often the most senior and most experienced member of the research team. While the results of our study may have been different had we chosen to focus on a different author (e.g., the first author), this choice is unlikely to have introduced bias given that the corresponding author was examined in the case of each manuscript. The fact that the sex of the corresponding author was assigned on the basis of first name could have resulted in misclassification for authors with names commonly associated with the opposite sex, but this situation is unlikely to have occurred very often. The fact that we chose not to adjust for multiple comparisons in conducting exploratory analyses may be considered a limitation because of an increased risk of detecting false positives. However, in making this choice, we followed the example set by previous authors of similar studies, including Olson et al.⁴ and Lee et al.³. Finally, we followed the example of prior studies^3,4 by using odds ratios to estimate risk ratios. While odds ratios can overestimate or underestimate risk ratios when outcomes are common⁹, this was not the case in our study, given that acceptance rates were ≤30% for almost all groups.

While many authors may find it difficult to get their research published by prestigious orthopaedic journals such as The Journal of Bone and Joint Surgery, researchers from outside the United States or Canada, without grant support, or without extensive publication experience may face barriers that are especially restrictive. The specific reasons underlying these difficulties are unclear, but it is possible that they could be addressed by outreach efforts on the part of orthopaedic journals, experienced researchers, and others. Investigators conducting research in less developed regions of the world may be able to benefit from editorial support provided by The Journal of Bone and Joint Surgery and other prestigious journals. Similarly, authors without prior publications in top orthopaedic journals may be able to improve the quality of their submissions by seeking advice from researchers who have been successful in getting their work published in the past.

Orthopaedic research should be published on the basis of scientific merit, not author identity. By leveling the playing field and eliminating the aforementioned barriers to publication, the orthopaedic community will be acting to ensure that high-quality research gets published, regardless of the nonscientific factors associated with the manuscript. Such actions have the potential to improve not only the quality of the orthopaedic literature but also—in the current era of evidence-based medicine—the care of our patients as well.