Age of Data in Contemporary Research Articles Published in Representative General Radiology Journals

Abstract

Objective

To analyze and compare the age of data in contemporary research articles published in representative general radiology journals.

Materials and Methods

We searched for articles reporting original research studies analyzing patient data that were published in the print issues of the Korean Journal of Radiology (KJR), European Radiology (ER), and Radiology in 2017. Eligible articles were reviewed to extract data collection period (time from first patient recruitment to last patient follow-up) and age of data (time between data collection end and publication). The journals were compared in terms of the proportion of articles reporting the data collection period to the level of calendar month and regarding the age of data.

Results

There were 50, 492, and 254 eligible articles in KJR, ER, and Radiology, respectively. Of these, 44 (88%; 95% confidence interval [CI]: 75.8–94.8%), 359 (73%; 95% CI: 68.9–76.7%), and 211 (83.1%; 95% CI: 78–87.2%) articles, respectively, provided enough details of data collection period, revealing a significant difference between ER and Radiology (p = 0.002). The age of data was significantly greater in KJR (median age: 826 days; range: 299–2843 days) than in ER (median age: 570 days; range: 56–4742 days; p < 0.001) and Radiology (median age: 618; range: 75–4271 days; p < 0.001).

Conclusion

Korean Journal of Radiology did not fall behind ER or Radiology in reporting of data collection period, but showed a significantly greater age of data than ER and Radiology, suggesting that KJR should take measures to improve the timeliness of its data.

INTRODUCTION

The purpose of scientific journals is to share new information and novel discoveries with professionals of the relevant specialty. This is critical in medicine, where the application of new knowledge from journals advances clinical practice. For this reason, the timeliness of research data is essential; indeed, some journals provide submitting authors with specific guidelines in this regard (1). Moreover, in radiology, data timeliness may be especially important since the field relies heavily on new technology particularly digital technology (2,3,4,5,6,7), which develops and changes faster than in other disciplines. Thus, the timeliness of data in radiology journals may be an important indicator of quality or impact, and the present study analyzed and compared several representative general radiology journals in terms of the age of the data published therein.

MATERIALS AND METHODS

This study did not require an Institutional Review Board approval.

Study Selection

We analyzed the Korean Journal of Radiology (KJR), European Radiology (ER), and Radiology, which have the highest impact factors among general radiology journals published in Asia, Europe, and America, respectively, according to the 2017 Journal Citation Reports (Clarivate Analytics, Philadelphia, PA, USA). We hand-searched the print issues of the journals published in 2017 (6 issues of KJR, 12 issues of ER, and 12 issues of Radiology) to find reports of original research studies that had collected and analyzed patient data. Experimental studies involving animals, phantoms, or other laboratory conditions were excluded. The literature was screened by two board-certified radiologists, both of whom were proficient in reviewing radiology research articles.

Data Extraction

Firstly, the eligible articles were reviewed to determine whether they had stated the start and end of the data collection period to the level of the calendar month (Fig. 1). The data collection period was defined as the time from recruitment of the first patient to the latest follow-up of the study patients, as defined elsewhere (Fig. 1) (8). We considered month, but not date, as it is rare for published radiology research studies to report the data collection period to the level of date. Next, we made a universal assumption that the start and end dates of the data collection period fell on the first and last days, respectively, of the reported months. For example, if the data collection period ranged from January 2011 to April 2015, the start and end dates were assumed to be January 01, 2011 and April 30, 2015, respectively. As most studies only report the shortest and longest follow-up durations, rather than the exact follow-up periods of each study participant, the exact time of each patient's latest follow-up is rarely given. Therefore, we determined the time of the latest follow-up by 1) assuming that the earliest study enrollee had the longest follow-up and that the last enrollee had the shortest follow-up, 2) calculating two time points by adding the longest and shortest follow-up lengths to the start and end, respectively, of enrollment, and 3) choosing the later of these two time points. This method gave a reasonable estimate of the latest follow-up time. Secondly, the type of study design was determined (retrospective, prospective, or unclear). Thirdly, the exact date of publication, according to the time at which the full-text article became available online, was recorded by referring to the PubMed data of ER and Radiology, both of which use electronic publication ahead of print, and through contact with the editorial office of KJR, which does not use electronic publication ahead of print. Two board-certified radiologists analyzed the eligible articles to extract data. When there was ambiguity, a third reviewer experienced in the relevant methodology was invited.

Fig. 1. Definitions of data collection period and age of data.

Outcome Measures and Statistical Analysis

The proportion of articles that revealed the data collection period to the level of the calendar month was calculated, along with its 95% confidence interval (CI). The proportion was compared between journals in a pairwise manner using Fisher's exact test: KJR vs. ER, KJR vs. Radiology, and ER vs. Radiology. In articles that revealed the data collection period, the age of the research data was calculated as the time between the end of the data collection period and the date of publication (Fig. 1). The distribution of the age of data was checked using a histogram and the Kolmogorov-Smirnov test, and appropriate summary statistics were obtained. Next, the age of the data was compared between journals in the aforementioned pairwise manner using the Wilcoxon rank-sum test. This statistical comparison was performed across all relevant articles, as well as separately for prospective and retrospective studies. The threshold p value for statistical significance was lowered to 0.017 (Bonferroni adjustment) to maintain the overall alpha at 5% after the three pairwise comparisons were made. Statistical analyses were performed using SPSS Statistics for Windows, version 21.0 (IBM Corp., Armonk, NY, USA).

RESULTS

In total, 50 KJR (9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,56,57,58), 492 ER, and 254 Radiology articles reported original research studies analyzing patient data. Of these, 44 (88%; 95% CI: 75.8–94.8%) (9,10,12,13,14,15,16,17,18,19,20,21,23,24,26,27,28,29,30,31,32,34,35,36,37,38,39,40,41,42,43,44,47,48,49,50,51,52,53,54,55,56,57,58), 359 (73%; 95% CI: 68.9–76.7%), and 211 (83.1%; 95% CI: 78–87.2%) articles, respectively, revealed the start and end of data collection to the level of calendar month. The point estimate value of this proportion was slightly larger in KJR than in ER and Radiology, although the difference was not statistically significant. In contrast, ER demonstrated a significantly lower proportion than Radiology in this regard (Table 1). Further breakdowns according to study type are also presented in Table 1, and the separate results from prospective and retrospective studies were mostly consistent with the overall results.

Table 1. Articles Included in Study and Proportion of Articles Reporting Data Collection Period.

Article Type	Number of Articles			P
	KJR	ER	Radiology	KJR vs. ER	KJR vs. Radiology	ER vs. Radiology
All eligible articles	50	492	254
Data collection period reported	44 (88, 75.8–94.8)	359 (73, 68.9–76.7)	211 (83.1, 78–87.2)	0.018	0.528	0.002*
Data collection period unreported	6	133	43
According to study design
Prospective	11	175	125
Data collection period reported	8 (72.7, 42.9–90.8)	121 (69.1, 61.9–75.5)	102 (81.6, 73.8–87.5)	> 0.999	0.440	0.016*
Data collection period unreported	3	54	23
Retrospective	36	313	126
Data collection period reported	33 (91.7, 77.4–97.9)	237 (75.7,70.7–80.2)	108 (85.7,78.5–90.9)	0.034	0.415	0.021
Data collection period unreported	3	76	18
Unclear	3	4	3
Data collection period reported	3 (100, 38.3–100)	1 (25, 3.4–71.1)	1 (33.3, 5.6–79.8)	0.143	0.400	> 0.999
Data collection period unreported	0	3	2

Data represent number of articles, with % of articles and corresponding 95% confidence intervals in parentheses. ^*Statistically significant after accounting for multiple comparisons (p < 0.017). ER = European Radiology, KJR = Korean Journal of Radiology

The age of data (the time between the end of the data collection period and the date of publication) was skewed to the left in all three journals (Fig. 2) and was significantly larger in KJR (median age: 826 days across all relevant articles) than in ER (median age, 570 days) and Radiology (median age: 618 days) (Table 2). The difference was more pronounced when retrospective studies were considered separately (Table 2).

Fig. 2. Distribution of age of data.

Table 2. Comparison of Age of Data between Journals.

Article Type	Age of Data, Median (Range)			P*
	KJR	ER	Radiology	KJR vs. ER	KJR vs. Radiology	ER vs. Radiology
All eligible articles
No. of articles	44	359	211
Kolmogorov-Smirnov test, p value	0.004	< 0.001	< 0.001
Median No. of days (range)	826 (299–2843)	570 (56–4742)	618 (75–4271)	< 0.001†	< 0.001†	0.505
Prospective study
No. of articles	8	121	102
Kolmogorov-Smirnov test, p value	0.199	< 0.001	< 0.001
Median No. of days (range)	666 (377–1351)	558 (56–3990)	556 (133–4271)	0.420	0.501	0.909
Retrospective study
No. of articles	33	237	108
Kolmogorov-Smirnov test, p value	0.003	< 0.001	< 0.001
Median No. of days (range)	982 (299–2843)	585 (69–4742)	643 (75–2501)	< 0.001†	< 0.001†	0.323

^*Comparison of age of data between journals, ^†Statistically significant after accounting for multiple comparisons (p < 0.017).

DISCUSSION

Korean Journal of Radiology seemed not to fall behind ER and Radiology regarding the proportion of articles that reported the data collection period, although the results of prospective studies may be inconclusive because too few relevant articles were published in KJR (11 articles). Nonetheless, this result may indicate that the journal has good quality control in the peer review and editorial processes. However, the age of data was significantly greater in KJR than in ER (approximately 8.5-month difference in median age) and Radiology (approximately 7-month difference in median age). Furthermore, the greater age of the data was more pronounced when retrospective studies were considered separately (approximately 13 months older than in ER and 11 months older than in Radiology). It is likely that the KJR contains older data because authors generally submit their manuscripts to higher-ranked journals first, descending the ranks of journals if their manuscript is rejected. As a result, lower-ranked journals would naturally contain older data. The journal impact factors of KJR, ER, and Radiology for 2017, according to the Journal Citation Reports (Clarivate Analytics), were 3.072, 4.027, and 7.469, respectively. This would explain why KJR differs from ER or Radiology in this regard.

However, there was no significant difference in the age of data between ER and Radiology, despite the apparent difference in journal impact factor, perhaps because ER publishes accepted articles more swiftly than Radiology in electronic publication format ahead of print publication. Specifically, in the articles analyzed in the present study, the median interval between initial electronic publication and official assignment to a monthly print issue was 225 days (range: 130–316 days) for ER and 138 days (range: 39–244 days) for Radiology.

Korean Journal of Radiology recently assessed the quality of its research articles in terms of conformity to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 guidelines (59) and the adequacy reporting reliability analysis for diagnostic tests (60). The current study revealed another area in which KJR could improve its quality and impact, and we would even suggest some specific measures. Firstly, because the age of data in the KJR differed more markedly from that in ER and Radiology in retrospective studies than in prospective studies, the journal could encourage authors to make data as recent as possible by updating study data in the revision process. Such an update would be possible in the case of retrospective studies, although it is likely infeasible in most prospective studies. Secondly, the journal could further shorten the time from initial submission to publication by ensuring more rapid review, allowing electronic publication before print, and publishing monthly in a smaller volume. Indeed, bimonthly publication delays those articles that were early accepted in the 2-month cycle period.

This study had several limitations. Firstly, as studies generally do not report each individual's exact follow-up duration, we assumed that the earliest study enrollee had the longest follow-up and the latest enrollee had the shortest follow-up; this may not always have been the case. However, we believe that this approach gave a reasonable estimate of the data collection period and was sufficient to analyze the macroscopic, between-journal difference. Secondly, one published study (8) defined the age of data slightly differently from the present study as the time from the mid-point of the data collection period to the publication date. This previous study analyzed prospective randomized trials, wherein their definition fit better. However, this definition would have been inappropriate in our study, wherein the majority of studies analyzed were retrospective. In retrospective studies, a data collection time extending further into the past may be more beneficial. However, for the purposes of the present study, it may have penalized the results, as the data age became greater. Thirdly, knowledge of the details of any rejections by other journals (number of rejections, by what journals, etc.) before submission to KJR would have been helpful in further understanding the greater age of KJR data. However, we empirically found that it was difficult to collect such “sensitive” information with consistency.

In conclusion, KJR did not fall behind ER or Radiology with regard to the proportion of articles that reported the age of data. However, the age of data was significantly greater in KJR than in ER and Radiology, suggesting that the journal should introduce some measures to improve the timeliness of data that it publishes.