Study designs and statistical methods in the Journal of Family and Community Medicine: 1994-2010

Abstract

Introduction:

The Journal of Family and Community Medicine (JFCM) is the official peer reviewed scientific publication of the Saudi Society of Family and Community Medicine. Unlike many peer medical journals, the contents of JFCM, have never been analyzed. The objective of this study was to perform an analysis of the contents of the JFCM over a 16-year period to discern the study designs and statistical methods used with a view to improving future contents of the journal.

Materials and Methods:

All volumes of the JFCM, from 1 January 1994 to 31 December 2010 were hand searched for research articles. All papers identified as original articles were selected. For every article, the study designs and the statistical methods used were recorded. Articles were then classified according to their statistical methods and study designs. The frequency of study designs was calculated as a simple percentage of the total number of articles, while the frequency of statistical methods was calculated as a percentage of articles that used those statistical methods.

Results:

A total of 229 articles were analyzed. Of these, 66 (28.8%) either reported no statistics or reported simple summaries. The cross-sectional design was used in 175 (76.4%) of all analyzed articles. Statistical methods were used in 163 (71.2%) articles. Chi-squared test was used in 111 (68.1%) articles, and t-test used in 48 (29.4%) articles. Other common statistical tests were: Regression, which was used in 35 (21.5%) articles, ANOVA used in 23 (14.1%) articles, and odds ratio and relative risk tests which were used in 22 (13.5%) articles.

Conclusions:

The JFCM has a wide range of study designs and statistical methods. However, no article on experimental studies has been published in the JFCM since its inception.

INTRODUCTION

The Journal of Family and Community Medicine (JFCM), the official scientific publication of the Saudi Society of Family and Community Medicine was established in 1994 as a peer-reviewed journal on general medicine.[1] Three issues are published annually in in-print and on-line forms, and 3000 hard copies of every issue are distributed all over the Kingdom of Saudi Arabia.[1,2] Recently, in an attempt to answer the question: “Are we on the right track after 12 years of publication?”[2] the editors of JFCM used a readership survey and the journal's records and archives in order to “strengthen the successes and reduce difficulties”.[2] Despite all obstacles, the journal has successfully continued to accommodate a wide range of articles on general medicine, and has served as an important source of research in Saudi Arabia for the last 19 years. However, the content of the journal has never been reviewed. Surveying peer-reviewed medical journals for their contents in terms of study designs and statistical methods is not an uncommon practice.[3–14] Such a review is expected to serve as an objective approach to improving the content of scientific peer-reviewed journals both qualitatively and quantitatively. However, journals on general medicine, including JFCM are under-researched in this respect.[12,14] Therefore, the objective of this work was to review the JFCM from the year 1994 till 2010 to find out the study designs and statistical methods that have been used with the aim of improving the content of the journal.

MATERIALS AND METHODS

This was a retrospective review of documents. All volumes of the JFCM, from 1 January 1994 to 31 December 2010 were hand-searched for research articles. All papers identified as original or leading articles, both in English or Arabic, were selected. Other papers such as review articles, case reports, debates, opinions, letters to the editor, conference abstracts, or conference reports, were excluded. For every article, the study design and the statistical method used were recorded. Articles were then classified according to their statistical methods and study designs using criteria applied elsewhere.[3,13] If more than one statistical method was used in a paper, all were recorded, but the same statistical method applied in the same article repeatedly was recorded only once. The frequency of each category of study design used in an article was calculated as a simple percentage of the total number of articles, while the frequency of statistical methods was calculated as a percentage of articles that actually used statistical methods. The Stata program was used for data entry and the SPSS program for analysis.

RESULTS

A total of 229 articles were reviewed. Cross-sectional study design was the most commonly used. This was used in 175 (76.4%) articles [Table 1]. Although articles were classified by the term “cross-sectional study”, this was not necessarily the term of choice by the journal. Other terms included in this category were “survey” a “questionnaire-based study”.

Table 1.

Study designs used in the Journal of Family and Community Medicine 1994-2010 n=229

Out of the analyzed articles, 66 (28.8%) articles either reported no statistics or reported simple summaries such as percentage, mean, median, and standard deviation. Table 2 shows the range of statistical methods reported. The number of methods exceeds the number of articles as some reported more than one method. Statistical methods were used in 163 (71.2%) articles. Of the 163 articles, Chi-squared test was used in 111 (68.1%) articles, and t-test was used in 48 (29.4%) articles. Other common statistical tests were: Regression which was used in 35 (21.5%) articles, ANOVA used in 23 (14.1%) articles, and odds ratio and relative risk tests used in 22 (13.5%) articles. A large number of articles reported a wide range of statistical methods, which occurred only once.

Table 2.

Statistical methods used in papers reporting statistics in the Journal of Family and Community Medicine 1994-2010 n=163

DISCUSSION

The papers published in the JFCM seemed to be somewhat similar to papers published in other general practice journals in terms of study designs and statistical methods used [Tables 3 and 4]. However, both retrospective design and cross-section studies were more used in the JFCM than in any of the other journals (Cramer's V = 0.28, P < 0.0001). The most commonly used study design was the cross-sectional survey. This finding is consistent with other studies.[13,14] Rigby et al. in a survey, covering a period of one year, of study designs used in general practice journals in the UK, found that cross-sectional design was used in 24.1% of the articles reported in the British Medical Journal (BMJ), 39.5% of the articles in British Journal of General Practice (BJGP) and 35.1% of the articles in Family Practice.[14] In their review of papers published in five leading Chinese journals in 1995 and 1985, Wang and Zhang found that cross-sectional study was used in 47.4% and 42.5% in 1995 and 1985 respectively.[13] This is probably because cross-sectional study is the most commonly used design in research conducted in the medical field, particularly in primary care, family medicine, and general practice. This can be attributed to the difficulties encountered when conducting clinical trials in primary care, family medicine, and general practice.[15–18] These difficulties could be in methodology,[16] logistics,[17] or ethics.[18]

Table 3.

A comparison of study designs used in the Journal of Family and Community Medicine with other journals

Table 4.

A comparison of statistical methods used in the Journal of Family and Community Medicine with other journals

Consistent with other studies,[3,13,14] about one-quarter of all articles reported no statistics or reported simple summaries. Rigby et al. found that articles that reported no statistics or reported simple summaries constituted 29.1%, 32.4% and 40.7% of all studies published in the BMJ, BJGP and Family Practice, respectively.[14] Wang and Zhang found that 39.9% and 59.8% of all studies published in five leading Chinese journals in 1995 and 1985 respectively either reported no statistics or reported simple summaries.[13] In their study of 332 original research articles published in 1979 in the New England Journal of Medicine, Emerson and Colditz found that articles that reported no statistics or reported simple summaries constituted 27% of all articles.

Consistent also with other studies was the finding that the most commonly used statistical tests in JFCM were Chi-squared test and t-test. Rigby et al. found that the most commonly used tests were t-test and Chi-squared test in all studies published in the BMJ, BJGP and Family Practice in 1995. Wang and Zhangal so found that Chi-squared test and t-test were the most commonly used tests in all studies published in five leading Chinese journals in 1995 and 1985.

It seems that not much has changed in the last 30 years with regard to the most commonly used statistical methods. Besides the findings in this study, the above mentioned reviewed studies which covered a long period (1979-2004) showed no major change in the trend in the use of study designs and statistical methods. In their study of “the transfer of new statistical methods into the medical literature” Altman and Goodman indicated[19] stated that, “newer technical innovations still typically take 4-6 years before they achieve 25 citations in the medical literature.”

CONCLUSION

Apart from the absence of experimental studies, the content of JFCM was, somewhat, similar to other general medicine journals published in different parts of the world, in terms of study designs and statistical methods used. The Saudi Society of Family and Community Medicine can play a major role in improving the content of JFCM by encouraging doctors to conduct intervention research in primary care and general practice settings for publication in the JFCM. This would, consequently help secure JFCM's place as an important source for research in family and community medicine.