Scientific Production of Research Fellows at the Zagreb University School of Medicine, Croatia

Abstract

Aim

To evaluate scientific production among research fellows employed at the Zagreb University School of Medicine and identify factors associated with their scientific output.

Method

We conducted a survey among research fellows and their mentors during June 2005. The main outcome measure was publication success, defined for each fellow as publishing at least 0.5 articles per employment year in journals indexed in the Current Contents bibliographic database. Bivariate methods and binary logistic regression were used in data analysis.

Results

A total of 117 fellows (response rate 95%) and 83 mentors (100%) were surveyed. The highest scientific production was recorded among research fellows employed in public health departments (median 3.0 articles, interquartile range 4.0), compared with those from pre-clinical (median 0.0, interquartile range 2.0) and clinical departments (median 1.0, interquartile range 2.0) (Kruskal-Wallis, P = 0.003). A total of 36 (29%) research fellows published at least 0.5 articles per employment year and were considered successful. Three variables were associated with fellows’ publication success: mentor’s scientific production (odds ratio [OR], 3.14; 95% confidence interval [CI], 1.31-7.53), positive mentor’s assessment (OR, 3.15; 95% CI, 1.10-9.05), and fellows’ undergraduate publication in journals indexed in the Current Contents bibliographic database (OR, 4.05; 95% CI, 1.07-15.34).

Conclusion

Undergraduate publication could be used as one of the main criteria in selecting research fellows. One of the crucial factors in a fellow’s scientific production and career advancement is mentor’s input, which is why research fellows would benefit most from working with scientifically productive mentors.

Decreasing interest in scientific involvement among young graduate physicians has been identified in a number of studies (1-6). Various solutions for reversing this trend have been proposed in an attempt to increase the interest in scientific research among physicians. What was most commonly reported as having a positive effect on physicians’ research interest was undergraduate involvement in scientific work and subsequent publication of a research article (7-9). Other studies demonstrated that program characteristics and faculty size had an effect on publication output (10,11). The role of a supportive mentor (10,12), or a Resident Research Director (13) were also positively associated with scientific production. An additional year devoted to clinical research among surgical residents increased their scientific output (14). However, a recent study failed to confirm undergraduate scientific involvement as the predictor of productive scientific career in radiology (15). Only critical attitude, independence, inventiveness, and curiosity were correlated with research activity (16). Other undergraduate indicators, including grade point average, did not contribute to increased scientific production later in career (16). The situation is becoming even more worrying knowing that high initial interest in scientific research among recently graduated physicians decreased as their residency progressed (17). The reasons for a decreased publication output might include a lack of time, low interest in research, and insufficient mentor support (18). Inadequate senior staff and statistical or secretarial support were identified as major barriers to research activity (19). The same study identified high demand for clinical productivity, lack of protected research time, and a lack of research funding as additional obstacles (19).

In 1991, Ministry of Science, Education, and Sports of the Republic of Croatia established a program for research fellows, with an aim to attract the best graduate students to the positions at academic and research institutions. The number of research fellows gradually increased from 995 in 1991 to 2510 at the end of 2005 (20). Career advancement criteria for research fellows are strict, and defined by the Law on Scientific Work and Higher Education (21). Research fellows employed by the Zagreb University School of Medicine represented a total of 5% of all research fellows in Croatia in 2004 and 26% of all fellows employed in the biomedical field (20). The aim of this study was to evaluate the scientific production among research fellows from Zagreb University School of Medicine. We also aimed to identify the factors associated with a successful scientific output, which is the main career advancement requirement for research fellows.

Subjects and methods

Setting

Academic institutions in Croatia employ the best graduates as research fellows, with undergraduate grade point average as the main selection criterion. Fellows are employed on a six-year contract, and their salaries are provided by the Ministry of Science, Education, and Sports of the Republic of Croatia.

Zagreb University School of Medicine most commonly employs graduate physicians (MDs) as research fellows. Fellows are attached to a research project funded by the Ministry of Science, Education, and Sports. The principal investigator of the project has a crucial role in the process of selecting the fellows and becomes the fellows’ mentor.

Fellows are required to take part in undergraduate and postgraduate teaching. Those employed in clinical and public health sciences can apply for residency training (specialization) in the field of medicine related to the research grant. Fellows employed at pre-clinical departments are also awarded a residency, when applicable.

Fellows are also required to complete a three-year postgraduate doctoral study program at Medical School. The main condition for the completion of a doctoral program and advancement to the PhD degree is scientific production. A fellow has to obtain a strictly defined score, based on the completion of postgraduate courses and publications in internationally visible journals. The journals are classified in two main groups: those indexed in Index Medicus bibliographic database and those indexed in the Current Contents bibliographic database. Journals indexed in the Current Contents database receive an impact factor of the Thompson Scientific, former Institute for Scientific Information, as a measure of the journal’s scientific impact (22). Articles in these journals are highly valued for academic and research advancement in Croatia. In addition, a fellow must be the first author of an article related to his or her PhD thesis, published in a journal indexed in the Current Contents database.

Surveys

This study was based on two separate questionnaires (web extra; available at http://www.cmj.hr). One was distributed to research fellows and the other to their mentors. Questionnaires were simultaneously distributed in June 2005.

The fellows’ questionnaire consisted of 23 questions, grouped in 8 sections: general data, employment details, research involvement, publication details, professional development, teaching obligations, clinical obligations, and other remarks. All data on fellows’ self-reported publications were checked against PubMed (http://www.pubmed.com), while the information on journal citation coverage was obtained from JAKE (http://jake.med.yale.edu/index.jsp). PubMed search was also performed for fellows who did not respond the survey, in order to obtain a full data set. Employment details for research fellows who did not respond were obtained from the Human Resources Department of the School.

Questions with open-ended answers were independently coded and compared by two authors (OP and IK). Differences in the coding schemes were further evaluated and unified. Four answers were omitted from the study due to persistent discrepancies.

The mentors’ questionnaires consisted of 9 questions assessing the fellow’s postgraduate education, thesis preparation process, research involvement, teaching skills, and performance in the fellow’s professional commitments. Open-ended answers were processed in a similar way as those in the fellows’ surveys. Overall mentor’s assessment was coded as positive, neutral, or negative. An assessment was considered positive if the mentor indicated that the fellow had achieved progress in any area of the work. If the mentor simply listed the fellow’s activities without any evaluation, it was regarded as a neutral assessment. Mentor’s scientific output details were taken from the School’s database on employees’ publication data (23).

Statistical analysis

Due to previously reported differences in scientific production among pre-clinical, clinical, and public health departments (24), these three department groups were analyzed separately. Research fellows employed at the Pathology Department were categorized as belonging to a pre-clinical department, as well as fellows working on a research grant related to the Croatian Medical Journal Editorial. Fellows employed at Andrija Štampar School of Public Health were considered to be employed in the public health departments.

Non-normal distribution of data warranted the use of non-parametric statistical tests (Mann-Whitney and Kruskal-Wallis test). χ² test was used to analyze categorical data. Fisher exact test was used if a contingency table contained less than 5 cases in a cell. Due to variable fellowship duration, annual scientific production was calculated as the number of published papers indexed in Current Contents bibliographic database divided by the number of employment years. Multivariate analysis was performed with binary logistic regression. Criterion variable for logistic regression was publication success, defined as positive in fellows who authored or co-authored at least 0.5 articles per employment year in journals indexed in the Current Contents bibliographic database. Fellows who published less were considered unsuccessful.

Mentors were ranked into three ordinal groups according to their scientific production in the last three years. Quartile values were calculated and the following three groups were formed: below average (1st quartile), average (2nd and 3rd quartile), and above average (4th quartile). All analyses were performed using the Statistical Package for the Social Sciences, version 12.0.0 (SPSS Inc., Chicago, IL, USA), with the significance level set at P<0.05.

Results

In June 2005, Zagreb University School of Medicine employed a total of 123 research fellows. Out of this number, 117 research fellows took part in this study (response rate 95%). The remaining 6 were absent, due to maternal or sick leave, or professional training abroad. A total of 83 mentors responded to the survey (some mentors supervised more than one research fellow).

Median duration of research fellows’ employment in the whole sample was 3.0 years (interquartile range 3.0 years). A total of 109 fellows (89%) graduated from the Medical School, while remaining 14 (11%) had other degrees (molecular biology or psychology). Over a half of fellows with a medical degree (n = 58; 53%) were residents, or have just completed the residency (specialization). Two thirds of all fellows were women (n = 82; 67%), with no gender differences among pre-clinical, clinical, and public health departments (χ²₂ = 3.19, P = 0.203) (Table 1).

Table 1.

Scientific production, and meetings and training courses attended by research fellows at Zagreb University School of Medicine, surveyed in 2005

	Fellows
Characteristics of research fellows	pre-clinical	clinical	public health	total	P
No. of fellows	43	61	19	123
Women (No., %)	32 (74)	36 (59)	14 (74)	82 (67)	0.203*
Total number of authorships in journals indexed in the Current Contents database	64	57	54	175
Fellows who published at least 0.5 articles in journals indexed in the Current Contents database, per employment year (No., %)	11 (26)	16 (26)	9 (47)	36 (29)	0.169*
Average number of articles per fellow (median and interquartile range)	0.0 (2.0)	1.0 (2.0)	3.0 (4.0)	1.0 (3.0)	0.003†
Average annual production per fellow (median and interquartile range)	0.0 (0.5)	0.0 (0.5)	0.25 (1.0)	0.0 (0.5)	0.123†
Fellows who published articles in journals indexed in the Current Contents database as undergraduate students (No., %)	4 (9.3)	9 (15)	4 (21)	17 (14)	0.028‡
Number of fellows who did not publish a single article in journals indexed in the Current Contents database in more than 3 employment years (No., %)	6 (14)	7 (12)	1 (5.3)	14 (11)	0.052‡
Average number of international meetings attended (median and interquartile range)§	2.5 (6.5)	3.0 (8.0)	3.0 (5.0)	3.0 (7.0)	0.324†
Average number of international training courses attended (median and interquartile range)§	1.0 (1.0)	0.0 (1.0)	1.0 (2.5)	0.8 (1.0)	0.003†

*χ² square test.

†Kruskal-Wallis test.

‡Fisher exact test.

§Calculated only for the survey respondents (n = 117).

A total of 175 authorships and co-authorships in journals indexed in the Current Contents bibliographic database were recorded in the entire sample, with no difference between the genders (Mann-Whitney Z = -0.43, P = 0.668). There were significant differences in the average number of articles per fellow among the 3 department groups (Kruskal-Wallis P = 0.003) (Table 1). Between-group comparison revealed that fellows in public health published significantly more than those in pre-clinical (Mann-Whitney Z = -2.84, P = 0.005) and clinical departments (Z = -3.31, P = 0.001), with no difference between clinical and pre-clinical departments (Z = -0.45, P = 0.651). There were no significant differences in the average annual production among the 3 department groups (Kruskal-Wallis P = 0.123).

A total of 36 (29%) fellows published at least 0.5 articles in journals indexed in the Current Contents database per employment year, without differences among the 3 investigated department groups (χ²₂ = 3.56, P = 0.169).

We recorded significant differences in the number of fellows who published scientific articles as undergraduate students, with the highest number of fellows who published in public health departments (Fisher exact test, P = 0.028). We detected a borderline insignificant difference among department groups in the number of fellows who were employed longer than three years without publishing a single article in journals indexed in the Current Contents database (Fisher exact test, P = 0.052) (Table 1).

During employment, research fellows attended a median of 3.0 international meetings (interquartile range 7.0), with no difference among the 3 department groups (Kruskal-Wallis test, P = 0.324). Significant differences were recorded in the average number of international training courses (Kruskal-Wallis test, P = 0.003). Between-group comparison revealed that fellows in public health participated at more training courses than those in pre-clinical (Mann-Whitney test, Z = -2.50, P = 0.013) or in clinical departments (Z = -3.33, P = 0.001), with no indication of differences between clinical and pre-clinical departments (Z = -1.32, P = 0.186).

Fellows’ publication success was significantly associated with mentor’s scientific production group (Fisher exact test, P = 0.001) (Table 2).

Table 2.

Research fellow’s publication success and mentor’s scientific production group among the research fellows from the Medical School, University of Zagreb*

	No. of publications of fellows by mentor’s scientific production group
No. of articles per employment year in journals indexed in the Current Contents database	below average (1st quartile)	average (2nd and 3rd quartile)	above average (4th quartile)	total
Publication success (>0.5) No. (%)	2 (11)	19 (27)	15 (46)	36 (29)
Unsuccessful fellows (<0.5); No. (%)	17 (90)	52 (73)	18 (55)	87 (71)
Total	19	71	33	123 (100)

*P = 0.001, Fisher exact test.

Binary logistic regression model with fellow’s publication success as a criterion variable provided a good data fit (Hosmer-Lemeshow test, P = 0.442), with Nagelkerke R² = 0.22. Three variables were significantly associated with fellows’ publication success: undergraduate publication of an article in journals indexed in the Current Contents database (OR, 4.05; 95%CI, 1.07-15.34), positive mentor’s assessment (OR, 3.15; 95%CI, 1.10-9.05), and mentor’s scientific production group (OR, 3.14; 95%CI, 1.31-7.53) (Table 3).

Table 3.

Publication success of research fellows defined as publishing at least 0.5 articles in journals indexed in the Current Contents per one employment year

Characteristic	Odds ratio (95% confidence intervals)
Gender:
male	1.00
female	0.80 (0.30-2.12)
Undergraduate publication of article in journals indexed in the Current Contents database:
did not publish	1.00
published	4.05 (1.07-15.34)
Longer employment duration	0.93 (0.75-1.14)
Degree from the School of Medicine:
no	1.00
yes	2.31 (0.58-9.27)
Research area:
pre-clinical	1.00
clinical	1.87 (0.59-6.00)
public health	1.93 (0.51-7.30)
Mentor’s gender:
male	1.00
female	0.66 (0.20-1.96)
Higher mentor’s scientific production	3.14 (1.31-7.53)
Mentor’s assessment:
neutral	1.00
positive	3.15 (1.10-9.05)

Discussion

We identified mentor’s scientific production, positive mentor’s assessment, and undergraduate publication in journals indexed in the Current Contents database as factors associated with successful scientific production of a research fellow. The results confirm previous reports that undergraduate publications and mentor’s supportive role are the main factors associated with a successful scientific output (7-10,12).

Zagreb University School of Medicine management acknowledged several difficulties related to its scientific production: (i) problems related to structural organization of the School of Medicine (administratively and geographically isolated research units), (ii) high workload of researchers with teaching and routine professional obligations, (iii) insufficient and obsolete equipment, and finally (iv) insufficient funding (25). Lack of any scientific output during 1995-1999 was recorded for 15% professors and 45% assistants from the School of Medicine (24).

According to our study, research fellows in public health had the highest scientific production. Lower scientific output among fellows in clinical departments may be explained by their higher workload, while financial requirements and more complex research area might explain weak pre-clinical scientific production. In contrast, research fellows in public health and clinical sciences have a relatively easy access to large quantity of data routinely produced by the health care system. Less intensive workload and higher attendance at training courses could explain the highest scientific output by public health fellows.

Participation in research projects among undergraduate students is common in Croatia (26-28) and is highly valued in career advancement. The results of this study confirm a previous report that an undergraduate publication of scientific article may serve as a significant success predictor (7). Although there is an association between the grade point average and undergraduate involvement in scientific research (26), we did not include grade point average in our model because it presented the primary selection criterion for research fellows.

There are several limitations to this study. First, the study included a diverse group of research fellows that consisted of three subgroups (pre-clinical, clinical, and public health departments). Each of these subgroups has specific publication-related problems and trajectories of the scientific production. Some specialties, like family medicine, are less likely to be involved in research activities (29). Additionally, substantially smaller scientific output than expected was recorded for some clinical areas in Croatia, eg, cardiology (30). Another potential bias might have occurred in mentor’s assessment, as mentor’s decision could have been affected by the research fellow’s scientific production. Finally, the main measure in this study was authorship or co-authorship of research fellows, which is the main element of the career advancement criteria. The consequence of this is that the number of authorships presented here overestimates the actual number of published articles, as two or more fellows could have co-authored a single paper.

Evidence-based evaluation and careful planning are in essence of any functional human resources management system. This is even more pronounced in medical science, where an individual’s career takes a very long time to develop. Research fellows are valuable human resources for the development of the Medical School’s future staff. Results presented here suggest that undergraduate publication of an article in journals indexed in major international databases should be one of the main criteria when choosing research fellows. Mentor’s role in young researcher’s career advancement is crucial. Attachment to a mentor who has low scientific production is likely to result in an unsuccessful fellow’s scientific output and a delay in advancement to the PhD degree. Research fellows in biomedicine would benefit most from working with mentors who are more scientifically productive.

APPENDIX

Fellow's survey

1. Name and surname

2. Mentor's name and surname

3. Ministry of Science, Education, and Sports project identification number

4. Which Medical School Department are you employed at?

5. Employment start date

6. Postgraduate program enrolment date

7. Are you currently a postgraduate student? If yes, please write in which semester.

8. Have you submitted a PhD or MSc thesis proposal to the Medical School Thesis Board consideration?

9. Have you already published a PhD or MSc thesis? If yes, please provide the publication date and thesis name.

10. Please describe your activities within the Ministry of Science, Education, and Sports research project

11. Number of published papers indexed in the Current Contents bibliographic database

12. Number of published papers indexed in the other bibliographic databases

13. Number of other published papers

14. Number of published abstracts at international congresses

15. Number of published abstracts at national congresses

16. Number of presented lectures at various congresses

17. Have you been involved in preparation of the teaching materials? Please provide details.

18. Please provide the details of your scientific training (duration, affiliation, etc.)

19. Are you involved in the undergraduate teaching at the Medical School?

20. In which forms of the teaching do you participate in?

21. Number of annual teaching hours?

22. Residency (specialty) in? Please provide start date, status information.

23. Name your other professional activities?

Mentor's survey

1. Name and surname

2. Please comment the fellow’s postgraduate education progress

3. Please comment the fellow’s thesis preparation process

4. Please describe the fellow’s activities related to the research project

5. Please comment the fellow’s scientific development

6. Is the fellow involved in undergraduate teaching?

7. In which forms of education is the fellow participating in?

8. Please comment the fellow’s teaching skills

9. Please comment the fellow's professional development