Abstract
Background
The American Thoracic Society (ATS) Methods in Epidemiologic, Clinical, and Operations Research (MECOR) Program aims to build research capacity in low and middle-income countries. MECOR has three levels, during which students learn to develop a research protocol and write a manuscript. MECOR Turkiye has been offered every year since 2008.
Objective
The aim of this paper is to report the number and impact of published articles generated from research questions developed by students in levels 1, 2, and 3 of the ATS MECOR Program in Turkiye between 2008 and 2018.
Methods
We collected the research questions developed in all levels of the ATS MECOR Program in Turkiye between 2008 and 2018. We searched Google Scholar, PubMed, Web of Science, and ResearchGate in April 2022 to see how many of these research questions were published as articles and, if published, in which journals.
Results
Of the 176 research questions collected, 49 had been developed in level 1, 82 had been developed in level 2, and 45 had been developed in level 3. Of those 176 research questions, 55 (31.3%) generated articles that were accepted for publication. The frequency of published articles based on MECOR-developed research questions increased linearly as the course level in which they were developed increased (18.4% in level 1, 30% in level 2, 46.7% in level 3; P = 0.012; linear-by-linear association, P = 0.003). The median time from the development of the research question to publication was three years overall and did not differ significantly among the course levels (P = 0.36). Of the research questions developed, 43 were published in Science Citation Index or Science Citation Index Expanded–indexed journals.
Conclusion
Acceptance of an article for publication is one way to measure the impact of the ATS MECOR Program. Our data describe significant research output among our participants, which increases with their length of participation in the program.
Keywords: research outcomes, MECOR Program, ATS, Turkish Thoracic Society
The Methods in Epidemiologic, Clinical, and Operations Research (MECOR) Program, developed by the American Thoracic Society (ATS), aims to build research capacity in low- and middle-income countries and has been implemented in seven different regions of the world since 1994 (1, 2). The program consists of an interactive course in research methods designed to train medical professionals who primarily work with pulmonary diseases, critical care, and sleep (1–4).
The MECOR Program in Turkiye is funded jointly by the ATS and the Turkish Thoracic Society. In partnership with the Turkish Thoracic Society, the ATS has offered the course annually in Turkiye since 2008, specifically targeting academic pulmonologists in the country (5).
The ATS MECOR Program comprises three levels, and each student progresses through the levels in order (6, 7). The instruction has two components: classroom lectures and practical lessons focused on identifying an important research question and developing it into a research protocol either in small groups (level 1) or individually (levels 2 and 3). In each level, students are closely mentored by faculty, who provide daily progress reports and feedback. Level 1 focuses on the development of a research question meeting criteria such as being feasible, interesting, novel, ethical, and relevant (8). In level 2, each student works on developing their individual project with a mentor; each faculty member typically mentors three or four students. When a student completes level 2, they are expected to conduct the research and collect the relevant data, thus arriving at level 3, which focuses on advanced statistical analysis and the elements of a manuscript. Each student is expected to bring their data and a draft of their manuscript to class, where they work with a faculty mentor. In level 3, each faculty member mentors one or two students in how to improve their statistical analysis and their manuscript (1).
Personal communication with the graduated students has shown that many of the students who complete all three levels of the course achieve academic success, receiving advanced degrees, such as becoming associate professors or professors, organizing research training courses at their institutions, being placed in administrative positions such as hospital chiefs or department chairs, participating actively in international scientific communities like ATS assembly committees, or any combination of those. However, to date, there has been no objective evaluation of the outcomes of the program in Turkiye.
The aim of this study was to evaluate the success of the ATS MECOR Program in Turkiye based on the number and impact of articles generated from the research questions developed in the course between 2008 and 2018.
Methods
Study Design and Population
This was an observational retrospective cohort study performed in 2022 in Turkiye. The study included all of the research questions developed during the ATS MECOR Program between 2008 and 2018. This research was approved by Manisa Celal Bayar University Institutional Review Board (17.8.2016/20.478.486.318).
Data Collection and Variables
The outcome variable was defined as publication of the research questions developed during the course. The MECOR Program has an impact on various aspects of development of the students as researchers and teachers. However, considering that the program is performed in different countries with different sociocultural backgrounds, an outcome that can be used universally in all the courses needed to be defined. This outcome was set as completing the course with a published research question by the MECOR steering committee for evaluation purposes.
During the time period of the study, MECOR students worked on developing a research protocol in different levels. In level 1, a group of students worked on forming one research question and developed the study protocol to perform the research. In level 2, each student developed their own research question and the study protocol. Finally, in level 3, the student worked on statistical analysis and writing a manuscript. Each question was presented at the end of the course by the student or the group (3). The list of research questions presented is recorded for the course files. Mostly, the students worked on different research questions at each level; if a student used one of the research questions developed in level 1 or 2 for level 3 work, then it was recorded only once in the study as a level 3 research question.
We collected all of the research questions developed by the students in all levels of the ATS MECOR Program during the 2008–2018 period. We chose to enroll only the questions that were developed until 2018, because in 2018 the structure of the MECOR program was modified: at level 1, the students did not work in groups but worked on their own to develop a research question, and the flipped classroom approach began to be used in teaching. Moreover, the students began to be expected to complete all three levels of the course working on the research question they developed in level 1.
In April 2022, we searched Google Scholar, PubMed, Web of Science, and ResearchGate for publications related to those research questions, using the relevant keywords and student names. The course level in which the research question was developed, the topic of the question, the year of publication, and the indexing of the publishing journal were also recorded.
Statistical Analysis
We determined the number of research questions developed in each level of the ATS MECOR Program in each year and evaluated and summarized the frequency of published articles based on those questions, calculated as percentages by course level and year. The number of years from the course date to the publication date is expressed as the median (interquartile range) and compared among the levels with the Wilcoxon signed-rank test. To compare the proportion of papers published in Science Citation Index (SCI)-indexed journals among the levels, we used the chi-square test. Values of P < 0.05 were considered statistically significant.
Results
Publications Generated at Each Course Level
During the 10-year period studied, 176 research questions were developed via the ATS MECOR Program: 49 (27.8%) in level 1, 82 (46.6%) in level 2, and 45 (25.6%) in level 3. Of those 176 research questions, 55 (31.3%) generated an article that was accepted for publication (Table 1). The frequency of published articles based on a MECOR-developed research question increased linearly as the course level in which it was developed increased (8.4% in level 1, 30% in level 2, 46.7% in level 3; P = 0.012; linear-by-linear association, P = 0.003) (Figure 1).
Table 1.
Research questions developed during the American Thoracic Society Methods in Epidemiologic, Clinical, and Operations Research Program in Turkiye, and the articles generated from those questions that were subsequently accepted for publication
| Year/Period | Questions Developed [n (%*)] | Articles Published [n (%†)] |
|---|---|---|
| 2008 | 6 (3.4) | 1 (16.7) |
| 2009 | 17 (9.7) | 5 (29.4) |
| 2011 | 31 (17.6) | 6 (19.4) |
| 2012 | 25 (14.2) | 11 (44.0) |
| 2013 | 18 (10.2) | 7 (38.9) |
| 2014 | 15 (8.5) | 8 (53.3) |
| 2015 | 20 (11.4) | 6 (30.0) |
| 2016 | 23 (13.1) | 7 (30.4) |
| 2018 | 21 (11.9) | 4 (19.0) |
| 2008–2018 | 176 (100.0) | 55 (31.3) |
Proportion of the total number of research questions developed over the 10-year period.
Proportion of the total number of research questions developed in the year or period indicated.
Figure 1.
Proportion of research questions developed during the American Thoracic Society Methods in Epidemiologic, Clinical, and Operations Research Program in Turkiye that generated papers subsequently accepted for publication, by course level. *Chi-square test.
The median time from the development of the research question to publication was 3 years overall and did not differ significantly among the course levels (P = 0.36) (Table 2).
Table 2.
Characteristics of the articles generated from research questions developed during the American Thoracic Society Methods in Epidemiologic, Clinical, and Operations Research Program in Turkiye and subsequently accepted for publication, by the course level in which the research question was developed
| Characteristic | Level 1 (n = 9) | Level 2 (n = 25) | Level 3 (n = 21) | P Value |
|---|---|---|---|---|
| Years from research question development to publication, median (IQR) | 3 (3–5) | 3 (2–4) | 2 (2–4) | 0.36* |
| Publication in an SCI- or SCIE-indexed journal, n (%) | 7 (77.8) | 17 (68.0) | 19 (90.5) | 0.08† |
Definition of abbreviations: IQR = interquartile range; SCI = Science Citation Index; SCIE = Science Citation Index Expanded.
Wilcoxon signed-rank test.
Chi-square test.
Characteristics of Publications Developed
Of the 55 articles that were accepted for publication, 43 were published in SCI- or SCI Expanded–indexed journals; 6 were published in PubMed-indexed journals; 3 were published in Emerging Sources Citation Index–indexed journals; 2 were published in Turkish national journals; and 1 was published in a Google Scholar–indexed journal. Most of the journals were pulmonology-specific journals.
The most common research question topics were as follows: intensive care unit protocols (n = 36), asthma (n = 25), chronic obstructive lung disease (n = 26), pediatric asthma (n = 15), sleep-disordered breathing (n = 15), and smoking cessation (n = 15). When the research questions that generated articles subsequently accepted for publication were grouped by topic, the most common topics were intensive care unit protocols (n = 11), chronic obstructive lung disease (n = 6), asthma (n = 6), pediatric asthma (n = 5), and infectious diseases (n = 5).
Discussion
The results of this study have shown that one-third of the research questions developed during the program courses completed between 2008 and 2018 generated articles that were subsequently accepted for publication. We also found that the research questions developed in level 3, the final competency level of the program, generated a higher number of such articles than did the research questions developed in levels 1 and 2. In Turkiye, the ATS MECOR Program primarily trains physicians working in the field of pulmonology and critical care who have an academic background and wish to expand their knowledge of research methodology and manuscript preparation. Since 1994, the ATS MECOR Program has contributed to improving global lung health by helping build capacity for high-quality research in low- and middle-income countries (1–5). One of the main outcome measures is the publication of articles based on the research questions and protocols developed during the course. We found that approximately one-third of the research questions developed during the course generated published articles. That relatively low proportion could be explained by country-specific aspects, including high patient loads, the unavailability of mentorship, and workplace transfers for juniors because of compulsory government work. Only 14% of the research questions developed in level 1 generated articles that were subsequently published. There may be various reasons underlying this low rate, which requires further qualitative research on the issue. The possible reasons that require further evaluation are that level 1 is the initial level during which the students are expected to learn how to develop a good research question, and minimal information about methodology and statistics is provided. Another possible explanation is group work, which makes it difficult for students in different working institutions to come together for research once the course is over.
The fact that the research questions developed in level 3 generated a higher number of articles subsequently accepted for publication than did those developed in the other levels was an expected outcome, given that level 3 ATS MECOR Program students have assimilated the basic background information and work on data previously collected or on a draft manuscript. Therefore, these students are the ones who are closest to publishing an article at the end of the course.
The median time from the development of the research question to publication was approximately 3 years, and this was not significantly different for different levels. For level 1, this duration is expected; it takes 2 years to complete all three levels, and until level 3 the students do not get education on writing a manuscript. For level 3, this duration is longer than expected, because this level is usually completed with a draft of a manuscript. Possible explanations for this delay may be that the students are at the early stages of their academic career, their patient care workload at the hospital is high, and finding time to edit, submit, and revise a manuscript may be challenging.
Strengths and Limitations
The major strength of our study is that it is an objective evaluation of the 10-year history of the MECOR Program in Turkiye, which is ongoing. However, our study has some limitations. Although there are a number of outcome variables that could demonstrate the success of the program, we chose only one: the number of published articles produced. Verbal communication with the students indicates that the program had a qualitative influence on the individual students, many of whom went on to become faculty members at their institutions, mentor many students about research methodology, and participate in international scientific research activities. In addition, the faculty members who taught in the ATS MECOR Program in Turkiye reported improving their teaching skills by mentoring. Neither of those variables was measured in the present study. Further qualitative research on the evaluation of the results of the MECOR Program is planned.
Conclusions
The results of our study may guide future strategic planning of the ATS MECOR Program in Turkiye. Given that the main aim of the program is to increase research capacity, the number of published articles produced is a valuable but limited outcome measure. We have demonstrated that the research questions developed in level 3, the final competency level of the program, generated a higher number of such articles than did the research questions developed in other levels, suggesting that the program contributes to increase research capacity. There is a need for further studies to evaluate the qualitative effects that the program has on the research capacity of students, such as the impact of publications, the number of citations, and the mentoring capacity of faculty.
Footnotes
Supported by the American Thoracic Society–Turkish Thoracic Society Methods in Epidemiologic, Clinical, and Operations Research Program.
Author Contributions: Concept and design: Z.K., O.Y., H.A., P.A., C.D., C.G.G., M.A., and S.B. Data acquisition: Z.K., O.Y., and H.A. Data analysis and interpretation: Z.K. and O.Y. Drafting of the manuscript: Z.K., O.Y., H.A., P.A., C.D., C.G.G., M.A., and S.B. Critical review of the manuscript: Z.K., O.Y., H.A., P.A., C.D., C.G.G., M.A., and S.B. All authors reviewed and approved the final version of the manuscript.
Author disclosures are available with the text of this article at www.atsjournals.org.
References
- 1.American Thoracic Society. https://www.thoracic.org/about/global-public-health/mecor-program/
- 2. Schraufnagel D, Hansen-Flaschen J. Editors’ introduction to ATS reports. Ann Am Thorac Soc . 2013;10:361. doi: 10.1513/AnnalsATS.201306-181ED. [DOI] [PubMed] [Google Scholar]
- 3. Parry V. The American Thoracic Society Methods in Epidemiologic, Clinical, and Operations Research program: research capacity building in action. Ann Am Thorac Soc . 2013;10:383–385. doi: 10.1513/AnnalsATS.201304-082PS. [DOI] [PubMed] [Google Scholar]
- 4. Buist SA, Parry V. The American Thoracic Society Methods in Epidemiologic, Clinical, and Operations Research program: a research capacity–building program in low- and middle-income countries. Ann Am Thorac Soc . 2013;10:281–289. doi: 10.1513/AnnalsATS.201304-081OT. [DOI] [PubMed] [Google Scholar]
- 5.https://toraks.org.tr/site/community/workingGroups/p/VEdfTUVDT1I
- 6. Fessler HE, Le T, Roberts DH. Editors’ introduction to ATS reports: medical education. Ann Am Thorac Soc . 2014;11:222–223. doi: 10.1513/AnnalsATS.201309-331ED. [DOI] [PubMed] [Google Scholar]
- 7. Roberts DH, Schwartzstein RM, Weinberger SE. Career development for the clinician-educator: optimizing impact and maximizing success. Ann Am Thorac Soc . 2014;11:254–259. doi: 10.1513/AnnalsATS.201309-322OT. [DOI] [PubMed] [Google Scholar]
- 8.Cummings SR, Browner WS, Hulley SB. In: Designing clinical research. 4th ed. Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB, editors. Philadelphia, PA: Lippincott Williams and Wilkins; 2013. Conceiving the research question and developing the study plan; pp. 14–22. [Google Scholar]