Comparison of Virtual and In-Person ATS Methods in Epidemiologic, Clinical, and Operations Research Programs

Abstract

Background

The Methods in Epidemiologic, Clinical, and Operations Research (MECOR) program of the American Thoracic Society aims to strengthen research capacity in low- and middle-income countries.

Objective

To compare the impact of online versus in-person MECOR level 1 and level 2 courses in Türkiye on program and research project completion rates and to understand student preferences for alternative program delivery.

Methods

We enrolled all students who participated in MECOR Türkiye level 1 courses from 2020 to 2023. Through an online questionnaire, we collected information on progress of the research project developed at MECOR levels 1 and 2 and the students’ progression through the end of the program. Research project progression (from design to data analysis) and three-level MECOR course completion was compared between the delivery methods, and student preference of online versus in-person courses was evaluated.

Results

Sixty-five students attended MECOR course level 1 from 2020 to 2023, among whom 61 responded to the questionnaire. For the level 1 course, there was no statistical difference in completion rates of research projects in online versus in-person courses (P = 0.22). The preference for online teaching was higher (40%) among those who attended the online offering than among those who attended the in-person course (9.8%; P = 0.005). Forty students attended a MECOR level 2 course; numerically more students (87.5%) who attended an in-person course were at a data collection/analysis stage of their research project compared with the online students (56.3%; P = 0.10). At level 2, 12 students (37.5%) who attended online and 4 (50%) who attended in person also completed level 3 (P = 0.56). There was no significant difference between the groups who took level 2 in person versus online in preference of course format (78.1% and 87.5% preferred in-person level 2, respectively; P = 0.86).

Conclusion

Completion rates of research projects planned during the course and the overall program completion were not significantly affected by the virtual or in-person nature of the level 1 or level 2 MECOR courses, and the students’ preferences for virtual courses vary depending on their own experience.

Health equity may be defined as equal opportunities for and rights to health care in people with various environmental and social backgrounds (1). Achieving healthcare equity globally is one of the major goals of the World Health Organization (WHO), expressed in the constitutional objective as “the attainment by all peoples of the highest possible level of health” (2). This goal can only be achieved through policies and practices based on best scientific evidence. Therefore, the WHO strategy on research for health emphasizes helping to strengthen national health research as one of the interrelated goals (2). The Methods in Epidemiologic, Clinical, and Operations Research (MECOR) program of the American Thoracic Society addresses this point of strengthening research capacity in low- and middle-income countries (3–6). The program began in 1994 in Latin American and added Africa, Türkiye, India, China, and Southeast Asia in subsequent years, now having more than 2,500 graduates (6, 7). The program is designed with three levels: level 1 is the identification of a research question and protocol development, level 2 is preparation of a manual of procedures, and level 3 is manuscript preparation.

The MECOR program in Türkiye is a partnership of the American Thoracic Society and Turkish Thoracic Society. The program, traditionally an in-person course, was forced to be presented as an online course during the coronavirus disease (COVID-19) pandemic. The positive evaluation of the first online level 1 course led to the presentation of the course as an online option for the level 2 students the following year.

In this study, we compared the impact of online versus in-person MECOR Türkiye level 1 and level 2 courses on the completion rate of research projects planned during the course and the overall program completion. In addition, we evaluated students’ preferences for face-to-face or online MECOR courses.

Methods

Study Population

We enrolled all students who participated in MECOR Türkiye level 1 courses in 2020, 2022, and 2023. This research was approved by the Manisa Celal Bayar University Institutional Review Board (approval 20.478.486/2555). The participants consented to the use of the questionnaire information by marking the consent box on the questionnaire. Among the 65 level 1 students, 61 consented to be enrolled and completed the questionnaire, and all the level 2 students participated in this research.

Data Collection

Survey forms were prepared using Google Forms and shared through text messages with the participants in June 2024.

The survey had four components:

1. Demographic and professional characteristics of the participants such as age, sex, marital status, and parenthood, together with specialty and stage of career at the first year of MECOR course attendance.

2. Progress of the research protocol developed at MECOR level 1: was the protocol approved by the institutional review board? Were the data collected and did the participant continue to work on the same research protocol in MECOR level 2 if attended? Similarly, if the research question worked on during MECOR level 2 was different from level 1, was the protocol approved by the institutional review board? Were the data collected and did the participant continue to work on the same research in MECOR level 3 if attended? Finally, the students were asked if the research protocol worked on during the MECOR levels attended were presented as abstracts at scientific meetings or published as articles.

3. For the respondents who did not attend level 2 or level 3, the reasons for not attending were assessed with a multiple-choice question with an “other” option among the choices.

4. The preference for online or in-person MECOR courses based on personal experience was also sought.

Statistical Analysis

Statistical analysis was performed using JAMOVI software, version 2.3 (2022). Sociodemographic and career-related characteristics such as specialty and stage of career were categoric variables, summarized as counts and percentages and compared between the groups who attended online versus in-person level 1 and level 2 MECOR courses using the χ² test. Potential predictors of completing a level 2 course were tested with logistic regression analysis; the model consisted of online versus in-person level 1 course attendance, age group (41–50 yr regarded as the reference), marital status, specialty (pulmonology regarded as the reference), and stage of career (specialist regarded as the reference) when first attending the course. Similarly, potential predictors of completing a level 3 course were tested with logistic regression analysis. The logistic regression model included level 2 attendance type, age group, marital status, and specialty. Statistical significance was defined by a P value lower than 0.05.

Results

Sociodemographic Characteristics of the Study Population

We enrolled 61 participants among the 65 who attended MECOR level 1 courses between 2020 and 2023 and 40 students who attended MECOR level 2 courses in 2022 and 2023. Among the 61 level 1 participants, 20 participated in an online course, and, among the 40 level 2 participants, 32 participated in an online course; the remaining students had attended the in-person courses.

Among the participants, approximately 60% in both levels were married and approximately 40% had children. Age, sex, marital status, parenthood, or stage of career were not significantly different among the participants who attended an online or in-person course for either level. Most (51.7%) were adult pulmonologists; 25% were pediatric pulmonologists or allergists. Online participants were less frequently pulmonologists (45%) and morefrequently pediatric pulmonologists or allergists (40%) than their in-person counterparts (P = 0.046) (Table 1). The majority of online (78.1%) and in-person (62.5%) level 2 participants were aged 31–40 years (P = 0.48), and most were residents (online, 50%; in person, 62.5%; P = 0.82) (Table 2).

Table 1.

Sociodemographic characteristics of the study population according to level 1 attendance

Characteristic	Level 1		Total (N = 61)	P Value*
	Online (n = 20)	In-Person (n = 41)
Age group				0.83
20–30 yr	3 (15%)	6 (14.6%)	9 (14.8%)
31–40 yr	14 (70%)	31 (75.6%)	45 (73.8%)
41–50 yr	3 (15%)	4 (9.8%)	7 (11.5%)
Male sex	5 (25%)	8 (19.5%)	13 (21.3%)	0.62
Married	14 (70%)	24 (58.5%)	37 (61.7%)	0.39
Parent	10 (50%)	14 (34.1%)	24 (39.3%)	0.23
Specialty				0.046
Chest diseases	9 (45%)	23 (56.1%)	32 (52.5%)
Thoracic surgery	—	2 (4.9%)	2 (3.3%)
Pediatric pulmonology or pediatric allergy	8 (40%)	7 (17.1%)	15 (24.6%)
Allergy and immunology	3 (15%)	1 (2.4%)	4 (6.6%)
Intensive care	—	5 (12.2%)	5 (8.2%)
Other	—	3 (7.3%)	3 (4.9%)
Stage of career				0.90
Resident	6 (30%)	14 (34.1%)	19 (32.8%)
Specialist	11 (55%)	20 (48.8%)	31 (50.8%)
Attending	3 (15%)	7 (17.1%)	10 (16.4%)

^*P values determined by χ² test.

Table 2.

Sociodemographic characteristics of the study population according to level 2 attendance

	Level 2		Total (n = 40)	P Value*
	Online (n = 31)	In-Person (n = 9)
Age group				0.48
20–30 yr	3 (9.4%)	2 (25%)	5 (12.5%)
31–40 yr	25 (78.1%)	5 (62.5%)	30 (75.0%)
41–50 yr	4 (12.5%)	1 (12.5%)	5 (12.5%)
Male sex	6 (18.8%)	1 (12.5%)	7 (17.5%)	0.68
Married	19 (59.4%)	6 (75%)	25 (62.5%)	0.41
Parent	12 (37.5%)	4 (50%)	16 (41%)	0.52
Specialty				0.32
Chest diseases	17 (53.1%)	4 (50%)	21 (52.5%)
Thoracic surgery	1 (3.1%)	—	1 (2.5%)
Pediatric pulmonology or pediatric allergy	7 (21.9%)	2 (25%)	9 (22.5%)
Allergy and immunology	1 (3.1%)	2 (25%)	3 (7.5%)
Intensive care	3 (9.4%)	—	3 (7.5%)
Other	3 (9.4%)	—	3 (7.5%)
Stage of career				0.82
Resident	16 (50.0%)	5 (62.5%)	21 (52.5%)
Specialist	11 (34.4%)	2 (25%)	13 (32.5%)
Attending	5 (15.6%)	1 (12.5%)	6 (15.0%)

^*P values determined by χ² test.

Completion Rate of Research Projects Planned during the Course

Stage of research protocol developed in online and in-person level 1

There was no statistical difference in the rates of completion of research protocols developed in the online and in-person level 1 courses; 6 online students (30%) and 19 in-person students (46.3%) were not able to carry out the level 1 research protocol they developed (P = 0.22) (Figure E1 in the data supplement). Among the online level 1 course participants, four (20%) were collecting data, nine (45%) were at the stage of statistical analysis and manuscript preparation, and one (5%) had published the article already. Among the in-person level 1 course participants, 7 (17.1%) were collecting data and 15 (36.6%) were at the stage of statical analysis and manuscript preparation.

Ten online level 1 students (50%), but only 5 in-person level 1 students (12.2%), had presented their research as an abstract at a national or international conference (P = 0.001).

Among the subjects who proceeded to attend level 2, 54.5% of the online level 1 graduates and 96.4% of the in-person level 1 graduates continued to work on the research protocol they developed in level 1 during the level 2 course (odds ratio, 23.3; 95% confidence interval, 2.3–238; P < 0.001). The most common reason stated for changing the research question in level 2 was detecting that it was not feasible.

Stage of research protocol developed in online and in-person level 2

A total of 56.3% of the subjects who attended online level 2 courses were collecting data or had proceeded to analysis of the data they derived from the research protocol they worked on during level 2; 87.5% of the subjects who attended the in-person level 2 course were at the data collection or analysis stage of their project (P = 0.10) (Figure 1).

Figure 1.

Comparison of the percentage of in-person and online Methods in Epidemiologic, Clinical, and Operations Research level 2 students who proceeded to carry out the research question worked on during this level.

Among the students who took online and in-person level 2 courses, respectively, 9.4% and 62.5% had presented their research as an abstract at a national or international conference (P < 0.001).

Among the 16 subjects who proceeded to attend level 3, there was no significant difference in the percentage of students who worked on the research questions developed in previous MECOR levels (58.3% of online level 2 vs. 100% of in-person level 2 graduates; P = 0.12). The reasons stated for changing the research question in level 2 were completing data collection for other research before level 3, detecting that it was not feasible, and publishing the previous research question worked on.

Overall Program Completion

Attendance of levels 2 and 3 MECOR courses in online versus in-person level 1 participants

There was no difference in the rate of attending level 2 after completing level 1 online (55%; n = 11) or in person (70.7%; n = 29; P = 0.23) (Figure E2). The most common reason for not attending level 2 was a lack of time (77.8% and 36.4% of online and in-person level 1 attendees, respectively). Other reasons reported were cost, preference for in-person level 2, and institutional permission problems.

Logistic regression analysis of the potential predictors of completing a level 2 course did not reveal a significant variable. The logistic regression model consisted of level 1 course online versus in-person attendance, age group (41–40 yr regarded as the reference), marital status, specialty (pulmonology regarded as the reference), and stage of career (specialist regarded as the reference) when first attending the course (R² = 0.103; overall model test, χ² = 6.65; P = 0.83).

Among the students who attended level 1 online, 5 (45.5%) completed a level 3 course, compared with 11 (37.9%) of those who attended level 1 in person (P = 0.66).

Attendance of level 3 MECOR course among online and in-person level 2 participants

Twelve students (37.5%) who attended level 2 online and four (50%) who attended level 2 in person completed a level 3 course (P = 0.56) (Figure 2). The most common reason for not attending level 3 was not being able to collect the data (40% and 25% of those who attended level 1 online and in person, respectively). Other reasons reported were lack of time, cost, family, and institutional permission problems.

Figure 2.

Attendance percentage of Methods in Epidemiologic, Clinical, and Operations Research level 3 based on the level 2 course attended.

The logistic regression analysis model for the potential predictors of completing a level 3 course included level 2 attendance type, age group, marital status, and specialty. The overall model was not significant (χ² = 6.28, P = 0.85), with a Cox and Snell R² value of 0.41. None of the variables were significant predictors for completion of a level 3 course.

Students’ Preferences

Among the 20 students who attended the level 1 course online, 12 (40%) stated that they preferred online courses. Among the 41 students who attended level 1 in person, only 4 (9.8%) preferred the online level 1 course (P = 0.005).

However, when the percentage of in-person versus online level 2 preferences were compared, there was no significant difference between the groups who took level 2 in person and online in terms of preference for course format (78.1% and 87.5% preferred in-person level 2, respectively; P = 0.86).

Discussion

The results of this research have shown that the completion rate of research projects planned during a MECOR course and the overall program completion were not significantly affected by the virtual or in-person nature of the level 1 or level 2 course. However, presenting their data in conferences was more common in students who took the courses in person. Students’ preferences for a virtual versus in-person level 1 course correlated with their own experience; the ones who took the virtual course preferred virtual and the ones who had an in-person course preferred the in-person format. This was not the case for level 2 course format; all participants preferred the in-person format.

The main goal of the MECOR program is local research capacity building. Current evidence-based patient care depends mainly on data from developed countries with more advanced technology and different priorities in health care compared with low-and middle-income countries. This disparity leads to underrepresentation of the low- and middle-income countries as healthcare users (8). Therefore, WHO has adopted a strategy on research for health that emphasizes the strengthening of national capacities for research (2). Thus, the aims of the program are in line with this strategy. The MECOR program is composed of a research methodology training curriculum that is tailored to the needs of the local research milieu and mentoring provided by local and international faculty allowing for long-term support for the local researchers. Therefore, not only the curriculum but also the mentoring and interaction with the faculty members and peers form the basis of teaching methodology with the expectation of long-term change in attitudes and behavior in research culture in the local scientific community (9–11). Previous research about the influence of the Pan African Thoracic Society MECOR program on increasing research capacity in Africa has demonstrated that approximately three fourths of the students who attended at least one level of MECOR course between 2007 and 2016 had been listed as an author of a scientific research article (12).

The travel restrictions during the pandemic have led to the adoption of distant teaching methods for many educational institutes. The most critical part of this period was the need to reconstitute the educational content and methodology to incorporate digital tools in teaching (13). This journey of online education started with virtual didactic sessions through various online platforms, but then a flipped-classroom approach was added to increase teacher–student and student–student interactions and discussions (14). MECOR was not exempt from this, and MECOR programs in different parts of the world went online (7). Many students were actively working as physicians at their institutions, but all educational activities were suspended. Therefore, MECOR Türkiye was planned to proceed as a virtual program, but, considering the time allocated and faculty availability, only level 1, which covers the development of a research question and study protocol, was held. Learning outcomes and teacher and student characteristics and satisfaction all determine the success of these online programs (14). The results of the present study indicate that the sociodemographic and career characteristics of the participants in the online and in-person MECOR level 1 courses during the 3-year period were not significantly different.

This experience of online learning that carries the potential for flexibility in scheduling as well as learner-focused interactive learning has taught us to use the online teaching methodology for sharing knowledge, self-directed communication, and development of critical thinking (14). However, a higher fraction of the online level 1 graduates indicated that they preferred online courses compared with the in-person course graduates. Therefore, after the pandemic emergencies ended, MECOR Türkiye adopted a blended methodology of teaching the three levels of the course as reported by many other education programs (9). In this context, level 1 and level 3 were held as in-person courses, whereas level 2 was offered as an online course. This had two rationales: to decrease the cost of the overall course and the time lag between level 1 and level 2, enabling completion of all three levels over a period of 12 months. This paper compares the outcome of the research protocols worked on during the course, the completion rate of the program, and the preferences of the students who attended in-person and online level 1 and level 2 courses. One of the outcomes of the MECOR program is publishing the research protocol developed during the course as a research article. Although this is not the only outcome of the course, it provided the basis of the measurable outcome defined for this study. Previous research demonstrated that nearly 30% of the research questions developed during MECOR Türkiye courses between 2008 and 2018 was published as scientific articles. Moreover, the students who graduated from MECOR level 3 published their research questions more frequently compared with the earlier-level graduates, so it may be concluded that completing all three levels of the MECOR course contributes to the development of the student as a researcher (15).

Previous research that compared online and in-person teaching of various knowledge and skills report favorable outcomes of both methods. An online and in-person course for introductory surgical methods revealed that both methods resulted in increases in knowledge and confidence. The only component that was scored higher for the in-person course was the ease of engagement. The advantages of the online courses were reported to be the flexibility of schedule and cost effectiveness (16). On the contrary, research in a group of continuing medical education participants demonstrated that engagement was not significantly different between in-person and online courses, but the teacher effectiveness was higher for the in-person course. This same research demonstrated that other factors that influence engagement were presentation characteristics like the use of audience response and the time of the day the course is held (17). Another recent study compared the impact of in-person or online formats of a continuing professional development course on immediate postcourse behavioral intention and clinical behavior at 6 months. They demonstrated that both formats of education increased the behavioral intention in the course participants who stated that they intended to change their clinical behavior because of the course; however, intention and self-reported long-term behavior were not related (18). MECOR students who took level 1 online preferred the online level 1 course more frequently, but, for the level 2 courses, both groups stated that they preferred the in-person course. MECOR students who took level 2 online or in person stated that they preferred in-person level 2 courses.

MECOR students who developed their research questions during an online level 1 course tended to change it for the next levels for feasibility reasons, suggesting that the feedback or the time they get during in-person course helps them work on their questions more efficiently. On the contrary, this was not the case for the research questions students worked on during online or in-person level 2 courses. This may be due to the introductory nature of the level 1 MECOR course that needs more engagement on the student’s side. However, level 2 mainly consists of the development of manual of procedures that is based on an already formed research question by a student who has been introduced to the MECOR methodology of training. Similarly, the rates of carrying out the protocol worked on during online and in-person level 1 and level 2 research questions were similar, but presentation at a conference was more frequent for online level 1 and in-person level 2 graduates. This was attributed to the time that elapsed after these two courses, which is longer than the gap between in-person level 1 and online level 2 courses.

The major limitation of this program review is the expected influence of the COVID-19 pandemic on research conditions that influenced the level 1 group who attended the course online, considering the heavy workload of the physicians and the change in the number of patients presenting to hospitals during that period. The major strength of our research is the high participation rate, leading to representation of the majority of the students who participated in MECOR Türkiye during the 3-year study period.

In conclusion, there are advantages and disadvantages of online and in-person MECOR courses. Similar course completion rates and successfully carrying out the research questions developed during the online and in-person courses was noted despite variable student preferences for online and in-person courses. Future research evaluating the impact of hybrid courses with some virtual and some in-person levels needs to be performed to evaluate the impact on preferences, publications, and cost.