A Systematic Review and Meta-Analysis of Medical Students’ Perspectives on the Engagement in Research

Supplemental Digital Content is available in the text

Abstract

Engaging students in active learning lies at the center of effective higher education. In medical schools, students’ engagement in learning and research has come under increasing attention. The objective of this study was to synthesize evidence on medical students’ perspectives on the engagement in research. We performed a systematic review and meta-analysis.

Relevant studies were searched in electronic databases. The methodological quality of the included studies was assessed. Overall, 14 observational studies (with 17 data sets) were included. In general, many studies did not use the same questionnaires and the outcome measurements were not consistently reported; these presented some difficulties in pooling the results. Whenever data permitted, we performed pooled analysis for the 4 education outcomes. A Bayesian meta-analytical approach was supplemented as a measure of uncertainty.

A pooled analysis showed that 74% (95% confidence interval [CI]: 1.57%–11.07%; I²: 95.2%) of those students who engaged in research (while at the medical school) had positive attitudes toward their research experiences, whereas 49.5% (95% CI: 36.4%–62.7%; I²: 93.4%) had positive attitudes toward the study of medical sciences, 62.3% (95% CI: 46.7%–77.9%; I²: 96.3%) had self-reported changes in their practices, and 64% (95% CI: 30.8%–96.6%; I²: 98.5%) could have published their work. There was substantial heterogeneity among studies. We acknowledged the caveats and the merit of the current review.

Findings showed that engagement in research resulted in favorable reactions toward research and academic learning. Future well-designed studies using standardized research tools on how to engage students in research are recommended.

INTRODUCTION

Engaging students in active learning lies at the heart of effective education,¹ and student engagement has been the subject of increasing scholarly research. Although there is no one all-encompassing definition of student engagement, it is generally accepted as a function to the time and energy that students devote to educationally purposive activities inside and outside of the classroom, and to the policies and practices that institutions used to induce students to take part in these activities.^2,3 In essence, it refers to student's involvement in activities and conditions likely to generate high-quality learning.^4,5

Although the concept of student engagement involves the elements of learning and the institutional environment, its main focus is on students. In medical schools, individual student engagement in learning and research has come under increasing attention. The General Medical Council has recommended that learning outcomes for all medical graduates should include research skills to enable them to study in-depth areas which are of particular interest to them and to obtain insight into scientific and research methods.⁶ Medical students’ engagement in research has been reported in the literature and has been associated with the acquisition of transferrable skills in the areas of communication, teamwork, time management, and critical thinking,⁷ gaining experience and motivation for a research career,⁸ improving critical thinking abilities,⁹ and fostering positive attitudes toward science and scientific methodology.^10,11

Although individual studies addressing students’ engagement in research are available, a synthesis of results in this area has to date not been undertaken. This gap in evidence motivated us to conduct a systematic review and meta-analysis on medical students’ engagement in research. Such analysis in this area could provide insights into the issues related to students’ involvement in research, which are otherwise not readily available from individual studies. To this end, the objective was to synthesize evidence on the medical students’ perspectives on the engagement in research.

METHODS

The present study conformed to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements¹² (see Supplemental Checklist S1, http://links.lww.com/MD/A319).

Study Search

Studies were searched in electronic databases such as Google scholar, PubMed, Medline, EMBASE, ERIC, and Social Science Citation Index. In Google Scholar, we limited the search to the title only. The MeSH term “students’ engagement” was searched in PubMed in combination with each of the following terms: “medical student” OR “undergraduate” AND “research” AND “engagement” OR “involvement” OR “participation.” We also searched studies in the student BMJ database. The reference lists of the included studies and the relevant reviews were also checked. Searches were limited to publications in English language until November 2014.

Study Selection

Studies were selected for the present review if they satisfied the following criteria: study participants: studies on medical students, regardless of sex, age, country, and any setting of institutions (college, university, school, and institute) were included. Medical students were defined as students enrolled in tertiary programs who will eventually qualify as medical doctors.¹³

Study Design

Primary studies of any design (observational or randomized trials) that focused on medical students’ involvement in research activities were included.

Study Outcomes

The key outcomes of interest were based on Freeth and associate's adaptation¹⁴ of the Kirkpatrick's model of educational outcomes.^15,16 This model comprises “learners’ reaction” (to the educational experience); learning (modification of attitudes/perceptions and acquisition of knowledge and skills); behavior (self-reported changes in practice and observed changes in practice, including new leadership positions); and results (which refer to change at the level of the organization).^14–16

We addressed 4 specific outcomes: the research experience (learners’ reaction), attitudes to science in medicine (learning), self-reported changes in practice (behavior), and change at the level of the organization (publication output).

Studies were excluded if they did not meet the inclusion criteria.

Data Extraction

Two authors independently screened the titles and abstracts of publications according to the inclusion criteria. The 2 authors then individually extracted information from each of the included studies using the piloted data extraction form prepared for the present study. Information collected were author, year of publication, country, student's characteristics (mean age, proportion of male sex, year in medical school), methodological characteristics (sample size, sampling frame, study design, response rate), and the reported outcomes. Disagreements between the 2 authors were resolved by discussion. Had there been any outstanding discrepancy, these would have been resolved by consulting a third author for final decision. The methodological quality of included studies was assessed according to checklists used in a published review¹¹ for rating surveys in the field of medical education. For each study, the 2 authors independently rated the study on the basis of 11 criteria described elsewhere.¹¹ This assessment is solely based on the judgment by the authors of the present study. Disagreements were resolved by consensus.

Statistical Analysis

For each study, individual survey questionnaire items that met the inclusion criteria were identified. Some questionnaire items were positively worded, whereas others were worded negatively. In the current analysis, the negatively worded items were reversed so that a higher numbered response on the scale (Likert scale) would indicate positive attitudes. For continuous data, mean and standard deviation (SD) were extracted. For categorical data with percentage (%) (eg, strongly agreed/strongly disagreed, yes/no), we assessed proportion and its 95% confidence interval (CI). If 2 or more individual studies reported the similar outcome measures, we performed a pooled analysis. The heterogeneity between these studies was assessed with the I² test. A value of I² over 50% indicated substantial heterogeneity.¹⁷ For pooling of the results, we used a random-effect model in order to account for heterogeneity among these included studies.

Of note is that random-effects meta-analyses need to be interpreted with due consideration of the whole distribution of effects. A Bayesian meta-analytical approach was, therefore, supplemented to take into account the within- and between-study estimates of variance. This approach has an advantage of flexibility, allowing incorporation of full uncertainty in all parameters (but not uncertainty in the model) and yielding inferences that could be more readily interpreted. This Bayesian meta-analytical approach yielded posterior distributions. A posterior distribution is likely to be skewed, and the median and appropriate percentiles (ie, 95% credible interval in this case) are used for points and interval estimations¹⁸ as measure of uncertainty.

For robustness of analysis, we performed leave-one-out sensitivity analysis by removing 1 study at a time from meta-analysis. Owing to paucity of data, we were unable to stratify the analyses by year in the school (year 1, year 2, etc.), by types of school (college, university, institute), or by geographical regions. Data entry and analyses were performed using STATA 12. 0 (StataCorp, TX), and Bayesian analysis was implemented in WinBUGS (Imperial College and MRC, UK). Ethical approval was not required as the present study used the published studies.

RESULTS

The initial search yielded 359 citations, of which 24 potentially met the inclusion criteria. Of these, 14 studies (with 17 data sets) were identified for the current review.^{7,8,10,19–29} Supplemental Figure S1, http://links.lww.com/MD/A319, illustrates the study selection process.

Studies were excluded because they did not provide sufficient data on student's responses,^30–34,36 had not captured (majority of) student's engagement in research,^9,35 and were conducted with postgraduate students.^37,38

Characteristics of the Included Studies

Supplemental Table S1, http://links.lww.com/MD/A319, presents the characteristics of the included studies. Four studies were carried out in the USA,^19–21,23 whereas 2 studies were in Canada^22,26 and the UK.^7,27 One study was done in each of the following countries: China,³⁴ Croatia,¹⁰ Malaysia,²⁸ the Netherlands,⁸ Norway,²⁵ and Pakistan.²⁴ The type of studies included cross-sectional design in 4 studies,^24,25,28,29 case-control design in 1 study,¹⁹ and longitudinal cohort design in 1 study.¹⁰ Some studies^7,8,20–23 did not provide sufficient information on their study design.

In the 14 included studies, the measurement tools for assessing the students’ responses to their engagement in research were diverse and were used differently in the individual studies. For instance, a web-based questionnaire of the institution (self-made questionnaire) was used in 1 study,²⁵ whereas a questionnaire adapted from previously published studies in medical education was used in 2 studies^7,10 and adaptation from a validated questionnaire in another study.²⁴ The questionnaires consisted of exclusive closed-ended response formats²⁶ or closed-ended response formats with an additional open question.²⁵ Also, in 1 study the mode of responses was not anonymous,²⁵ whereas code-matching was used in another study¹⁰ and anonymity of the responses in 1 study.²⁰ The response rate ranged from 34% at the lowest³² to 85% at the highest.²² Moreover, the outcome measurements varied among the included studies. In 2 studies, the outcome of association was measured and expressed as correlation coefficient.^10,24 The year in which medical students were involved in research also varied among studies. For instance, the participants were years 4 and 5 medical students in 1 study,²⁴ whereas in other studies they were year 1 to year 6 students.¹⁰ Of the 14 studies identified for this analysis, 7 studies provided sex-specific information, showing the majority of the respondents were males.^{7,10,20,22–25} In general, the outcome assessments were not done by using the same questionnaire across all studies, and even if used, the outcomes were not consistently reported; these differences rendered some difficulties in performing the pooled analysis.

Methodological Assessment

Some studies did not provide sufficient methodological details to enable a rating on their quality (see Supplemental Table S2, http://links.lww.com/MD/A319). The majority of studies in this review were judged as low quality, as they did not use/mention the (standard) survey construction method and the survey questions as well as data analysis method were inappropriate.

Research Experience (Learners’ Reaction)

From a pooled analysis of the 6 studies (with 8 data sets),^{20,23–25,27,29} the “learners’ reaction” was positive in 74% (95% CI: 60.8%–87.6%; I²: 97.9%) of those who were engaged in research (Figure 1). A pooled analysis of the 2 studies (with 3 data sets)^26,28 showed that 23% (95% CI: 15.5%–31.3%; I²: 73.8%) of the respondents were negative in response, indicating lack of interest in research activities (data not shown).

FIGURE 1.

Forest plot showing positive responses in the learner's reaction.

Attitudes to Science in Medicine (Learning)

A pooled analysis of 6 data sets of the 4 studies^7,9,26,28 showed that approximately half (summary prevalence: 49.5%; 95% CI: 36.4%–62.7%) of those students who engaged in research had positive attitudes toward research in medicine, indicating “research motivates/facilitates/advances further research/career.” Of note, there was substantial within-studies heterogeneity (I²: 93.4%) (Figure 2). In 1 study, a significant and positive but low correlation between research skills and motivation toward doing research was also reported (r: 0.324, P < 0.001).⁷ In an included study,¹⁰ the majority of respondents had positive attitudes toward science in medicine and this was more pronounced in the years 3 and 6 students (mean score 69.8 ± 9.5).

FIGURE 2.

Forest plot showing the learning outcome.

Self-Reported Changes in Practice (Behavior)

Seven studies reported data on students’ self-reported changes in practice,^20–26 indicating “ease/confidence in career choice/job opportunity.” The pooled analysis showed that the majority of students (62.3%; 95% CI: 46.7%–77.9%) who engaged in research (while in medical student) had positive behavior changes in practice after experiential learning of research. Of note, there is a substantial within-studies heterogeneity (I²: 96.3%) and wider confidence interval (Figure 3). A study¹⁰ showed that students’ attitudes and their Grade point average were weak but positively related among those students involved in research activities (r: 0.251, P < 0.002).

FIGURE 3.

Forest plot showing the self-reported changes in practice.

Change at the Level of the Organization (Publication Output)

Three studies reported data on change at the level of the organization pertinent to publications as their research output.^7,19,25 A pooled analysis showed that 64% (95% CI: 30.8%–96.6%) of students who engaged in research had published their work. Also, there were substantial within-studies heterogeneity (I²: 98.5%) and wider confidence interval (Figure 4). A study by Reinders et al⁸ reported that approximately 50% (51/103) of those students who gained extracurricular research experience published an article before graduation and an average of 6 articles published after graduation.

FIGURE 4.

Forest plot showing the publication output.

Other Outcomes

In a study,²⁷ 74% of the year 4 students reported that they have gained further understanding of some of the ethical issues underpinning medicine. This was 44.3% and 58.7% in the year 2 and year 1 students, respectively. In another study,²⁴ the vast majority of students (60.9%; 120/197) reported that they would plan and conduct research project without supervision. Either because of the differing questionnaire items or variation in measurement, pooled analysis was not possible on all items.

Results of Bayesian Approach

Posterior distribution of estimates for 4 educational outcomes were presented in Table 1. Overall, analyses with Bayesian approach showed the results, which were not nearly identical to the original estimations.

TABLE 1.

Bayesian Model Results

Sensitivity Analysis

After removing a study,²⁷ the positive responses toward “learner's reaction” increased from 74% (95% CI: 60.8%–87.6%; I²: 97.9%) to 83% (95% CI: 69.5%–97.2%, I²: 96.6%), but with a little change in I² (see Supplemental Figure S2, http://links.lww.com/MD/A319). After removing a study,²⁰ the positive responses toward “learner's attitudes” slightly decreased from 49.56% to 43.7%, but with a considerable increase in I² (93.4%–71.7%) (figure not shown). This tended to reflect an impact of study quality on the estimation. Owing to limited data, were are unable to do subgroup analysis. The number of individual studies reported in consistent manner was less than the recommended minimum number of 10;¹⁷ we were therefore not able to check for publication bias by using the visualizing funnel plot.

DISCUSSION

The present study attempted to provide insights into the students’ perceptions toward their engagement in research while at the medical school. Overall, the findings suggest that those students who engaged in research (while at the medical school) showed positive responses toward their research experiences, toward medical science, and self-reported changes in their practices. Students also contributed to the publication output of their university, reflecting better image at the organizational level. Indeed, such reactions are not separable, but linked to each other and can be explained theoretically.

Theoretical Senses

Elsewhere, for example,^36,39 it has been found that when students hold positive attitudes toward an academic subject (research activities in this case), it is easier for them to study that subject (to perform research in this case) and feel the subject useful to their studies and future career (self-reported changes for improvement in their practices in this case), they tended to possess higher “self-efficacy” in studying the relevant subject (eg, publication output in this case). The research productivity is regarded as an important outcome as it provides assurance that their efforts are recognized.³⁶ Self-efficacy is a person's belief in his or her ability to succeed in a particular situation⁴⁰ within the timeline set for such activities. Using “motivational theory,”⁴¹ positive learner's reaction to research activities further acts as motivational factor for them to be engaged in research activities, at least for the specific module, and encourages more effort and hence better achievement (in this case in research). Overall, carrying out research, while at medical school, provides an opportunity for students to study a particular discipline in which she/he is interested in⁴¹ and is an important factor influencing achievement in higher education.^42,43

Points to Ponder

Findings of this review indicate that some students were not interested in research. Those students with negative reactions would be less likely to put their efforts into studying research. A published systematic review on attitudes of medical studies to medical leadership reported that possible barriers to implementing research activities within medical schools are a lack of time given competing educational demands.⁴⁴ Attitudes are defined as a disposition or tendency to positively or negatively respond to an idea and are difficult to change.¹⁰ This might explain why the responses in the current review were diverse; some were responded to positives, whereas others responded differently.

Furthermore, because of the cross-sectional nature of the studies included in this review, even the findings of positive attitudes in the current review lacked the power to assess a possible causal relationship between the engagement in research and attitudinal changes. Thus, the increase observed in positive attitudes could be confounded by other contextual factors such as the type of learning activities used across the program,⁴³ supportive campus environment,^1,3,45 having a physician mentor with research experiences,³⁶ student-faculty contact,^1,3,46 and mandatory participation in research activities,^19,25 among others. Taken together, our findings suggest that multifactorial approach should be undertaken by medical institutions, with an aim to enhance the students’ involvement in research activities.

Study Limitations

The primary studies with the exclusive closed-ended response formats and/or nonanonymity of the respondents could allow guessing of answers correctly or provide socially desirable responses, leading to inaccuracies in the findings.⁴⁵ Variation in the study year could contribute to both selection and information bias. Student's knowledge and attitude toward health research, for example, may be attenuated significantly with increasing years of education at medical school.²⁴ Moreover, the majority of surveys included in the present study had a higher proportion of male respondents. This could create a bias toward source of response (information). What we still need to know is whether more males have joined medical schools according to the sex-specific entry quota of the universities; this is, however, beyond the scope of our objectives. In doing the pooling of studies, we acknowledge that limited number of studies with limited outcome measures might have attributed to type 2 statistical errors and selection bias. Furthermore, there existed a substantial heterogeneity among studies. Bayesian approach and sensitivity analysis showed uncertainty of the estimation. Hence, the findings should be interpreted in the light of these bias.

Strengths Encountered

The current review has included a considerable number of primary studies across settings, robustly assessed the methodological quality of the included studies, measured the well-accepted educational outcomes, applied a standard method of data synthesis, and explained the findings in the light of theoretical rationale. These could be regarded as the strengths of the current review. The Association for Medical Education in Europe (AMEE) has launched an ASPIRE program to promote excellence in medical education, and one of the recommendations on the range of students’ engagement in medical school is to include engagement in research.⁴⁷ Along this line, research within the curricula of medical schools becomes part of a recent trend in (many) medical schools.^6,35

CONCLUSION

This review has provided insights into the students’ engagement in research during medical school. Findings showed that engagement in research resulted in favorable reactions toward research and academic learning. The current work also sheds light on the need for future well-designed studies using standardized research tools on how to engage students in research.