Abstract
Background
To examine research background, attitudes, knowledge and skills of family medicine residents with regard to primary care research and to compare residents who elected to participate in the research stream with those who did not.
Methods
Mailed survey of Family Medicine residents at McMaster University in 1998, 70% (52/74) of whom responded. The main outcome measures consisted of research background; attitudes towards primary care research and research activities during residency program; knowledge and skills in applying it in biostatistics, epidemiology, and research design.
Results
The vast majority of the residents reported previous research experience and/or some training in epidemiology and biostatistics. Residents in the research stream were more likely to be female and were positive towards primary care research: they were more interested in research, more interested in obtaining more research training while a resident, and placed more importance on developing research early in medical education. The research stream residents had stronger views regarding perceived lack of support staff and lack of time for research. There were no statistically significant differences between the research stream and other residents in terms of research knowledge and skills in applying it.
Conclusions
Attitudes towards research rather than research knowledge or skills seemed to distinguish those selecting to be in our new research stream at the inception.
Background
Family medicine is evolving from a specialty established primarily to meet the demand for skilled clinicians to serve rural and urban communities, to a research based discipline mirroring other specialties whose progress in knowledge is based on methodologically sound research. Without primary care research to advance medical knowledge, many myths and misconceptions in family medicine will undoubtedly persist [1,2]. Unfortunately, there is a shortage of primary care researchers in North America. Only 0.2% of graduates of US family practice residencies surveyed in 1991 were primarily entering research careers [3]. To partially address this shortage of researchers, many family residency programs in US and Canada require residents to complete a research project prior to graduation in the hopes of stimulating their graduates to pursue research.
Although residency program directors may embrace research during residency training [1,4-9] there have been few studies on whether research during residency training increases the number of primary care researchers. The literature suggests that the secret to creating a successful research environment during residency involves: starting early in the residency program, support from the program director or a committed faculty member, allotment of time for research, an integrated and structured research curriculum, use of evidence based medicine, faculty involvement, a research committee, an easily accessible research professional, research visibility, and opportunities for presenting [5,6,10]. Frequently cited barriers to effective research curricula include: lack of protected time, insufficient funds for support staff, lack of mentors and lack of research skills in faculty members [1,2,4,7,8,10-14]. Studies have shown that postgraduate research involvement is strongly associated with research participation during medical school [3,10,14-16]. However, it appears that required research during medical school may not lead to more physician-investigators [16].
The "Family Medicine Research Stream" at McMaster University Postgraduate Program in Family Medicine was established in 1997/98 academic year and was designed to provide the environment needed for interested residents to engage in meaningful research. Our goal was to maximize optimal use of limited research resources by enhancing research experience for a self-selected group of family medicine residents.
The family medicine residency program at McMaster University is one of the handful of programs in North America that does not require its residents to submit a mandatory research or academic project. However, resident research is encouraged and an elective in research is available to those interested. The program has a research director and a full-time research coordinator who both have expertise in research methodology, statistical analysis, knowledge of research application software, and grant-writing experience.
Twelve out of 74 residents who were enrolled in the residency program in 1997/98 academic year joined the research stream. The research stream curriculum, while flexible in scope, provides structured resources for administrative, methodological and supervisory support. Residents are encouraged either to develop their own projects or to link with ongoing research activities within the department or elsewhere. Workshops are provided 3 to 4 times per year for educating residents on research methodology and statistical analysis, discussion of research projects, and brainstorming future ideas. Research Workshops also involve a half-hour plenary session followed by a roundtable discussion in which residents give a progress report on their activities. Residents in the research stream are invited to tap into the expertise of mentors, while addressing any obstacles, challenges, or successes they may be experiencing in their own research projects.
Our principle objective in doing the survey was to describe family medicine residents in terms of their research background, attitudes, knowledge and skills with regard to primary care research and to determine whether there were important differences between residents who chose to participate in the research stream and those who chose to remain in the regular curriculum. We were also anticipating that the survey will provide us with the information required to develop a relevant research curriculum for the family residency research stream.
Methods
Study population
The study sample consisted of family medicine residents attending the Departments of Family Medicine at McMaster University during 1997/98 academic year. All residents were eligible to participate.
Study design and survey procedures
A mailed survey questionnaire was distributed in January 1998 accompanied by a cover letter signed by the family medicine postgraduate director at McMaster University via interhospital mail to all family medicine residents enrolled in the program in the 1997/98 academic year. The follow-up strategies using the modified Dillman method [17] included mailed and emailed reminders, full mailings of the questionnaires, and telephone calls to non-respondents by a research assistant.
Survey instrument
As there was no existing survey instrument, a questionnaire was developed and screened for content by the investigators. While the questionnaire had adequate face and content validity, it was also pilot-tested on a convenience sample of family medicine faculty at McMaster University and then further refined. There were three sections to the self-administered questionnaire totaling four pages. Section 1 obtained information about educational background, past experience in research and biostatistics, and residents' research interest using a set of open-ended questions. Section 2 sought information about residents' attitudes regarding research in family medicine on 15 items assessed on a 10 point Likert scale with 1 being strongly disagree and 10 strongly agree. The last section consisted of 18 multiple-choice questions (MCQ) intended to assess respondents' knowledge and skills in biostatistics, epidemiology, and research design. The questionnaire is available from the principal author upon request.
Analysis
Data analysis was carried out using SPSS for Macintosh software (version 4.0 for Macintosh, SPSS Inc., Chicago, 1990). The descriptive statistics (frequencies, means and SD) were calculated for all questions. The mean percent MCQ score was calculated for the knowledge questions. The significance of differences in response between the research stream and other residents was measured using chi-square tests or Fisher's exact test for categorical variables and a multivariate analysis of variance (MANOVA) omnibus test followed by Student's t-tests for continuous variables. Because of multiple comparisons p < .01 was accepted as the minimum criterion for significance and two tailed tests were used in all analyses.
Results
Response rate and respondents profile
A total of 52 questionnaires, 11 from the residents in the research stream and 41 from the residents in the regular program were completed, resulting in a response rate of 70% (52/74). The response rate was higher for residents in the research stream (91%, 11/12) than for those in the regular curriculum (66%, 41/62) (Fisher's exact test, p = .095). The overall demographic and research background profile of respondents as well as a comparison between the research stream residents and other residents in terms of these characteristics is given in Table 1. Nineteen percent of residents (10/52) reported to have advanced degrees, 71% (37/52) described previous research experience, and 61% (32/52) had some training in epidemiology and biostatistics. There were no statistically significant differences between the research stream and other residents with regard to the above attributes.
Table 1.
Characteristic | Overall n = 52 | Research stream n = 11 | Regular stream n = 41 | p value |
Female: n (%) | 32 (62) | 10 (91) | 22 (54) | .035 |
McMaster graduate: n (%) | 21 (40) | 7 (64) | 14 (34) | .095 |
R1'S: n (%) | 29 (56) | 8 (73) | 22 (54) | .319 |
Advanced degree: n (%) | 10 (19) | 2 (18) | 8 (20) | 1.000 |
Research experience: n (%) | 37 (71) | 8 (73) | 29 (71) | 1.000 |
Training in biostatistics or epidemiology: n (%) | 32 (62) | 8 (73) | 24 (59) | .497 |
The most important factors that would facilitate research activity: | ||||
More time | 18 (35) | 3 (27) | 15 (37) | |
More support | 22 (42) | 8 (73) | 14 (34) | |
Not interested | 12 (23) | 0 (0) | 12 (29) | .039* |
* Chi-square test, all other comparisons used Fisher's exact test
The respondents indicated that broadly defined research support (42%) and adequate time (35%) were the two most important factors that would facilitate greater research interest and activity during family practice residency while 23% indicated no interest in research. The research support category included factors such as increased research mentorship, research opportunities, role modeling, biostatistical support and funding. While the research stream residents were more likely to be female and graduates of McMaster University, these differences were not statistically significant at p < .01. The participants were also asked to list their three main research or interest areas. The research stream residents listed more key words than those in the regular program did; however, the most frequent responses in both groups were similar and most frequently included health promotion/public health, preventive medicine, and adolescent health.
Attitudes
The residents' general attitudes towards the primary care research and research activities during residency program were assessed through 15 items presented in Table 2. Statements that received highest mean scores, either in terms of agreement or disagreement, included a strong endorsement for a statement advocating strengthening of the scientific foundations of clinical practice (mean = 7.9, SD = 1.7), agreement with a statement that there is a lack of time to pursue research during the residency training (mean = 7.4, SD = 2.0), and strong disagreement with statements that there are no opportunities to participate in primary care research (mean = 3.6, SD = 1.9) and that there are no family medicine research role models in our department (mean = 3.9, SD = 2.3). The remaining 11 statements received mean scores clustered around the middle of the scale.
Table 2.
STATEMENT (all items measured on 10 point Likert scale: 1 'Strongly disagree'; 10 'Strongly agree) | Overall Mean (SD) (n = 52) | Research stream Mean (SD) (n = 11) | Regular stream Mean (SD) (n = 41) | p* value |
With the increased emphasis on primary care, there is a need for strengthening the scientific foundations of clinical practice. | 7.9 (1.7) | 9.1 (1.5) | 7.6 (1.6) | .008 |
Family medicine residents should be required to carry out research projects as part of their residency. | 4.6 (2.8) | 6.2 (3.1) | 4.2 (2.6) | .037 |
Research is adequately emphasized in family medicine training at McMaster. | 5.6 (2.0) | 4.0 (2.0) | 6.0 (1.8) | .002 |
The lack of a critical mass of investigators is a major impediment to the development of primary care research. | 6.1 (1.9) | 7.5 (2.4) | 5.7 (1.6) | .007 |
Developing a culture of primary care research must occur early in professional medical education. | 6.6 (2.4) | 8.5 (2.3) | 6.1 (2.2) | .003 |
It is imperative that the "evidence" in the Evidence-Based Medicine paradigm is developed by primary care researchers. | 5.8 (2.3) | 5.9 (1.9) | 5.8 (2.4) | .845 |
Every family medicine resident should be assigned to a research mentor at the beginning of their training. | 5.2 (2.9) | 6.0 (2.9) | 5.0 (2.9) | .301 |
Family medicine faculty at McMaster are knowledgeable about research. | 6.3 (1.8) | 5.4 (2.0) | 6.5 (1.6) | .052 |
I am not interested in research activities. | 5.5 (3.3) | 2.2 (2.1) | 6.5 (2.9) | .000 |
There is a lack of dedicated time to pursue research activities during the residency training. | 7.4 (2.0) | 8.3 (1.7) | 7.2 (2.1) | .121 |
I have good research skills. | 5.4 (1.9) | 5.7 (2.6) | 5.3 (1.7) | .549 |
I would like to get more research training as part of the family medicine residency training. | 5.9 (3.1) | 8.8 (2.0) | 5.1 (2.8) | .000 |
There are no opportunities for family medicine residents to participate in primary care research. | 3.6 (1.9) | 3.5 (1.6) | 3.6 (1.9) | .838 |
There are no family medicine research role models in our department. | 3.9 (2.3) | 3.8 (2.1) | 3.9 (2.4) | .892 |
There is sufficient statistical/epidemiological support available to residents who are interested in research. | 5.8 (2.1) | 4.5 (2.0) | 6.2 (2.1) | .017 |
* Based on Student's t-tests
Overall attitudinal differences were first tested by multivariate analysis of variance with 15 items serving as dependent variables and resident research status as an independent variable. This analysis revealed significant omnibus effect of the resident research status across all 15 items [F(15, 36) = 2.96, p = .005], suggesting that the responses of the two groups of residents were significantly different. The univariate follow-up tests identified 6 statements for which the mean scores of the research stream residents differed significantly from those of other residents at p < .01. Not surprisingly, the research stream residents had significantly higher mean scores on the item "I would like to get more research training as part of the family medicine research training" and significantly lower scores on the item "I am not interested in research activities" than those residents not in the research stream. They were also significantly less likely to agree with the statement that "Research is adequately emphasized in family medicine training at McMaster". The responses of the research stream residents also differed significantly, and in the expected direction, with regard to 3 general statements concerning primary care research. While not significantly different at p < .01, the research stream residents were less likely to agree with the statement that family medicine faculty at McMaster are knowledgeable about research and that there is sufficient statistical/epidemiological support available to residents who are interested in research. They were also somewhat more likely to agree that required research projects should be part of residency training.
Biostatistical and epidemiological knowledge
Section 3 involved 18 MCQ questions testing knowledge of research methodology, epidemiology and biostatistics and it's application to the solution of problems. The mean percent MCQ knowledge score was 66.0% (SD = 17.8%). Questions regarding sampling strategies, statistical tests such as chi square, epidemiological definitions such as sensitivity, specificity, predictive value, validity, attributable risk, odds ratio, mean, mode, median, standard deviation were answered correctly by at least 60% of residents. Poorly answered questions included items on research ethics, research terminology such as triangulation, t-tests, cost-benefit analysis, incidence vs. prevalence vs. mortality/case fatality rates. The overall mean score of the research stream residents while more than 10 percent higher than those of other residents did not reach statistical significance (mean 74.7%, SD= 10.4% vs. mean 63.7%, SD = 18.8%; t = 1.9, df = 50, p = .07). Questions answered correctly by residents not in the research stream were answered correctly by slightly higher percentage of residents in the research stream while poorly answered questions followed a similar pattern in both groups.
Discussion
The internal/external validity of this study may be limited by several factors. First, this was a self-report survey. Second, the survey instrument itself was not completely validated. Third, because the maximum number of respondents was predetermined, some important differences in terms of background characteristics, research attitudes, or self-reported knowledge might have be missed because of inadequate statistical power. Finally, the sample surveyed might not be representative of family medicine residents elsewhere in North America.
Despite the above limitations certain findings can be restated: the vast majority of the residents reported previous research experience and/or some training in epidemiology and biostatistics and there were no differences in that regard between those in the research stream and those in the regular curriculum. Although most residents had positive attitudes towards primary care research, those who self-selected themselves to be part of the research stream tended to be more positive. Statistically significant differences were found between those in the research stream and those in the regular program in terms of research interest, interest in obtaining more research training while a resident, importance of developing research early in medical education, and adequacy of research emphasis at McMaster. In other words, this study suggests that it comes down to interest in research, rather than knowledge, skills, or prior research involvement. While such conclusions are perhaps self-evident to some, they underscore the larger problem of family medicine research in North America.
Residents in the research stream had stronger views regarding perceived lack of support staff and lack of time for research. Temte et al also found that residents who were actively involved in a research project identified the importance of time and resource personnel to assist with research design and data analysis more than residents who were interested in pursuing research during residency, but were not currently active [3]. It was also surprising that those in the research stream were no more confident in their research skills than those in the regular stream even though they tended to perform better on the knowledge questions.
The study by Temte et al, in which they examined factors associated with research interest and activity in Wisconsin family medicine programs, reported that 85.3% residents felt that research experience was desirable for family physicians [3]. However, many McMaster residents remained neutral or disagreed with the statement that family medicine residents should complete research projects during their training or should be assigned a research mentor. Even those in the research stream were relatively lukewarm in their views on these topics (mean = 6.2, SD = 3.1 and mean = 6.0, SD = 2.9; respectively). As in the Wisconsin study, those residents who were interested in research tended to be of lower training year and female [3].
Studies indicate that residents differ in their attitudes towards required research projects. In a study by Konen et al, half of the family medicine residents from a community hospital based, university affiliated training program in Syracuse, NY felt that the exercise was an unnecessary annoyance [18]. A survey of family practice program's graduates who completed their training between 1987–1992 at the University of Toronto and who had participated in a required academic project during their residency program indicated that most research projects had been literature reviews. Furthermore, the majority of respondents felt neutral or negative about their projects and 79% would not have done a project if it had been optional [19]. This differs from a 1993 study that described a six-year experience with required research projects in a pediatric residency program. A questionnaire administered to residents after the completion of their project and presentation, found that 62% believed completing a research project was quite worthwhile, and only 19% felt unfavorable about the project [9].
Research training during residency can provide a structured environment where research activity is encouraged and valued and necessary resources are likely to be available. Most family medicine residency programs in North America have addressed the issue of strengthening the research base of primary care research by implementing required research projects, generally with mixed results. Our results suggest that there is a demand for enhanced research experience for a selected sub-group of family medicine residents. The research stream approach implemented at McMaster University might offer unique opportunities not only to optimize the use of limited research resources but also to create a model of multidisciplinary collaborative primary care research training program with potential applications in other residency programs. This baseline assessment was important to our understanding of the similarities and differences in those residents who self-selected to join our research stream at the inception.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Acknowledgments
Acknowledgments
This work was supported by a Teaching and Learning Grant from McMaster University and was presented at the 1999 Trillium Primary Care Research Forum, Toronto, Ontario.
Contributor Information
Kaczorowski Janusz, MA PhD, Assistant Professor, Email: kaczorow@mcmaster.ca.
Sellors John, MD MSc CCFP FCF, Senior Medical Advisor, Email: jsellors@path.org.
Walsh Allyn, MD CCFP FCFP, Associate Professor, Email: walsha@McMaster.ca.
References
- DeHaven MJ, Wilson GR, Murphree DD, Grundig JP. Family practice residency program directors' views on research. Family Medicine. 1997;29:33–7. [PubMed] [Google Scholar]
- Giveon S, Kahan E, Kitai E. Factors associated with family physicians' involvement in research in Israel. Academic Medicine. 1997;72:388–390. doi: 10.1097/00001888-199705000-00020. [DOI] [PubMed] [Google Scholar]
- Temte JL, Hunter PH, Beasley JW. Factors associated with research interest and activity during family practice residency. Family Medicine. 1994;26:93–7. [PubMed] [Google Scholar]
- Culpepper L. Family medicine research: Status at the end of the first decade. JAMA. 1983;249:63–68. doi: 10.1001/jama.249.1.63. [DOI] [PubMed] [Google Scholar]
- DeHaven MJ, Wilson GR, O'Connor-Kettlestrings P. Creating a research culture: What we can learn from residencies that are successful in research. Family Medicine. 1998;30:501–7. [PubMed] [Google Scholar]
- Geyman JP. Research in the family practice residency program. The Journal of Family Practice. 1977;5:245–248. [PubMed] [Google Scholar]
- Norton PG, Dunn EV, Shaughnessy R. A required resident project in a family medicine program. Can Fam Phys. 1989;35:89–94. [PMC free article] [PubMed] [Google Scholar]
- Wilson JL, Redman RW. Research policies and practices in family practice residencies. The Journal of Family Practice. 1980;10:479–483. [PubMed] [Google Scholar]
- Winter RJ, Unti SM, Collins JW. Research during residency training. Teaching and Learning in Medicine. 1993;5:96–101. [Google Scholar]
- Mills OF, Zyzanski SJ, Flocke S. Factors associated with research productivity in family practice residencies. Family Medicine. 1995;27:188–93. [PubMed] [Google Scholar]
- Alguire PC, Anderson WA, Henry RC. Teaching research skills: Development and evaluation of a new research program for residents. Teaching and Learning in Medicine. 1993;5:37–43. [Google Scholar]
- Brouhard BH, Doyle W, Aceves J, McHugh MJ. Research in pediatric residency programs. Pediatrics. 1996;97:71–73. [PubMed] [Google Scholar]
- DeHaven MJ, Wilson GR, Murphree DD. Developing a research program in a community-based department of family medicine: One department's experience. Family Medicine. 1994;26:303–8. [PubMed] [Google Scholar]
- Shapiro J, Coggan P, Rubel A, Morohasi D, Fitzpatrick C, Danque F. The process of faculty-mentored student research in family medicine: Motives and lessons. Family Medicine. 1994;26:283–9. [PubMed] [Google Scholar]
- O'Brien Gonzales A, Westfall J, Barley GE. Promoting medical student Involvement in primary care research. Family Medicine. 1998;30:113–6. [PubMed] [Google Scholar]
- Segal S, Lloyd T, Houts PS, Stillman PL, Jungas RL, Greer RB., 3d The association between students' research involvement in medical school and their postgraduate medical activities. Academic Medicine. 1990;65:530–533. doi: 10.1097/00001888-199008000-00010. [DOI] [PubMed] [Google Scholar]
- Dillman DA. Mail and telephone surveys: The total design method. New York, NY: John Wiley & Sons. 1978.
- Konen JC, Fromm BS. A family practice residency curriculum in critical appraisal of the medical literature. Family Medicine. 1990;22:284–7. [PubMed] [Google Scholar]
- Morris BAP, Kerbel D, Nguyen L-T. Family practice residents' attitudes toward their academic projects. Family Medicine. 1994;26:579–82. [PubMed] [Google Scholar]