Creating a Centralized Infrastructure to Facilitate Medical Education Research

Abstract

PURPOSE

Building research capacity and increasing scholarly productivity are identified needs of the specialty of family medicine. The Accreditation Council for Graduate Medical Education (ACGME) has increased the scholarly requirements for residency programs, placing even more pressure on faculty to be productive in the scholarly realm. The Council of Academic Family Medicine Educational Research Alliance (CERA) was created by volunteer members of the specialty with shared interests in overcoming barriers and increasing scholarly production.

METHODS

CERA has developed the infrastructure and expertise to regularly conduct omnibus surveys of key family medicine educational leaders. Proposals are centrally collected and competitively chosen. The omnibus survey process includes collaboration with experienced mentors, centralized institutional review board clearance, pilot testing, and centralized data collection. The survey results are disseminated back to research teams for presentation and publication of the findings.

RESULTS

To date, over 115 research teams have had their projects included in CERA omnibus surveys. Projects have been led by research teams from across the country and with a wide variety of research experience. This collaborative work has resulted in more than 75 scientific presentations and over 55 peer-reviewed papers in the medical literature. The raw data are now available online and serve as a repository for future secondary analysis and as an educational resource.

CONCLUSIONS

The CERA infrastructure has allowed a large number of research teams to conduct meaningful scholarship at a fraction of the typical cost in terms of time and energy. CERA has expanded family medicine research by removing barriers for teams with limited experience or resources.

BACKGROUND

The specialty of family medicine has struggled to establish a research agenda and expand its research footprint for decades.^1,2 Building research capacity has been a stated goal of key leaders in the specialty.^3,4 Increased training in scholarship and increased research productivity were key recommendations of the Future of Family Medicine project.⁵ Yet, family medicine researchers remain relatively underfunded and underproductive.^3,6

The Family Medicine Review Committee of the Accreditation Council for Graduate Medical Education has increased the scholarly requirements several times over the last 2 decades.² Family medicine residents are now expected to produce 2 scholarly projects during their residency and faculty are expected to be regularly productive.⁷ Evidence suggests that family medicine residency programs struggle to meet these requirements.⁸

Scholarly activity is primarily driven by individuals, although there has been a move toward team-based science. Even in the context of team-based science and translational research (eg, Clinical and Translational Science Awards), most research is conducted with institutional resources. Clinician educators and those conducting medical education research tend to have few resources for conducting research.² Many individuals in family medicine residency programs have little funding and few experienced colleagues for collaboration.^5,8

To help address these needs, the Council of Academic Family Medicine Educational Research Alliance (CERA) was created.⁹ The Council of Academic Family Medicine (CAFM) is an administrative committee made up of the leadership of 4 family medicine organizations: The Society of Teachers of Family Medicine (STFM), The Association of Departments of Family Medicine (ADFM), The Association of Family Medicine Residency Directors (AFMRD), and the North American Primary Care Research Group (NAPCRG). This initiative is unique in that it is not an individual institution-based program but rather a specialty-wide activity bringing together individuals through voluntarism to increase family medicine scholarship.

The original published mission and goals of CERA (Table 1) established an agenda that included: facilitating collaboration, reducing barriers to research, creating opportunities for novice researchers, and facilitating resident scholarship. Addressing these goals will allow larger numbers of family medicine faculty to become more deeply involved in research. We have depicted several levels of involvement in the Research Roles Pyramid (Figure 1). CERA facilitates faculty “climbing to a higher level” on this pyramid. Participating in the middle levels of the pyramid, where new knowledge is generated, provides individuals tacit experience with the research process which in turn makes them better consumers of research as well as better teachers of research principles. Having more family medicine faculty functioning at higher levels on the pyramid will create a virtuous cycle that normalizes scholarship and generates even more scholars within the specialty.

Table 1.

The Original Mission and Goals of CERA

The mission of CERA is:

To guide the specialty by providing leadership and vision in the arena of medical education research

To set a standard within family medicine for medical education research that is rigorous and generalizable

To facilitate collaboration between medical education researchers

To provide mentoring and education to junior researchers

The goals of CERA are:

Increase research and scholarly activity among members of CAFM organizations by providing infrastructure and consultation

Improve the process for surveying constituents with better questions and fewer surveys

Facilitate scholarship by creating a clearinghouse of data that CAFM members and family medicine residents can access

CAFM = Council of Academic Family Medicine; CERA = Council of Academic Family Medicine Educational Research Alliance

Figure 1.

Research roles pyramid.

This pyramid depicts the spectrum of roles faculty can perform when it comes to medical research. The bottom 2 levels represent continuous learning about, and the translation of, new medical knowledge. All faculty participate in research through consumption of the literature and all those that provide patient care participate by being direct utilizers of research findings. The middle 2 levels represent actual participation in the generation of knowledge. Ideally, a robust number of faculty will actively collaborate in the research generation in order to produce broadly applicable findings. A smaller number, with significant experience and interest, will become principal investigators answering their own questions. The top 2 levels represent leading others in the generation of new knowledge. Those researchers with the most and broadest experience will become positioned to lead research programs or even to set research agendas for institutions and broad networks of researchers.

METHODS

CERA has begun achieving its goals through regular omnibus surveys. Target groups for these surveys included residency program directors, clerkship directors, family medicine department chairs, and primary care practice–based research network directors, as well as the CAFM general membership.

The process by which CERA surveys are created and administered was iteratively refined and standardized.¹⁰ A call for proposals is put out to all CAFM members through a variety of mechanisms, including listservs and e-mail notifications. A panel of experienced researchers evaluates and scores the proposals to help the survey director identify the best question sets to be included in each survey. The researchers whose proposals are selected for inclusion are then paired with an experienced mentor. In collaboration with their mentors, research teams refine their portion of the survey and submit the final questions to the survey director. Each survey embeds core demographic questions that have remained relatively unchanged over time.

Once all questions to be included in a survey are finalized and collated, the survey is piloted by the survey director among a small group of experienced researchers. This insures content validity and clarity of the questions. Pilot testing often results in small survey refinements. The final version is submitted to the Institutional Review Board of the American Academy of Family Physicians for approval with the survey director fulfilling Principal Investigator duties.

The surveys are electronically sent by CERA administrative personnel to the target audience utilizing commercially available software. Reminders and encouragement to complete the surveys are periodically sent via the software, through listservs of the various CAFM organizations, or through direct contact. Data are collected, stored, and minimally cleaned as needed by the CERA administrative staff. The research teams, along with their mentors, are then sent deidentified demographic results and their question set results. Teams are encouraged to check with their local Institutional Review Boards to determine if local approval is desired. Teams and mentors have an exclusive 90-day window to analyze their data. Mentors are expected to meet the minimum standard for authorship on any presentations or publications that result from this process. After the exclusive 90-day window, results are placed on online and can be accessed for secondary analysis by any CAFM member. The data can also be used as educational tools to teach research methods.

RESULTS

As of October 2017, CERA has completed 30 omnibus surveys. More than 300 proposals were reviewed by CERA and over 115 have been accepted for inclusion in the surveys. The experience level of accepted research teams ranges from complete novices to highly experienced career researchers. Some proposals included family medicine residents among the teams. Most topics were educational but policy, professional, and clinical topics were also well represented.

More than 75 scientific presentations of survey results were accepted at regional, national, and international conferences. Over 55 subsequent papers were published in the peer-reviewed medical literature. Journals publishing findings from CERA surveys included top-tier medical education journals and several medical subspecialty journals, as well as family medicine journals. Table 2 details the productivity of CERA projects to date. Dozens of presentations and publications from the most recent surveys are in various states of production. Additional scholarly projects may exist but are unknown to the authors.

Table 2.

CERA Productivity as of October 2017

CERA Projects	No.
Completed CERA omnibus surveys	30
Received proposals	>300
Accepted proposals	117
Known accepted presentations at medical meetings	77
Known accepted publications in the medical literature	58

CERA = Council of Academic Family Medicine Educational Research Alliance.

Note: An updated list is available online at http://www.stfm.org/Research/CERA/CERAScholarlyActivity.

Some research teams have successfully analyzed and published secondary analyses of the data available on the CERA website.¹¹ Some family medicine residencies are encouraging residents to use CERA as a strategy to fulfill their scholarly activity requirements either by submitting proposals for inclusion to the omnibus surveys or by conducting secondary analyses of the existing data.

DISCUSSION

The centralized oversight and processing methods utilized by CERA take a great deal of the burden of conducting research off of individual research teams. This has resulted in an impressive number of individuals being able conduct meaningful research at a fraction of the investment in terms of time and energy. CERA enables research to be conducted by teams with less experience or modest resources. Medical learners are also able to utilize this resource to meet the increasing standards of scholarly activity.

CERA has now matured and stabilized. Its infrastructure has grown considerably. CERA now conducts 4 to 5 surveys per year on a rotating schedule. Research teams are getting more sophisticated in how they utilize the survey. For instance, some teams have been able to use results as pilot data to apply for research grants. Questions can now be repeated to evaluate changes over a 6-plus year time frame.

CERA is now looking to expand its capabilities beyond this one study design. An initial extension being explored would link survey findings of program directors focused on curricular features to future surveys of graduates focused on their current practice patterns. This will allow correlation of curricular features to practice outcomes of interest and would guide future curriculum design. CERA would like to grow into a central coordination hub that can facilitate multicenter medical educational studies within medical schools and residencies. A robust, centralized research organization with existing infrastructure and experience will allow participation of smaller institutions without significant local resources to become more active players in medical educational research. As family medicine faculty become more familiar with using CERA, they will encourage resident use to fulfill ACGME requirements for scholarly activity. The long-term impact of all this will be to create a larger cadre of family researchers that are more capable of developing a coherent research agenda for the specialty.

The concept of omnibus surveys is not unique. The Clerkship Directors in Internal Medicine do an annual omnibus survey of their members.¹² Using similar methods, CERA surveys many target audiences and serves the entire community of family medicine educators.

There are some limitations to using CERA as a research platform. The calls for proposals and competitive selection process does exclude some teams from the final omnibus survey. This will be viewed as a hurdle by some while being viewed as quality control by others. Additionally, most of the proposals submitted are relatively family medicine–centric which may call into question the generalizability of the survey results.

CONCLUSIONS

CERA has increased family medicine’s overall research productivity and increased the actual number of amily medicine faculty actively participating in research. By creating an infrastructure capable of overcoming some key barriers to performing research, CERA has facilitated scholars with less experience, or less resources, to successfully perform meaningful scholarship. The CERA model could be replicated in other specialties to facilitate collaborative research.