Skip to main content
SAGE Open Medicine logoLink to SAGE Open Medicine
. 2017 Sep 11;5:2050312117730245. doi: 10.1177/2050312117730245

Research Associates Program: Expanding clinical research productivity with undergraduate students

Wirachin Hoonpongsimanont 1,, Preet K Sahota 1, Nathan N Ng 1,2, Maryam J Farooqui 1, Bharath Chakravarthy 1, Bhakti Patel 1,3,4, Shahram Lotfipour 1
PMCID: PMC5598797  PMID: 28932396

Abstract

Objectives:

Clinical research is often time-consuming and difficult to conduct in busy academic institutions. Previous studies have proposed methods to integrate undergraduate students as a means to increase research productivity. The authors aimed to describe the possibility to enhance emergency department research productivity at an academic emergency department in the United States, using undergraduate students in an Emergency Medicine Research Associates Program.

Methods:

The authors described the Emergency Medicine Research Associates Program curriculum and its implementation. We also conducted a retrospective study at a university-based emergency department from January 2005 to December 2014 to demonstrate the benefit of having an established Emergency Medicine Research Associates Program. The primary outcomes were number of Emergency Medicine Research Associates Program–related studies, number of enrolled patients, extramural/intramural funding, abstract presentations, and peer-reviewed publications. The authors analyzed the data using descriptive statistics.

Results:

Over the 10-year period, 110 Emergency Medicine Research Associates Program–assisted research studies were conducted, with research associates enrolling 46,219 patients. These studies yielded a total of 31 peer-reviewed publications and 77 abstract presentations (13 international, 27 national, 37 state/regional). The Emergency Medicine Research Associates Program–related studies were used as pilot studies to obtain US$1,751,036 in extramural grant funding and US$31,047 in intramural grant funding.

Conclusion:

The implementation of Emergency Medicine Research Associates Program can enhance emergency department clinical research productivity, and the inclusion of supplemental academic programs enhanced the undergraduate students’ research experience.

Keywords: Research associates, research productivity, research program, undergraduate, emergency department, funding

Introduction

Since the formation of academic emergency medicine (EM) as a specialty in the 1960s, emergency departments (ED) in the United States have struggled with conducting patient-oriented research simultaneously with fulfilling clinical responsibilities.14 Despite the recent increase in the number of EM journals,5 the success of academic EM research programs continues to be hindered by high patient volume, lack of protected research time, weak research infrastructure, and little federal research funding.2,611 Compared to other specialties, academic EM historically has received the lowest amount of funding per faculty member and less than 1% of National Institutes of Health (NIH)-sponsored extramural funding.8 Consequently, many academic EM research efforts faced inadequate funding and required alternative approaches.1,12, 13

One alternative approach focused on the recruitment of undergraduate college students as trained research associates (RAs) to conduct clinical research.6,1417 Implementing this strategy in nursing and the ophthalmology department resulted in greater numbers of prospective clinical studies, patient enrollments, and publications among faculty and residents.1420

The Emergency Medicine Research Associates Program (EMRAP) was first implemented in 1997. EMRAP consisted of both a clinical research component and an academic curriculum for up to 25 undergraduate students per quarter. Students have learned clinical research methodology, prepared grant proposals and manuscripts, and planned for careers in healthcare.

Our study described the impact of EMRAP on research productivity in the ED. We have outlined a cost-efficient, structured student research program that enhances ED research productivity and provides a solid clinical research foundation for undergraduate students.

Materials and methods

Study design

We described the EMRAP curriculum and its implementation at our institution. We also conducted a retrospective analysis of the research productivity, utility, and benefits of the EMRAP program from January 2005 to December 2014. The following resources provided the data presented in this analysis: weekly meeting notes, faculty physician curriculum vitae (CV), the Undergraduate Research Opportunities Program (UROP) database, ED research funding records, and the PubMed journal database. The study received institutional review board (IRB) exemption as non-human subject research.

Data analysis

Our primary outcomes were the following: number of EMRAP-related studies, number of enrolled patients, extramural/intramural funding, abstract presentations (regional/state, national, international), and peer-reviewed publications. We analyzed the data using descriptive statistics in Microsoft Excel Version 15.13.1 and GraphPad Prism 7.0 software (GraphPad Software, La Jolla, CA).

EMRAP

Program development and leadership

Since 1997, EMRAP has trained undergraduate RAs to enroll patients and collect data for multiple ED studies while educating students on clinical and epidemiologic research methodology. A faculty director supervises the program, the interactions between RAs, and communication with principal investigators (PI). The faculty director selects two RAs to become program coordinators and one RA to become an academic coordinator. Program coordinators manage the administrative branch of EMRAP and lead weekly meetings with the RAs to discuss research productivity. The program coordinators work with the academic coordinator to formally train RAs in conducting clinical research. Upon completion of training, RAs have the opportunity to serve as a research coordinator (called EMRAP liaison) for specific research studies. Under the supervision of the PI, liaisons are responsible for the coordination and operational oversight of the research project in the ED (Figure 1).

Figure 1.

Figure 1.

Research program organization and leadership.

EMRAP academic curriculum

In 2012, EMRAP launched an academic-based curriculum led by the faculty director and academic coordinator to supplement clinical research education for the RAs. The EMRAP academic curriculum consists of faculty-led lectures and workshops, techniques on writing manuscripts, and oral and poster presentations. Through the academic program, RAs have gained a more comprehensive understanding of the research process, while acquiring skills in writing grant proposals and conducting basic statistical analyses. To assess the RAs’ understanding of ongoing research studies—including study design, inclusion/exclusion criteria, enrollment procedures, and sample size calculations—a written examination is administered to all RAs at the start of the academic year. With the support of UROP, RAs have learned to write grant proposals, receive funding, and present their findings at the annual UROP symposium. RAs also present abstracts at other regional and national conferences and submit peer-reviewed manuscripts under direct supervision of the PI (Figure 2).

Figure 2.

Figure 2.

Research program training and academic curriculum.

RA training requirements

RAs complete annual training modules certified by the university, including Health Insurance Portability and Accountability Act (HIPAA) training tutorials and the Collaborative IRB Training Initiative (CITI program). Additionally, RAs receive annual IRB-compliance training from the institutional compliance officer. Once certified, incoming RAs undergo a training period for approximately 2 months under the direct supervision of veteran RAs and the program coordinators. RAs also receive training in the Research Electronic Data Capture (REDCap) database to store enrollment data and log ED shift productivity. The program coordinators use the REDCap tool to account for EMRAP productivity.

RA responsibilities

Enrollment of patients

RAs are assigned to an average of two 4-h ED shifts per week from 8 a.m. to midnight, 7 days a week. Shifts comprised screening, approaching, and enrolling eligible patients into active studies. Up to three RAs cover each shift, for a total of 28 shifts per week, to maximize patient enrollment. EMRAP can accommodate patient enrollment from midnight to 8 a.m. for specific studies on a case-by-case basis. RAs assist with all research projects within the department, including prospective studies, retrospective studies, randomized controlled trials (RCT), and pharmaceutical trials. RAs are able to initiate the consent process; however, the faculty researchers and departmental research staff must finalize consent for subject participation, for each study.

Research coordinator (liaison)

RAs can pursue leadership opportunities as research coordinators (liaisons). The coordinator’s role is to facilitate communication between the PI and the RAs, supervise patient screening, verify data collection, and work with the PI to facilitate study completion. Coordinators have the opportunity to assist in the preparation of a research abstract, present findings at conferences, and prepare manuscripts for publication.

Program sustainability: time and cost

EMRAP consists of up to 25 RAs quarterly, providing a 1-year program commitment, who each dedicate 10–12 h weekly to research. The undergraduate students participate in the program for academic credit. The program director contributes approximately 2 h/week to manage the program with a budget of US$2000 yearly for administrative expenses.

Results

Clinical studies and patient enrollments

From 2005 to 2014, EMRAP assisted in 110 ED studies, in which RAs participated in enrolling 46,219 subjects. The number of yearly clinical research projects increased from three studies in 2005 to 20 studies by 2014 with an average of 11 studies per year (Table 1). RAs’ annual patient enrollments increased from 441 subjects in 2005 to 4297 subjects in 2014 with an average of 4622 enrollments per year (Table 1, Figure 3).

Table 1.

Variable totals and averages per year (2005–2014).

Average (per year) Total
EMRAP-assisted studiesa
 Active ED studies 11 110
 Patients enrolled 4622 46,219
 Peer-reviewed publications 3 31
 Abstract presentations
  State/regional 4 37
  National 3 27
  International 1 13
Funding
 Extramural grants 2 15
 Extramural grant amount awarded US$175,103.63 US$1,751,036.25
 Intramural UROP grant proposals 9 86
 Intramural UROP grant amount awarded US$3104.70 US$31,047
 Studies attending UROP symposiums 8 83
 Intramural SURP amount US$620 US$6200
 ED studies participating in UROP 13 129

EMRAP: Emergency Medicine Research Associates Program; ED: Emergency Department; UROP: Undergraduate Research Opportunities Program; SURP: Summer Undergraduate Research Program.

a

Numeric values rounded up to the nearest whole number.

Figure 3.

Figure 3.

Number of EMRAP-related (a) active clinical studies and (b) enrolled subjects from 2005 to 2014.

Published manuscripts and abstract presentations

From 2005 to 2014, EMRAP-assisted research studies had one published manuscript in 2005 and seven manuscripts in 2014, for a total of 31 publications. The number of abstract presentations at the state/regional, national, and international meetings averaged eight abstracts presented annually (Figure 4).

Figure 4.

Figure 4.

Number of EMRAP-related (a) published manuscripts and (b) abstract presentations from 2005 to 2014.

Extramural and intramural funding

From 2005 to 2014, EM faculty obtained over US$1.75 million in extramural funding through EMRAP-assisted studies, either by using EMRAP-assisted study results as pilot study data or by listing EMRAP as one of the institutional resources available to ensure data enrollment feasibility. Extramural funding constitutes all funding received from external institutions and agencies to conduct EM research within EMRAP.

Intramural funding constitutes all funding received from the university’s UROP program. RAs obtained a total of US$31,047 intramural, UROP grant funding under their PI supervision. There was an increase in UROP grant proposals and funding during this duration. While no proposals were submitted in 2005, US$4350 in UROP grant funding was obtained in 2014 through 13 undergraduate grant proposals, with an average of US$3105 awarded each year. The highest amount of funding awarded was US$4700 in 2013. The lowest amount of funding awarded was US$1975 in 2007, with the exception of US$0 awarded in 2005 (Figure 5).

Figure 5.

Figure 5.

Number of EMRAP-related studies receiving (a) extramural and (b) intramural funding and (c) submitted UROP proposals from 2005 to 2014.

Discussion

In EMRAP-assisted studies, RAs were able to enroll 46,219 subjects over a 10-year period. ED faculty published 34 manuscripts, presented 77 abstracts (state/regional, national, and international), and received over US$1.78 million in extramural/intramural funding. Implementation of EMRAP enhances ED clinical research productivity.

In comparison, Hollander and Singer16 reported an increase in publication rate from 2 to 20 manuscripts and abstracts per year using a similarly structured research program. However, it is unclear whether these publications are from studies only conducted by their RAs program or from the entire ED. In contrast, the studies and resulting publications reported in our study are all from original research and only include those conducted through EMRAP.

RAs receive graded academic units in place of hourly compensation; thus, lowering the cost of research conduction. Clinical-oriented research has significant costs to maintain a robust patient enrollment, as one study reported an annual cost of US$52,833 using 10 college-level paid RAs.6 In contrast to other RA programs, which require external funding for hiring full-time managers and staff,6,18 EMRAP remains both cost and time efficient, requiring 25 RAs with 10 h/week, at a cost saving of US$130,000 annually.

A significant proportion of enrollments were from retrospective studies, suggesting that low-risk, minimal intervention research studies may be optimal for college-level RAs in a university-level research program.18 These studies do not require any active patient screening or consent and reduce the time needed per enrollment. Subsequently, we observed higher enrollments during years with higher proportions of retrospective studies. In 2010, over 15,000 of the 17,800 enrolled patients were from a single retrospective study. Similarly, in 2011, 30% of the studies were retrospective (7215 enrollments), while only 11%–15% of the studies were retrospective from 2012 to 2014 (2184–4297 enrollments).

Supplementing the EMRAP clinical research with an academic component prepares EMRAP for submitting grant proposals and encourages RAs to obtain intramural funding. Through the academic curriculum, RAs were trained in grant proposal planning, manuscript writing, and delivering oral presentations.

Similar programs report involvement of RAs in authored publications and presentations.15 RAs also received valuable clinical and research experience that helped guide their decisions to pursue careers in healthcare and postgraduate school.14,18,20 Participation in RA programs may potentially be attributed to the higher rates of acceptances to medical schools (79% compared to Association of American Medical Colleges (AAMC) national average of 49%) observed in student RAs.21 Thus, many of the observed benefits to RAs arise from early undergraduate clinical research exposure, which in turn may lead to greater interest in health careers that include patient outcomes research.2,22

These results are not exclusive to an academic EM setting. The implementation of an undergraduate research program can offer other clinical departments and specialties a method to conduct original research projects and maintain clinical-duty time, costs, and overall productivity, as previous studies have shown.1420 This study contributes to the existing literature on the effectiveness of undergraduate research programs for both the faculty and students.

Limitations

The reported data relied on the accuracy of faculty CVs, the UROP database, and archived meeting notes. Missing data, such as missing meeting notes or out-of-date CVs, may result in underreported values for each variable. A prospective evaluation of EMRAP would potentially yield more accurate results, as described in previous studies.6,14,15,20 Additionally, there are no data on the total number of patients screened for each study.

RAs are not assigned from midnight to 8 a.m. due to low patient volume during these hours. Consequently, the enrollments may represent a convenience sampling for EMRAP-related studies. The department should provide the additional coverage for these hours to capture consecutive samples.

While our ED experienced a robust increase in research productivity from 2005 to 2014, the increase may be a result of other factors that occurred during this time period aside from the implementation of EMRAP, such as hiring new faculty and staff.

Furthermore, there is no method available to objectively evaluate the quality of data obtained by the RAs in addition to controlling for variation in productivity between the RAs. There were no control or comparison groups for this study. An association between the implementation of EMRAP and an increase in ED research productivity cannot be made as baseline information is not available.

Conclusion

Over the past 10 years, we observed an increase in EMRAP-assisted active research studies and subjects enrolled per year. Consistent with previous findings on the effectiveness of using undergraduate students as RAs, we support the implementation of a low-cost, well-trained undergraduate RAs program in conducting clinical research in the ED. Additional specialties may consider the adoption of a program similar to EMRAP to incorporate students early on in the healthcare setting while not compromising the physician’s focus on providing care for patients. Supplemental academic programs could enhance RAs’ research experience and research program funding.

Supplementary Material

Supplementary material

Footnotes

Declaration of conflicting interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval: The research study received Institutional Review Board (IRB) exemption as non-human subject research.

Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplementary Material: Supplementary material is available for this article online.

References

  • 1. Suter RE. Emergency medicine in the United States: a systemic review. World J Emerg Med 2012; 3: 5–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Holsti M, Adelgais KM, Willis L, et al. Developing future clinician scientists while supporting a research infrastructure. Clin Transl Sci 2013; 6: 94–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Short A, Holdgate A, Ahern N, et al. Enhancing research interest and collaboration in the interdisciplinary context of emergency care. J Interprof Care 2009; 23: 156–168. [DOI] [PubMed] [Google Scholar]
  • 4. Gallagher EJ, Goldfrank L, Anderson GV, et al. Current status of academic emergency medicine within academic medicine in the United States. Acad Emerg Med 1994; 1: 41–46. [DOI] [PubMed] [Google Scholar]
  • 5. Lee CH, Shih CP, Chang YC, et al. The evolution of academic performance in emergency medicine journals: viewpoint from 2000 to 2009 journal citation reports. Acad Emerg Med 2011; 18: 898–904. [DOI] [PubMed] [Google Scholar]
  • 6. Cobaugh DJ, Spillane LL, Schneider SM. Research subject enroller program: a key to successful emergency medicine research. Acad Emerg Med 1997; 4: 231–233. [DOI] [PubMed] [Google Scholar]
  • 7. Kaji AH, Lewis RJ, Beavers-May T, et al. Summary of NIH medical-surgical emergency research roundtable held on April 30 to May 1, 2009. Ann Emerg Med 2010; 56: 522–537. [DOI] [PubMed] [Google Scholar]
  • 8. Bessman SC, Agada NO, Ding R, et al. Comparing National Institutes of Health funding of emergency medicine to four medical specialties. Acad Emerg Med 2011; 18: 1001–1004. [DOI] [PubMed] [Google Scholar]
  • 9. Rosenzweig JS, Van Deusen SK, Okpara O, et al. Authorship, collaboration, and predictors of extramural funding in the emergency medicine literature. Am J Emerg Med 2008; 26: 5–9. [DOI] [PubMed] [Google Scholar]
  • 10. Wright SW, Wrenn K. Funding in emergency medicine literature: 1985 to 1992. Ann Emerg Med 1994; 23: 1077–1081. [DOI] [PubMed] [Google Scholar]
  • 11. Williams GH, Wara DW, Carbone P. Funding for patient-oriented research. Clinical strain on a fundamental linchpin. JAMA 1997; 278: 227–231. [PubMed] [Google Scholar]
  • 12. Ernst AA, Houry D, Weiss SJ. Research funding in the four major emergency medicine journals. Am J Emerg Med 1997; 15: 268–270. [DOI] [PubMed] [Google Scholar]
  • 13. Nathan DG. Clinical research: perceptions, reality, and proposed solutions. National Institutes of Health Director’s Panel on Clinical Research. JAMA 1998; 280: 1427–1431. [DOI] [PubMed] [Google Scholar]
  • 14. Bradley K, Osborn HH, Tang M. College research associates: a program to increase emergency medicine clinical research productivity. Ann Emerg Med 1996; 28: 328–333. [DOI] [PubMed] [Google Scholar]
  • 15. Hollander JE, Valentine SM, Brogan GX. Academic associate program: integrating clinical emergency medicine research with undergraduate education. Acad Emerg Med 1997; 4: 225–230. [DOI] [PubMed] [Google Scholar]
  • 16. Hollander JE, Singer AJ. An innovative strategy for conducting clinical research: The academic associate program. Acad Emerg Med 2002; 9: 134–137. [DOI] [PubMed] [Google Scholar]
  • 17. Davis DP, Poste JC, Kelly D. The UCSD research associate program: a recipe for successfully integrating undergraduates with emergency medicine research. J Emerg Med 2005; 28: 89–93. [DOI] [PubMed] [Google Scholar]
  • 18. Steadman PE, Crudden J, Boutis K. Implementation of a volunteer university student research assistant program in an emergency department: the nuts and bolts for success. CJEM 2015; 17: 586–589. [DOI] [PubMed] [Google Scholar]
  • 19. Jamerson PA, Fish AF, Frandsen G. Nursing Student Research Assistant Program: a strategy to enhance nursing research capacity building in a Magnet status pediatric hospital. Appl Nurs Res 2011; 24: 110–113. [DOI] [PubMed] [Google Scholar]
  • 20. Marcus I, Salchow DJ, Leung SL, et al. Academic Associate Program impact on clinical study enrollment in an academic ophthalmology practice. J Acad Ophthalmol 2013; 6: 69–75. [Google Scholar]
  • 21. Sparano DM, Shofer FS, Hollander JE. Participation in the Academic Associate Program: effect on medical school admission rate. Acad Emerg Med 2004; 11: 695–698. [PubMed] [Google Scholar]
  • 22. Steadman PE, Crudden J, Naranian T, et al. The professional benefits for volunteer research assistants in the pediatric emergency department. J Emerg Med 2014; 48: 287–293. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary material

Articles from SAGE Open Medicine are provided here courtesy of SAGE Publications

RESOURCES