Evaluation of a Required Senior Research Project in a Doctor of Pharmacy Curriculum

Abstract

Objective

To characterize the dissemination of study findings and assess project preceptor attitudes towards a required senior research project in a doctor of pharmacy (PharmD) curriculum.

Methods

A survey was conducted to determine preceptors' perceptions of the value of a required pharmacy student research project and dissemination of research results.

Results

One hundred fifteen project preceptors (92.0%) responded. Most preceptors agreed that the projects provided a valuable learning experience to the students (87.5%) and were of value to them professionally (82.1%) and to their institution (78.2%). Study findings were disseminated primarily through institutional forums (47.3%). A smaller percentage of projects were disseminated externally through presentations at professional meetings (23.7%, poster presentations; 4.0%, platform presentations), and peer-reviewed publications (5.3%).

Conclusions

Despite a modest level of dissemination of project results through presentations at professional meetings and a low level of dissemination via published manuscripts, the majority of preceptors perceived a required student research project to be of value.

INTRODUCTION

The American Association of Colleges of Pharmacy (AACP) Commission to Implement Change in Pharmaceutical Education has advocated research as a means to advance the profession of pharmacy, increase knowledge, develop critical analytical skills, and “foster among graduates an obligation to participate in inquiry and professional improvement.” It further stated that “a profession dedicated to continued learning enhances the profession's ability to serve the public.”¹ In response to this charge, a number of colleges and schools of pharmacy have incorporated a student research project as a requirement within the PharmD curriculum.

Previous reports have assessed the status of research-related efforts in colleges and schools of pharmacy across the United States.^2-4 In 2006, Murphy and colleagues evaluated PharmD programs that offered senior research projects in addition to research-related coursework in pharmacy school curricula.² In this survey of 88 US colleges and schools of pharmacy, 20 of 79 (25%) PharmD programs that responded required the completion of some component of a research project, and 12 (15%) required students to fulfill all components of a complete project. This included a project proposal, institutional review board (IRB) submission, collection and analysis of data, and a final presentation and/or written report defending the findings of the project.² In a 2008 survey of 95 colleges and schools of pharmacy, similar results were obtained with 20 of 79 respondents (25%) reporting completion of a research project as a graduation requirement.⁴

Because the overall value of research is diminished when the findings are not disseminated, tracking the extent of dissemination of results from student research projects is important. Furthermore, given the significant time required of research project preceptors, many of whom are volunteer faculty members, assessing preceptor attitudes toward the required project, including perceived value and impact on student learning, is also important. At the time of this writing, only 1 published study had examined faculty members' attitudes toward required senior research projects for PharmD candidates.⁵ Although faculty members were generally positive about the value provided by the senior research project, the study did not attempt to evaluate the full roster of faculty members who served as project preceptors, and the extent of the dissemination of research results was only estimated.

The PharmD curriculum at the University of California, San Francisco (UCSF) School of Pharmacy includes research-related classroom work, and all graduates are required to complete a senior research project. The research project has been a graduation requirement for all students since 2002, when 3 curricular pathways were instituted at UCSF: pharmaceutical care, pharmaceutical sciences, and pharmaceutical health policy and management. The majority of students (approximately 80%) are enrolled in the pharmaceutical care pathway, which has a greater emphasis (compared to the other more research-oriented pathways) on patient care and the development of clinical skills in a variety of settings during the advanced pharmacy practice experience (APPE) phase of the curriculum. For this pathway, the goals of the required senior research project are to:

• Apply the scientific process and the various components of research methodology to investigate and answer a professional or practice-oriented question (eg, hypothesis testing, data collection, evaluation, IRB approval, data analysis, data critique);

• Provide a capstone experience that enables the students to integrate and apply information or concepts learned in previous courses (eg, statistical analysis, drug information, therapeutics, literature evaluation);

• Foster collaboration and teamwork among students and project preceptors;

• Prepare students for future careers in pharmaceutical care;

• Encourage dissemination of project results at professional meetings and through publications; and

• Provide value to APPE sites and contribute to the advancement of the profession.

With 6 years of pharmaceutical care pathway student projects completed, our aim was to conduct a systematic evaluation to determine the extent to which the goals of dissemination and providing value to APPE sites have been met. Specifically, the objective of this study, which in itself was conducted as a student research project, was to characterize dissemination of study findings and to assess project preceptor attitudes toward the required senior research project.

METHODS

Research Project

The UCSF pharmaceutical care pathway defines the student research project as a “collection, analysis, interpretation, critique, and presentation of quantitative or qualitative data required to answer a professional or practice-oriented question.” Students in the pharmaceutical care pathway initiate their research project at the end of the third year, concurrent with the start of their clinical APPEs. Students can complete the project individually or in teams under the guidance of paid or volunteer faculty preceptors with the expectation that each student will devote approximately 80 hours of work to the project. In general, team research projects can be conducted by a maximum of 4 students, commensurate with the scope and personnel necessary for the project. The projects typically are started in April or May and concluded by March of the following year (Table 1). All studies involving human subjects must be reviewed and approved by the IRB of the research site or the UCSF Committee on Human Research, as applicable. After completion of the project, students are required to submit a final paper and formally present their findings to faculty members and fellow APPE students. Of note, the time allocated to project completion is separate from the time devoted to APPEs. Students are responsible for managing their time and coordinating with team members and project preceptors, while fulfilling all concurrent APPE requirements. Research projects included in this investigation were completed by graduates in the pharmaceutical care pathway from 2002 to 2007. Projects were completed while enrolled at APPE sites located throughout California. To determine the types of projects completed, investigators reviewed the available written documentation for each project (eg, final reports, IRB applications, conference presentation handouts) and independently classified each project as either experimental (treatment) or nonexperimental (observational).

Table 1.

Suggested Timeline for Required Senior Research Project^a

Abbreviations: IRB = institutional review board.

^aTimelines may vary due to nature of project (eg, analysis of existing data versus primary data collection).

^b IRB training is required of all students during coursework in the third year.

Survey Instrument

To characterize dissemination outcomes for each project and preceptor perceptions of student projects in general, 2 brief Web-based survey instruments were developed: (1) a project-level survey instrument, assessing the individual project experiences and dissemination outcomes, and (2) a global survey instrument, assessing project preceptor perceptions of the student research project assignment in general. Draft survey instruments were reviewed by faculty members (n = 5) and revised as needed, after which the primary preceptor for each project was asked, by e-mail, to complete 1 survey instrument for each project and the global survey instrument. The survey procedures were as follows: (1) a notification e-mail describing the study was sent 1 week prior to survey launch; (2) an e-mail with a unique preceptor-specific link to the survey instrument was sent at survey launch; (3) a reminder e-mail was sent 2 weeks after survey launch; and (4) a final reminder e-mail was sent 2 weeks after the first reminder e-mail. The faculty preceptor telephoned nonresponders to verify receipt of the e-mails and to serve as a final reminder to complete the survey instruments. The survey instrument solicited the following information about each project:

Dissemination of research findings.

The project-level survey instrument solicited dissemination outcome data, including whether the project was presented at an institutional forum (eg, a pharmacy and therapeutics committee meeting, departmental quality assurance meeting, or other institutional meeting), presented as a poster at a professional meeting, presented as a platform presentation at a professional meeting, or submitted for publication. Poster and platform presentations were further classified as being presented at local, state, national, or international professional meetings. Publication submission was characterized as: (1) submitted but not accepted, no plans to resubmit; (2) submitted but not accepted, plan to resubmit; (3) revised and resubmitted, currently under revision; and (4) accepted and published, or manuscript in press. If a project was reported as published, the citation was requested for verification.

Preceptor perceptions of individual projects.

Preceptors were asked to respond to a series of statements regarding each project, including (1) precepting this project was valuable to me professionally; (2) the results of this project were valuable to my institution/organization; (3) the students were adequately prepared to conduct this research project; (4) the project provided a valuable learning experience for the student(s); and (5) the students viewed the research as an important contribution to their overall education. Response options included 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Finally, an estimate of the total number of hours devoted to precepting the project was also requested, with an option to specify inability to recall the total number of hours.

Preceptor global perceptions of the student research project assignment.

The global survey instrument consisted of 4 statements: (1) the pharmaceutical care project provides a valuable learning experience for students; (2) the school of pharmacy should continue to require that all students complete a pharmaceutical care pathway project; (3) students' time would be better spent in other learning experiences; and (4) the pharmaceutical care project should be an optional honors thesis for those interested in learning about research, instead of a requirement for all students. Response options included 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree.

Statistical Analyses

Statistical analyses involved computation of simple summary statistics to characterize the survey responses. For group comparisons, t tests and/or nonparametric comparisons were computed, as appropriate. Analyses were conducted using SPSS for Windows, version 17.0 (SPSS Inc, Chicago, IL). Study procedures were approved by the UCSF Committee on Human Research.

RESULTS

From 2002 to 2007, 605 students graduated from the pharmaceutical care pathway at the UCSF School of Pharmacy; of these, 601 (99.3%) were linked to one of 235 completed projects (2.6 ± 1.2 students per project). Of 125 primary preceptors, 115 (92.0%) completed the Web-based survey instruments, providing dissemination outcomes and perceptions for 224 of the 235 projects (95.3%). The majority of projects were nonexperimental or observational in nature (n = 213, 95.1%).

Of 224 projects for which dissemination data were provided by primary preceptors, 47.3% were disseminated as a presentation at an institutional forum, 23.7% were disseminated as a poster, 4.0% were disseminated as a platform presentation, and 10.7% were submitted for publication. At the time of data collection, at least 1 year after completion of all projects included in the analysis, manuscripts from 12 projects (5.3% of total; 50% of those projects for which a manuscript had been submitted) had been accepted, and either was published or in press (Table 2).^6-17 A student served as the lead author on 8 of the publications,^6-8,12-16 and no students were authors on 2 of the publications.^9,11 Overall, 22.8% of projects were externally disseminated (ie, poster/platform presentation or publication), and 34.4% were disseminated via an internal institutional forum only. Projects precepted by paid faculty members were more likely to be published (p < 0.005), but were not more likely to be externally disseminated (p = 0.71).

Table 2.

Dissemination of Research Findings (n = 224)^a

^a42 projects (18.8%) were disseminated via multiple channels.

Most preceptors (82.1%) agreed that precepting the project was valuable to them professionally, and 78.2% agreed that the results of the project were valuable to their institution (Table 3). The majority of preceptors (73.2%) believed that the students were adequately prepared to conduct the research, and 87.5% believed that the project provided a valuable learning experience for the student(s). Nearly two-thirds of preceptors agreed that the students viewed the project as important to their overall education.

Table 3.

Preceptor Perceptions: Project-level Survey (n = 224 Projects)

Response scale range: 1 = strongly disagree to 5 = strongly agree. Data presented as percentages of valid total.

Preceptor perceptions about the student research projects in general (the global survey instrument) were provided by 114 preceptors (99.1% of all preceptors). The majority (87.7%) agreed that the pharmaceutical care project provides a valuable learning experience for students, and 74.5% of preceptors agreed that the project requirement should be continued. Some preceptors (14.0%) believed that students' time would be better spent in other learning experiences, and 32.5% of preceptors believed that the project should be an optional honors thesis for those interested in research. For items 1, 2, and 4 in Table 4, faculty members who had precepted fewer than 3 projects (n = 90) viewed the student learning experience more positively than did faculty members who had precepted 3 or more projects (n = 24) (p values < 0.05).

Table 4.

Preceptor Perceptions: Global Survey Assessing the Research Project Assignment in General (n = 114 Preceptors)

Response scale range: 1 = strongly disagree to 5 = strongly agree. Data presented as percentages of valid total.

For 153 projects (68.3%), preceptors reported the total number of hours allocated per project; the total number of hours could not be recalled for 71 projects. On average, preceptors reported allocating an average of 38.9 ± 47.7 or median of 25 hours per project (IQR = 15-50 hours).

For the subset of 153 projects for which the number of faculty hours was reported, an estimate of the total hours per published manuscript was computed. This computation incorporated average faculty member time (as reported in the survey instrument) and student time (average, 2.6 students per project at 80 hours each, according to credit-hour designations). Overall, 37,655 hours were allocated toward the 153 projects, from which 5 publications derived. This equates to 7,531 hours per peer-reviewed publication.

DISCUSSION

Previous studies suggest that about a quarter of colleges and schools of pharmacy in the United States require completion of a research project as a PharmD degree graduation requirement.^2,4 While professional organizations^1,18,19 strongly endorse the integration of research within PharmD curricula, there are limited published data evaluating the perceptions of faculty members⁵ and students²⁰ in programs that incorporate research as a required curricular experience, and even less information on the impact and outcomes of these experiences with respect to the creation and dissemination of new knowledge.^5,21 In this comprehensive evaluation of a 6-year experience with a required research project in a PharmD curriculum, we found overall favorable impressions of the experience from preceptors, a modest level of dissemination of study findings through institutional forums (47.3%) or presentations at professional meetings (23.7% poster presentations; 4% platform presentations), and a low rate of peer-reviewed publications (5.3%).

Our dissemination findings are comparable to those of a previous study in which 8% of the approximately 400 projects conducted by PharmD students resulted in publication, while 17% of projects resulted in a presentation at a professional meeting.⁵ Several reasons could account for the small number of publications resulting from the senior research projects. From the preceptors' perspective, some projects might not be worthy of submission to a peer-reviewed journal after consideration of the significance of the study findings or the overall quality of the project. In other cases, the amount of time and effort necessary to refine the research project (eg, collect additional data, perform further analyses, revise the manuscript) may discourage the preceptor from taking the project to publication. Additionally, preceptors who are not paid members of the faculty (eg, volunteer faculty members) might have less incentive to publish study findings given differing job expectations and competing responsibilities. Indeed, our data suggest that projects led by paid faculty members were more likely to be published than projects led by volunteer faculty members. From the students' perspective, reasons may include that final paper submissions occur close to the time of graduation when more immediate concerns and obligations (eg, studying for licensing examinations, postgraduation travel, relocation efforts) compromise the time available to pursue publication. In many cases, students who satisfy course requirements for the project are uninterested or unable to devote the time necessary to revise the manuscript to be suitable for publication.

Survey respondents indicated that nearly half of all projects were used in a manner beyond fulfilling a research course requirement and were disseminated through presentations within the preceptors' organization. The immediacy and accessibility of this venue relative to publication or presentation at a professional meeting likely accounts for the substantial number of projects that were disseminated through institutional forum presentations. Furthermore, projects addressing questions or issues salient or of potential value to the preceptor's institution likely would have been presented at institutional meetings or disseminated through internal publications (eg, newsletters, reports, or other documents). While 18.8% of all research projects were disseminated through multiple channels (eg, institutional presentations, posters/platform presentations at professional meetings, publications in professional journals), approximately a third of the projects were exclusively presented within the preceptors' organization. For many preceptors, it is possible that the final desired project outcome was a report suitable for internal institutional purposes only, versus a paper to be published or presented at a professional meeting.

Similar to a previous study,⁵ our data suggest that project preceptors, in general, had favorable perceptions of the required senior research project. The majority of respondents agreed or strongly agreed that the project was valuable to the preceptor professionally, valuable to the students, and valuable to the preceptor's institution/organization. Most preceptors also agreed that the students viewed the projects as an important contribution to their overall education, and that students were prepared adequately to conduct the research. Although there may be general agreement that the senior research project is of inherent value to students as well as project preceptors and their institutions, there were more mixed opinions concerning whether students' time would be better spent in other learning experiences, or if the senior research project should be offered as an optional honors thesis only for those students interested in research (eg, not a requirement for all graduates). Reasons for this ambivalence may include the fact that some preceptors perceive time constraints, limited resources, and other competing responsibilities, coupled with the general lack of students' desire to pursue further dissemination as reasons to offer the project as an elective experience. Many of the written comments indicated that research projects should be required only for those students truly interested in research or those considering careers in research or academia. Murphy et al pointed out that requiring research projects from senior student pharmacists leads to opportunity costs in the form of time taken away from other pursuits such as full devotion to APPEs, studying, and other activities.³ One preceptor in this study commented that for students who want to work in retail settings following graduation, a mandatory research experience may not be practical and may preempt opportunities to learn other practical pharmacy issues. Other respondents indicated that the time commitment for faculty preceptors is excessive, with limited return on investment, and while it might be a valuable learning activity, in this time of limited resources it is not wise to overburden faculty members. Given that our survey estimated that project preceptorship requires a significant time commitment, colleges and schools of pharmacy considering the addition of a required research project should consider the impact on faculty workload, particularly if a primary goal is to foster dissemination through peer-reviewed publications (estimated 7,531 hours per peer-reviewed publication).

This study has limitations, including the potential for recall bias given that respondents were surveyed in 2008 about projects that were completed up to 6 years earlier. Because of the close working relationship required for preceptorship of a student research project (1 faculty preceptor per 1-4 students), recall bias is more likely to affect the estimate of hours allocated to preceptorship than for the faculty members' perceptions of the individual projects or global impressions of the required research experience. Additionally, our data might underestimate the actual number of projects that were submitted or accepted for publication given the length of time necessary for manuscript preparation and the inherent lag time in the peer review process. Finally, even though dissemination outcomes and faculty members' perceptions of the required research experience were evaluated, the perceptions of students who completed these projects were not assessed. In a published evaluation of student perceptions of a required senior research project experience, survey results indicated that students considered the activity to be a valuable learning experience that should be a continued requirement in the PharmD curriculum.²²

CONCLUSION

This study characterizes dissemination outcomes and preceptor perceptions of a required senior research project in a PharmD curriculum. The majority of projects conducted were nonexperimental or observational in nature. Preceptors perceived projects to provide institutional value (eg, through internal presentations), and nearly half of all projects were disseminated via an institutional forum. From a scholarly perspective, the projects provide less value with respect to external dissemination (23.7% poster; 4.0% platform presentation). Peer-reviewed publication was achieved by a small percentage (5.3%) of projects. In an era of limited resources, the added student and faculty workload associated with a required research project should be considered in tandem with the overall goals of the project.