Pathways to Improve Student Pharmacists’ Experience in Research

Abstract

Objective. To describe the implementation of a student research program and to provide outcomes from the initial 4 years’ experience.

Design. Students conducted individual research projects in a 4-year longitudinal program (known as Pathway), with faculty member advising and peer mentoring. A prospective assessment compared perceptions of those who completed the Pathway program with those of students who did not. Descriptive statistics, t tests, and analysis of variance (ANOVA) were used.

Assessment. The class of 2013 was the first to complete the Pathway program. In the Pathway assessment project, 59% (n=47) of students who responded reached self-set goals. Pathway students agreed that this research experience improved their ability to work/think independently, evaluate literature, and distinguish themselves from other students.

Conclusion. The Pathway program helped students understand the research process and reach other self-set goals.

INTRODUCTION

The University of Mississippi School of Pharmacy (UMSOP) is a research-intensive public institution whose 4-year professional curriculum is administered for 2 years on the main liberal arts campus, then transitioned to an academic medical enter campus for the third and fourth professional years. The Department of Pharmacy Practice (DPP) has a presence on both campuses and responsibility for courses in all 4 years of the professional program. In 2009, DPP began discussions to change the format of the Seminar Skills Development for Health Professions II course (hereafter referred to as “seminar”) from a presentation by fourth year (PY4) students describing advanced pharmacy practice experiences (APPEs) to a capstone presentation that incorporated a student research project supported by faculty advisors. These discussions led to the development of a new DPP initiative, the Pharmacy Practice Pathway (frequently referred to as “Pathway”), which was adopted in 2010. The purpose of this manuscript is to describe the development and implementation of the Pathway program and to provide outcomes from the first 4 years of experience.

In 2009, the DPP transitioned leadership and the resulting strategic planning outlined an expectation of an increase in research and scholarship from faculty members. During this time, DPP discussions related to curriculum focused on the future of the pharmacy profession, the role of research and scholarship in education, and strategies to develop student ability to thoughtfully answer questions encountered in practice and other areas. The Accreditation Council for Pharmacy Education (ACPE) 2007 Standards and Guidelines emphasized scholarship and research. Specifically, Standard 23, Guideline 23.4 referenced the need to implement strategies and programs to broaden professional horizons of students in areas such as scientific inquiry and the relevance and value of research.¹ With this guidance, DPP faculty members moved forward to broaden the learning objectives for the seminar course and to develop a longitudinal program with a research component.

A group of 6 DPP faculty members, who would later form the DPP Pharmacy Practice Pathway Committee, began the task of redesigning the PY4 seminar course and developing an overall 4-year program that incorporated best practices for research experiences. Prior to implementation, a review of literature and national search of best practices in capstone experiences was completed. A review of relevant literature supported the need for inclusion of research in schools of pharmacy curriculum.^2,3 In 2007, Murphy et al reported that about half (53%) the pharmacy schools in the United States required courses in research methods, but few (25%) required a research project and that these numbers had not changed significantly from the prior report in 1997.² In a response to the Murphy report, Ascione provided insight into the University of Michigan’s decision to continue its research requirement, stating that the university recognized pharmacy research experience had a greater impact on students beyond exposing them to research careers.³ He noted that graduates from the University of Michigan reported the research experience improved their decision-making skills, resulting in better marketability and more effective functioning in their careers.³ The 2009 report of research skills training in US schools and colleges of pharmacy by Fuji and Galt sought to further delineate the types of research skills taught and whether the design and completion of a research project was a graduation requirement.⁴ Most respondents reported teaching literature searching and critical literature evaluation (98%) and many taught interpreting research findings (75%) and selecting appropriate data-analysis procedures (61%). However, few respondents required actual study design (32%). Consistent with Murphy et al in 2007, Fuji and Galt reported that, of the schools of pharmacy responding, only 25% required completion of a research project for graduation.^2,4

Programs that required students to conduct research with faculty mentors were of particular interest to our institution. The DPP had identified a goal of implementing strategies to “ease” student transition from the liberal arts campus to the medical center campus. Establishing an advising program with advisors on both campuses seemed to be a possible strategy to bridge the locations and introduce students to distant faculty members while in their PY1 year.^5,6 The University of California, San Francisco (UCSF), not only required all graduates of a curricular pathway to complete a research project, but it also highlighted a faculty-student mentoring relationship as part of the program.^5,7 Though not published at the time, many of the UCSF program specifics were shared with our committee through e-mail and that insight assisted in the development of Pathway. Adopting an advising requirement became a priority in designing the program at the UMSOP.

DESIGN

Two specific goals of Pathway were established and serve as the basis for program design, implementation, and continued administration. The primary goal is to increase student knowledge and skills with all aspects of research through hands-on experiences. This is intended to develop a student’s ability to answer questions that may be encountered in practice or other areas and, thereby, increase graduates’ distinctiveness as they enter an increasingly competitive employment marketplace—not only for pharmacist positions, but also for residency positions, fellowships, and management roles. A second major goal of Pathway is to facilitate the student transition from campus to campus by introducing faculty members from the medical center campus earlier in the curriculum and continuing professional relationships with the liberal arts campus faculty members into later years of the program.

Course-specific objectives were developed to address Pathway program goals with student and faculty feedback. The seminar was previously formatted as PY4 student presentations detailing a specific APPE to an audience of faculty members and the PY3 class. Prior to Pathway and in the early years of implementation, PY3 attendance was required for a 1-hour pass/fail course (Seminar Skills Development for Health Professions I). Faculty members expressed concern with the quality of the presentations, repetitive content, and pass/fail nature of the course, all contributing to only 1-2 faculty members present and no outside audience.

Development of Pathway provided an opportunity for the seminar course objectives to specifically address these concerns. A second course impacted was the Problem-Solving course offered in the PY3 year. Problem-Solving is part of the problem-based learning (PBL) course series in the PY3 year; it is delivered over 4 blocks in the fall and spring semesters.⁸ Prior to implementation of Pathway, Problem-Solving course grades consisted entirely of a single examination at the end of each block. Student feedback for PY3 courses indicated a desire to expand on the single examination assessment in the Problem-Solving series. Thus, including additional opportunities for graded assignments in the Problem-Solving course series was an objective of the Pathway implementation. Additional courses were impacted by Pathway but were not a target of the program.

Pathway was created with 2 underlying components: (1) longitudinal research skills development and (2) a faculty-student advising system (Figure 1). A unique opportunity existed to incorporate longitudinal skill development over the 4 years of the professional DPP curriculum. During the time the committee was evaluating the seminar course, the school was implementing a new series of practice skills laboratories sequentially for all 4 semesters of the PY1 and PY2 years. Also, an Information Skills course moved from PY3 to PY1. These curricular revisions adopted just prior to development of Pathway provided courses within the DPP to incorporate different aspects of the program that would lead to incremental development of research skills, culminating in a PY4 seminar of a research project.

Figure 1.

Longitudinal Requirements of the Pharmacy Practice Pathway (PS=problem-solving; A/A=adviser/advisee; G=graduation)

Longitudinal research skills development is incorporated via embedded lectures and activities in DPP courses during PY1 and PY2. The members of the Pathway committee provide introductory Pathway material in the fall semester of the PY1 Information Skills course. In the spring, a lecture during the PY1 Skills Lab series assists with formulating a research topic, including how to generate a research hypothesis. Students submit their hypotheses in the spring semester of PY1 for review by the Pathway committee. During PY2 courses, development of a student’s chosen topic is supported through lectures and laboratory activities focused on writing a research proposal and Institutional Review Board (IRB) application. During this year, students submit their research proposal background and references to their advisor (graded for completion by advisor) and research proposal to the Pathway committee (approved and graded for content by the committee). In the PY3 year, students submit an IRB application (if needed) and conduct the project. Research progress benchmarks are embedded in the PY3 Problem-Solving course series and are assessed each block for a course grade. In the PY4 year, students present the outcomes of their research in the seminar course. Students may complete any of the assignments ahead of schedule in PY1-PY3, if desired.

The faculty-student advising system was designed so that students would be part of a “family” consisting of 2 to 3 students in each professional year and one faculty advisor. This structure allows faculty advising as well as peer input. Upper-level students can help guide underclassmen through the research process, as well as in other areas of professional school. Faculty advisors and students on both campuses have required meetings at which research projects are discussed. Faculty members and students use technology such as Skype (Microsoft, Redmond, VA), chat boards, FaceTime (Apple Inc., Cupertino, CA), and when available, video conferencing, to unite students from both campuses and at a variety of APPE sites.

Initially, student and advisor assignments were based on professional and personal common interests. A personal profile form was developed and completed by both students and advisors then matched by a Pathway committee member. This process has since moved to an electronic system, E*Value (Advanced Informatics, Minneapolis, MN), in which students rank their advisor choices. Most DPP faculty members are assigned 2 families.

The Pathway committee meets twice monthly to address implementation issues if any, to assess program outcomes, to develop student and advisor training programs, and to plan for future activities. Training opportunities for advisors are provided at least twice yearly during scheduled DPP faculty development sessions, during which timelines, changes to IRB policies, and advisor expectations are reviewed. Advisor feedback for additional training needs determines the topics for these sessions. For example, an improvement in data analysis knowledge and skills was determined to be a need, so personnel from the Department of Biostatistics provided workshops. It was also determined that protocols demonstrated a lack of sufficient grammar, therefore the writing center offered their resources to faculty members. Additionally, committee members were available for advisor needs, including one-on-one development sessions, if necessary.

Pathway was introduced to the class of 2013 as PY1 students during the 2009-2010 academic year, and they were the first class to complete the 4-year program. The outcomes for this class are the focus of the remainder of this paper.

The Pathway committee planned for formal assessment of the program while students from the initial class (of 2013) were completing their projects. This prospective study consisted of an assessment tool and a survey administered to students (Appendix 1). The assessment tool was designed to measure added value of Pathway and growth in research skills and knowledge. This was a 10-item paper instrument which asked students to answer questions following several research scenarios. The 13-item survey was designed to evaluate student perceptions and attitudes towards research and how research experience may impact their future in both a cross-sectional and longitudinal cohort manner. Personal goals students achieved through their Pathway project or the overall Pathway program were also assessed in the survey. The survey and assessment tool were administered concurrently for each class (Figure 2). The aims of the study were: (1) to compare assessment tool and survey results of students who went through Pathway (class of 2013; PY4 Pathway) to those of students who did not go through the program (class of 2012; PY4 baseline); and (2) to measure added value, goals accomplished, growth in research skills and knowledge, and differences in attitude towards research among the classes of 2012, 2013, 2014, 2015 and 2016 in a cross-sectional manner by comparing student assessment results from the different classes. Students in classes 2012 through 2016 were included. Students in the class of 2013 or later who answered “no” to the question regarding completion of a Pathway project were excluded from the study, as some students in the class of 2013 had started as class of 2012 students and were not part of the inaugural Pathway course.

Figure 2.

Timeline for administration of data collection surveys and objective assessment tool (dates indicate month survey was administered to specific class of students) *Additional mentoring questions added to the class of 2014 in October 2013

Unanswered items in the objective assessment tool were counted as incorrect provided the student initiated and returned the assessment. Assessment tools not attempted by the student were excluded from data analysis. The subjective survey gathered student opinions on the influence of Pathway on their pharmacy education. Students were asked on a scale of 1 to 4 (1=strongly agree, 4=strongly disagree) if they felt research experience/Pathway helped them become independent workers/thinkers, evaluate literature, problem solve, understand the research process, increase interest in performing research in the future, help in their future careers, distinguish themselves from other students, provide an opportunity for them to present at a meeting, and evaluate the importance of research in the pharmacy profession. The 4-point scale forced students to give a negative or positive response, instead of defaulting to a neutral response. Students were also asked whether Pathway helped them achieve any goals they may have identified as a student. If they answered yes, students were asked which goals they achieved from the following list: gaining research experience, understanding the scientific process, increasing competitiveness of their curriculum vitae, presenting at a meeting, increasing interaction with faculty members, gaining a mentor, improving their grades, or “other.” Students who selected “other” were provided space in which to expand upon the goal achieved. Students were not required to respond to both the objective tool and subjective survey.

Descriptive statistics were used to report subjective survey results. Student’s t tests were used to compare individual classes to nonPathway baseline and Pathway baseline in subsequent analyses. A global Kruskal-Wallis comparison of distributions was used to compare groups to each other, followed by pairwise comparisons based on Dwass, Steel, and Critchlow-Flinger method. Using an estimated sample size of 50 Pathway students for the class of 2014 (smallest class), 43 assessments/surveys per group needed to be completed to allow for a 5% margin of error with a 90% confidence interval (p=0.05).

Data related to academic performance in the seminar course and Problem-Solving courses were collected and analyzed. Seminar presentation and course scores were analyzed with a t test. Descriptive statistics were used when appropriate. Quantitative summaries of dissemination of student research related to Pathway and residency match rates were also assessed. An additional 2-item survey regarding mentoring and ease of transition to the medical center campus was administered to the class of 2014 in the fall of 2013. Administration of the surveys, objective assessment tool, and analysis of academic performance in the seminar and Problem-Solving courses was approved by the University of Mississippi Institutional Review Board.

EVALUATION AND ASSESSMENT

Overall, the response rate was 73.5% for all administrations of the survey and assessment tool. Response rates for the objective assessment for each class were as follows: PY4 baseline, 58.4% (n=59); PY4 Pathway, 87.4% (n=76); PY3 Pathway, 95.9% (n=47); PY2 Pathway, 78.4% (n=58); PY1 Pathway, 39.8% (n=45); PY1 baseline, 96.6% (n=112). Response rates for the subjective survey for each Pathway class were as follows: PY4, 86.2%; PY3, 98%; PY2, 78.4%, and PY1, 38.9%. Class size varied drastically due to a change from a 2-year to 3-year prepharmacy curriculum resulting in different admission prerequisite courses beginning with the class of 2014.

Students who participated in Pathway scored significantly higher on the objective assessment tool compared with either the PY4 baseline or PY1 baseline groups (Table 1). The PY4 Pathway students scored higher on the objective assessment tool than PY4 students who did not participate in Pathway (p<0.0001). The highest scoring students were the PY3 students, who were in the process of data collection and analysis when the assessment was administered.

Table 1.

Results from the Objective Assessment Tool By Cohort

Student responses to the subjective survey varied significantly by class, with the PY2 and PY3 Pathway students demonstrating the highest levels of agreement in most categories (Table 2). Fifty-nine percent (n=47) of the PY4 Pathway class indicated their Pathway project helped them achieve personal goals they had as students. Individual goals achieved via Pathway (Figure 3) included the following: gaining research experience, understanding the research process, gaining a mentor, increasing faculty interaction, increasing competitiveness of their curriculum vitae, and being able to present research at a professional meeting (state and/or national).

Table 2.

Student Perceptions on Pathway’s Impact on their Education ^a

Figure 3.

Student Goals Achieved by Fourth-year Pathway Participants

The research opportunities available for students and student awareness of these opportunities greatly increased since implementation of Pathway (100% of students perform a research project of their choosing). In contrast, the survey of the class of 2012 prior to graduation indicated that 44% of students disagreed or strongly disagreed that they had an opportunity to participate in research during their time in pharmacy school (n=25; 57 PY4 baseline respondents for subjective survey).

All PY3 PBL Problem-Solving courses now include a Pathway assignment, which accounts for 5% of the overall course grade, decreasing the high-stakes nature of the Problem-Solving examination. Changes in course scores are reported in Table 3. With the classes of 2013 and 2014, 17.9% of grades changed due to the Pathway assignment. Of note, 13 students achieved the minimum passing Problem-Solving course grade with the addition of the Pathway activity as a component of the overall course score. Seminar course scores did not differ significantly between the classes of 2012 and 2013. Mean seminar course scores for the 2 classes were 90.8% and 90.4%, respectively (p=0.6345). Additionally, mean presentation scores for the 2 classes were not significantly different (91.4% and 92.0%, respectively, p=0.367).

Table 3.

Impact of Inclusion of a Pathway Requirement on the PY3 PBL Problem-Solving Course Series

During the 2012-2013 year, some students in the class of 2013 presented their research at the Mississippi Society of Health-Systems Pharmacists (MSHP) poster competition, the School of Pharmacy (SOP) Student Research Day poster session, and the American Society of Health-Systems Pharmacists (ASHP) Midyear Clinical Meeting poster session, among others. All students disseminated their results in the PY4 seminar course through presentations to students, faculty members, and area pharmacists in attendance. Both the MSHP and SOP poster sessions were created as an avenue for students to share their findings with the school and pharmacy community. In the previous 4 years, UMSOP students had presented a range of 5 to 16 posters annually at ASHP Midyear session. At the 2012 ASHP Midyear meeting, students from the UMSOP presented 20 posters, of which 12 were Pathway projects. The class of 2012 (PY4 baseline) had 18 students participate in the ASHP Resident Matching Program, with 13 students matching (72.2%). This reflects a 12.9% overall class match rate. For the class of 2013, the first class to complete Pathway, 32 students participated in the ASHP Resident Matching Program and 23 students matched (79.3%), with an overall class match rate of 26.4%.

In addition, 59% (n=22) of students in the class of 2014 who completed the supplemental survey (n=37) felt that Pathway helped them establish a mentoring relationship. Twenty-two percent (n=8) of the students in the Class of 2014 felt Pathway helped ease the transition from the liberal arts campus to the medical center campus. Pathway is one of several implementations to improve this transition period.

DISCUSSION

Overall, both objective and subjective assessments showed benefits of Pathway for students, who demonstrated increased knowledge regarding research when compared to the class of 2012 who did not participate in Pathway and PY1 students freshly entering pharmacy school. Students from PY3 Pathway performed the best of all groups in the objective assessment. This finding may be explained by the fact that PY3 students had just submitted their raw data for class credit, thus resulting in recall bias compared to the PY4 class. Students from PY1 demonstrated a large performance improvement between baseline assessment (administered early in the PY1 year) and PY1 Pathway assessment (administered in late spring of the PY1 year). Possible explanations for this improvement include completion of their Information Skills course, information learned during Pathway group meeting discussions, or sampling bias of students who completed the PY1 Pathway assessment. The PY1 Pathway assessment was administered during a Skills Laboratory lecture which did not have a corresponding laboratory exercise that week. Less than half the class attended the Skills Laboratory lecture; those who chose to not attend lecture did not have the opportunity to participate in the PY1 Pathway assessment (n=45 in PY1 Pathway vs n=112 in PY1 baseline). Students who attended the lecture in general may have had higher academic performance and the ability to correctly answer the objective assessment compared to their peers who did not attend class, artificially inflating the objective results, and possibly the subjective results, for PY1 Pathway students. Although the assessment tool is not validated, all topics assessed were covered during lectures related to Pathway.

Results from the subjective survey were more favorable than we anticipated. Many students were initially opposed to Pathway, yet the survey results demonstrated that many students agreed or strongly agreed that Pathway helped or may help in many areas. Students from the PY4 Pathway tended to agree more than PY3 or PY2 students that their Pathway project improved skills (independent worker/thinker, literature evaluation, problem solving, understanding of the research process). The extent of agreement on the helpfulness of Pathway was similar between PY4 Pathway students, who had completed their project, and PY1 Pathway students, who were just beginning their project. Similarities between PY4 Pathway and PY1 Pathway answers may be due to progress in Pathway projects (completion and hypothesis formation, respectively) at the time students were surveyed. Moreover, PY4 Pathway students were surveyed after their seminar presentation, a time of lower stress, and they may have had greater appreciation for their project upon completion. Students from the PY1 Pathway had not invested a great deal of time in their project at the time of the survey, nor did they have an outstanding Pathway-related deadline. Conversely, PY3 Pathway students were surveyed at a time of higher stress, immediately prior to the deadline for raw data submission and near the end of the semester, with final examinations approaching. Students from PY2 and PY3 Pathways had similar views. The former were surveyed immediately prior to the deadline for proposal submission. Students reported that creating and submitting the proposal was a difficult task that caused stress, which may have influenced their attitudes towards the impact of Pathway on improving their skills.

In the 2013 residency match, 2495 candidates matched for a first-year postgraduate (PGY1) residency position and 1438 did not. This national match rate was 63.4%; for PY4 students who participated in the ASHP Resident Matching Program, the rate was 79.3% (23 students) for 2013. The class of 2013 overall match rate of 26.4% was higher than the 2008-2011 national rate of 14.2% for all colleges and schools and the 16.0% rate for public colleges and schools.⁹ National graduating class match rate data per school for the classes of 2012 or 2013 are not available. Although the increase in match rates between the classes of 2012 (PY4 baseline) and 2013 (PY4 Pathway) is small, the authors feel that participation in Pathway is helpful to students electing to participate in a residency since the process of completing individual research with deadlines is generally considered an important concept.

It is possible that curricular and admission changes could have influenced our assessment. For example, the variation in class size due to a change in prepharmacy curriculum could have influenced the results. In addition to this curriculum change, a residency interest group was developed. The objective of this group was to encourage student interest in residencies and to assist in the application and interview processes. A significant majority of PY4 students who participated in the interest group reported their Pathway project was used as a topic of discussion during a residency interview, and student comments were complementary towards Pathway prior to the 2013 Match Day. While our university has had a significant increase in poster presentations at the ASHP Midyear session, so have other schools. At the 2012 Midyear meeting, more than 1400 posters were presented by students which may reflect the increasing competition for residency positions nationally.

Another major goal of Pathway is to facilitate the student transition from the liberal arts campus to the medical center campus. Having pathway advisors located on both campuses provides an opportunity for students to meet and interact with faculty members from the medical center campus in the PY1 and PY2 years and allows faculty members from the liberal arts campus to remain in touch with students when they transition to the medical center. Faculty advisors and students on both campuses have required meetings, but many meetings extend to mentoring regarding professional development. In fact, a majority of students in the class of 2014 reported a mentoring relationship with advisors. Unfortunately, only a small percent of the class experienced improvement in transition between campuses.

While the nature of advisor assignments to students may not always result in meaningful mentoring relationships, in some cases, it evolves into a mentoring relationship resulting in personal and professional growth for both students and faculty members. The number of relationships that have developed on their own accord and surpassed the advisor-advisee roles reveals the impact of Pathway on cultivating a rapport between faculty members and students. Many meetings between advisors and students extend beyond Pathway research projects to include discussions of course work, professional involvement, and personal goals, which we believe helps ease student transition between campuses.

Additional goals of the program were to positively impact Problem-Solving and seminar courses. Changes to Problem-Solving had a positive effect on course grades, which improved UMSOP on-time graduation rates. Based on survey results, students recognized that Pathway helped improve their grades. In the seminar course, students are now able to present their Pathway project, which has led to more attentiveness and thoughtful audience questions during presentations by students and faculty members, according to anecdotal reports. Students also receive a letter grade for the seminar course (previously pass/fail), which has the potential to improve the student’s grade point average. The revised seminar process has strengthened the relationship with our pharmacy community by offering attending pharmacy practitioners free continuing education credit. Having outside pharmacists present in seminars also allows for dissemination of study results. The impact on community practices as a result of this attendance is an opportunity for assessment and evaluation.

Even though the Pathway program overall has benefited students and faculty members, it requires support from the entire DPP, especially the department chair, due to the demand on faculty resources. Time must be allocated by all faculty members for advising and assisting students with their Pathway project. The projects are “student driven” but require faculty input, especially during the developmental stages. Pathway advisors also assist their students in preparation for their seminar presentation in the PY4 year. As stated earlier, use of available technology is necessary to bring all members of a “family” together for required meetings. This obligated some faculty members and students to learn how to operate equipment and acquire the appropriate software for the technology, which was provided to faculty members.

Prior to the development of Pathway, faculty members were advisors in PY4 students’ seminars. This advising structure is similar in the Pathway program. Since advisors are involved in the content of each student’s seminar presentation, preparation for assisting students in the seminar has not meaningfully changed. Most faculty members have 5-6 students per year for 4 years, with a total of up to 24 students (although most have closer to 20 students). During the PY1 year and fall of PY2 year, faculty members are generally able to meet with all the students in a group meeting, which usually lasts 30 to 60 minutes, 4 times per year. As students progress through the program, they often need one-on-one meetings with advisors; these can take 15-30 minutes each and are held 3 to 4 times per year. In a year’s time, faculty members can expect to spend an additional 15 to 20 hours meeting with students, which is relatively low in the course of a year. Faculty members spend additional time reviewing students’ proposals and IRB applications prior to and between meetings.

SUMMARY

One of the most critical outcomes of Pathway was the increase in student research exposure. Every student in the PY4 Pathway group completed a research project; 44% of students in the PY4 baseline group disagreed or strongly disagreed that they had an opportunity to participate in research while in the PharmD program. Although students in Pathway may not have an interest in a future research-based career, conducting a small research project allows them learning opportunities and a chance to grow as professionals. It also gives students completed research experience if they change their mind about postgraduate training. Importantly, it gives students the foundation for a methodical and evidence-based approach to questions they may encounter in future practice settings.

Pathway continues to evolve based on student and advisor feedback. Student mentoring, faculty advisor development, and systems for students to analyze results are targeted areas for further development. Our model demonstrates that a longitudinal research component is feasible, and students benefit from the research and advising experience.

Appendix 1. Assessment Tool and Study Survey

Assessment Tool

Part 1: Background information

• 1. Have you ever completed a research project?

  • a. Yes

  • b. No

• 2. Did you complete a Pathway research project?

  • a. Yes

  • b. No

Part 2: Assessment

• 1. What does an Institutional Review Board (IRB) do?

  • a. Evaluates any research protocol

  • b. Evaluates human research protocol

  • c. Reviews hospital policies for compliance with JCAHO

  • d. Reviews school policies regarding research

• 2. A research protocol involving administration of a study drug or placebo to children would likely undergo what kind of IRB review?

  • a. Waiver

  • b. Exempt

  • c. Expedited

  • d. Full-board

• 3. When can you start the data collection process for a study?

  • a. As soon as you formulate your hypothesis

  • b. After creation of your protocol

  • c. After submission of your protocol to the IRB

  • d. After IRB approval of your protocol

• 4. You do not need to submit a research protocol application to the IRB if your data collection process involves only a survey with no identifying data collected.

  • a. True

  • b. False

• 5. After IRB approval of your study, changes need to be made to your study design. What course of action should you take?

  • a. Make the changes to your study and carry on with your research

  • b. File a protocol amendment with the IRB regarding the proposed changes to your study

  • c. File a new IRB application for the changed study

  • d. Nothing, once you get IRB approval for a study you cannot make any changes, therefore you will have to proceed with your study without making any changes.

• 6. Which of the following are considered vulnerable populations?

  • a. Children

  • b. Mentally handicapped persons

  • c. Women

  • d. A and B only

  • e. A, B, and C

• 7. The larger the sample size, the _________ the power:

  • a. Lesser

  • b. Greater

  • c. Neither, power is not determined by sample size

• 8. A survey administered one time investigating patients’ attitudes regarding herbal supplements would be considered what type of research design?

  • a. Prospective descriptive

  • b. Prospective cohort

  • c. Retrospective cross-sectional

  • d. Randomized control trial

• 9. When designing a research project it is important to:

  • a. Generate a broad hypothesis asking many questions

  • b. Generate a directed hypothesis asking a specific question

  • c. Design a feasible project

  • d. A and C only

  • e. B and C only

• 10. A chart review regarding use of a specific medication in a particular disease state is considered what type of research?

  • a. Prospective

  • b. Retrospective

  • c. Blinded

  • d. A chart review is not considered research

Survey regarding perceptions

• 1. Which of the following best describes your postgraduation career plans?

  • a. Retail (chain) pharmacy

  • b. Independent pharmacy

  • c. Institutional (staff) pharmacy

  • d. Residency (PGY-1)

  • e. Fellowship/additional postgraduate education (MS/PhD/MPH/MD)

  • f. Other (please describe): ____________

• 2. Did you have research experience in addition to your Pathway project?

  • a. Yes

  • b. No

• 3. Did your Pathway project help you achieve any goals you may have had as a student?

  • a. Yes

  • b. No

• 4. If you answered Yes to the question above, which goals did you achieve? Circle all that apply:

  • a. Gained research experience

  • b. Improved my understanding of the scientific process

  • c. Improved competitiveness of my CV/applications for residencies, jobs, or postgraduate opportunities

  • d. Presented or will present a poster

  • e. Increased interaction with faculty members

  • f. Gained a mentor (either faculty member or other Pathway family member)

  • g. Helped improve my grade in a class

  • h. Other (please list):_______________________________________________________

The following questions should be answered using the scale provided below:

 1=strongly agree 2=agree 3=disagree 4=strongly disagree

• a. Doing a research project has helped or will help improve my ability to work and think independently.

• b. My research experience has helped or will help improve my ability to read and understand journal articles and scientific literature.

• c. My research experience has helped or will help improve my problem-solving skills.

• d. I learned or will learn about the research process from my Pathway project experience.

• e. My research experience has made or will likely make me more interested in doing research in the future.

• f. My research experience is likely to help me in my future career.

• g. My research experience distinguishes me from PharmD students at other schools.

• h. My Pathway project has allowed or will allow me to present my findings at a pharmacy meeting and/or publish in a journal.

• i. I feel research is an important part of the profession of pharmacy.