Student Scientific Inquiry in the Core Curriculum

Abstract

Objective. To design and implement a required student-driven research program as a capstone experience in the doctor of pharmacy (PharmD) curriculum.

Design. A research proposal in the form of a competitive grant application was required for each of 65 fourth-year students in an inaugural PharmD class at Touro College of Pharmacy in New York. The focus of the proposals was on hypothesis-driven research in basic science, clinical research, health outcomes, and public health.

Assessment. Students’ research proposals were graded using a standardized grading instrument. On a post-experience survey, most students rated the overall experience positively, indicating increased confidence in their research skills. About two-thirds of faculty members were satisfied with their students’ performance, and the great majority thought the experience would be useful in the students’ careers.

Conclusion. The capstone research project was a positive experience for fourth-year students.

INTRODUCTION

Student-driven research in health sciences education has been credited with multiple educational benefits, including refining skills (eg, critical thinking, problem-solving, synthesis, integration, and application), fostering attitudes (eg, creativity, innovation, diligence), and facilitating career transition and professional success.^1-8 Moreover, exposing pharmacy students to research may increase the much-needed critical mass of pharmacist-trained researchers.^8,9 While several institutions are devoting time and energy to student research training, such efforts remain outside the mainstream in the doctor of pharmacy (PharmD) curricula.¹⁰

Professional organizations have made numerous calls for expanding research training in the PharmD curricula in the last decade. The Accreditation Council on Pharmacy Education recognizes in its accreditation standards for PharmD degree programs that research and scientific skills are essential for pharmacists,¹¹ and the American Association of Colleges of Pharmacy strongly recommends that all pharmacists entering academia conduct research during residency training.¹² The American College of Clinical Pharmacy (ACCP) Task Force on Research in the Professional Curriculum recommends research-related content for PharmD core curricula and suggests that a research experience is highly beneficial not just for students pursuing research careers but for all students.⁶ This position is consistent with the view of the American Society of Health-System Pharmacists, which regards research as a pharmacist’s obligation.¹³

In spite of numerous recommendations, few pharmacy colleges and schools have implemented the required research experiences. This is especially true among new institutions, potentially leading to a “chasm” in the scholarship of discovery.¹⁴ Although approximately half of US pharmacy colleges and schools teach some research-related coursework,¹⁰ most (75%) do not require a research project for graduation.^7,10,15

While 25% of colleges and schools reported that they required some form of research project, only 15% required a complete project.¹⁰ Most research-related content is offered as courses in biostatistics, epidemiology, critical reading of medical literature, and journal club activities.^7,10 While indisputably valuable, it is unclear whether such courses, most of which are lecture-based, have a long-lasting impact on students’ research skills, career options, and professional success. Some colleges and schools offer various additional formats, including research electives, summer camps, field experiences, internships, active-learning–based projects, and research tracks.⁶

Capstone experiences are culminating research experiences or scholarly endeavors that provide a well-defined opportunity for students to refine high-level skills and to synthesize, integrate, and apply the knowledge gained throughout their training.^16,17 Because of their positive effects on student learning, retention, and skill-formation, such experiences are considered to have a greater long-term impact on students, relative to traditional lecture courses.¹⁸ In a 2011 American National Survey of Student Engagement Report,¹⁹ culminating experiences, learning communities, and internships were rated as high-impact activities that boost student performance and improve critical thinking. Furthermore, capstone experiences and summer research programs are the only hands-on activities cited by the ACCP Task Force on Research in the Professional Curriculum as examples of successful strategies to deliver research content.⁶

Capstone experiences in general have been credited with exerting an impact on career advancement.^18,20,21 Employers of graduates in both research- and non-research-based settings are looking for employees with specific sets of skills,¹⁸including problem-solving, critical thinking, tenacity, team-work, information technology, retrieval and analysis, creativity, as well as communication skills and professionalism. If skills such as these are important, pharmacy colleges and schools must ensure that students receive adequate opportunities to develop them.

Several barriers to implementing intensive research opportunities in pharmacy and medical education have been described.^10,22 Common barriers, including curricular time, faculty time and expertise, limited resources, and administrative difficulties, have, in some cases, been considered insurmountable.¹⁰ This conundrum calls for the development of efficient and cost-effective research curriculum models.

Recognizing the need to provide a culminating experience that integrates the knowledge of various academic disciplines with student employability skills as they transition to the working world, educators at Touro College of Pharmacy, New York, decided to incorporate a student-driven research-based capstone experience as an integral element of the final year of pharmacy education. The aims of this assessment were to provide an overview of the key design and implementation features and results of the program and to describe the lessons learned about the benefits of and challenges in designing and implementing a research-based capstone experience in a college of pharmacy.

DESIGN

The rationale and the design of the capstone research experience were based on the college’s mission and vision, departmental objectives, and available evidence^23-28 regarding the design and effectiveness of student research concentrations in a required curriculum.We designed this capstone research experience around a central pedagogical philosophy (student-centered learning) and guiding principles including vision and goals, focus, format, and efficiency.

A student-centered learning philosophy was considered fertile ground for this capstone experience. To support this philosophy, students were encouraged to play an active role by nurturing their gradual empowerment to develop specific skills and facilitating their sense of autonomy and responsibility.

The goal was to develop an experience that was more than merely a platform for rehearsing previous knowledge and skills or observing research by providing an environment in which students could push the limits of knowledge, apply their enthusiasm, and generate creativity and innovation.

We required that all research across all disciplines (ie, basic sciences, clinical pharmacy, health outcomes, public health) be hypothesis-driven and employ rigorous research methodology. Thus, a student’s research project could not consist solely of nonviable activities, such as assisting a faculty member in the laboratory, writing drug monographs, participating in community volunteer work, or completing a literature review, unless formal meta-analyses were performed. The reasons for excluding these activities were that students routinely perform many of them as part of the PharmD curriculum irrespective of the capstone experience. Also, such activities were perceived by students and faculty members as “underachieving” activities when incorporated in culminating experiences at other institutions.²⁴

With these concerns and considerations in mind, we established the following criteria: the project had to be (1) hypothesis-driven research; (2) accommodate a wide variety of disciplines and areas of interest, from basic sciences to clinical research and public health; (3) amenable to consistency in grading across all disciplines; and (4) cost-effective so that equal research opportunities could be offered to all students in a resource-limited environment. A research proposal format, mirroring a competitive grant proposal, with data collection as an optional addition, was chosen to achieve these criteria. Experience at other institutions had shown that an independent research project, complete with quality data collection and analysis, was a challenging task for all students to complete on time, even when 10 to 12 months were allocated exclusively to a research experience.²⁴ Because the capstone students in this study had concomitant commitments, such as practice experiences and preparation for the North American Pharmacist Licensure Examination (NAPLEX), data collection was desirable but not required. Allowing students to gather incomplete or inadequate data would have been a disservice to the spirit and vision of the capstone experience. An additional deciding factor against the data-collection requirement, especially for patient-oriented research, was the potential for delays and hurdles in obtaining Institutional Review Board (IRB) approval for a multitude of students at multiple sites.

In contrast to an elective course, a required capstone experience would demand the availability of research resources for all students, a requirement that even well-funded universities find challenging,²² unsustainable, or impossible.¹⁰ Thus, a well-written research proposal was determined to be an optimal, cost-benefit solution, offering a maximum learning benefit with a minimum investment of resources, while preserving a consistent grading system for students across all disciplines. The Touro capstone experience was implemented in 2012 as a required, 5 credit-hour course beginning in the spring semester of the third year and ending the spring semester of the fourth year. Students were required to identify a focused research question and write a research proposal. The possible types of research included all areas specified in the ACCP Research Mission agenda⁸: basic research, clinical research, educational research, health-services research, translational research, and additional school-specific thematic areas in community and public health research.

As a culminating experience, the capstone grant proposal requirement allowed for seamless curricular integration with any field, discipline, and previous coursework. In the study institution curriculum, a required 4-credit course (60 contact hours) in research methods and biostatistics was offered in the second year of study. Literature search skills, information mastery, and drug information concepts were modules offered throughout the entire curriculum, and a medical writing skills course was offered as an elective.

At the beginning of the capstone experience, faculty members specified their respective areas of expertise and availability, and students chose their advisers using an online mentor-match program (E*Value, Advanced Informatics, Minneapolis, MN). The work took place on campus or in alternate settings, in this latter case, under dual faculty supervision. After the student-faculty advisor matching, the student identified a focused research question to pursue. At this stage, the role of the faculty advisor was to guide the student in narrowing the study question to one that was focused and answerable. This assistance was expected to consist of: (1) discussion highlighting the complexity of the problem, the need to streamline the research aims and objectives, and potential feasibility issues, and (2) guiding the student to a more focused literature search. Additionally, students had access to a select group of sample research proposals so they could learn about developing each proposal section.

The focused research question, formulated in a standardized format, served as declaration of intent/project declaration to be approved by the capstone director. The standardized format chosen was the PICO (problem/population, intervention, comparison, outcome) format, a concise, flexible and effective format recommended by the Centre for Evidence-Based Medicine, Oxford, United Kingdom²⁹ and advocated by the National Library of Medicine (PubMed Online and App Resources) and the National Network of Libraries of Medicine.^30,31 Once approved, the student developed the research proposal. The educational objectives of the capstone experience are listed in Table 1.

Table 1.

Educational Objectives of the Research Proposal Capstone Experience

The capstone experience did not require data collection to accompany the research proposal; however, if data existed, the student was encouraged to incorporate them. The capstone experience required 3 main deadlines: registration, declaration of intent, and final paper submission. All other intermediate deadlines were set and monitored by the advising faculty members.

The final proposal was submitted electronically to a repository (ie, student portfolios in the E*Value system). Additionally, the students were required to present their capstone projects in poster format during the college’s student research day. The aims of this day were to: (1) provide the environment for practicing the skill of scientific communication, and (2) facilitate interaction between students, faculty members and external guests, preceptors, community pharmacists, and others in discussing the potential and implications of student research proposals.

To enhance consistency and ensure the transparency of the capstone experience, a 35-page standard operations manual was developed. This manual contained information about the process (ie, registration, advisor selection, reporting flow, deadlines, deliverables format, potential troubleshooting), content (ie, goals, objectives, overall description, proposal writing tips, recommended readings), and an academic integrity note. The manual contained the full description of the assessment tools and rubrics, along with all survey forms. The manual was made available to students and faculty members from the beginning of the experience. The student survey instruments did not collect information about the actual scores or grades but rather were considered an evaluation of normal educational practice by our institutional IRB and thus were exempted from a formal IRB review.

As in most institutions, the level of research experience inevitably varied, and not all faculty members were formally research-trained. Among the 28 advising faculty members, 13 (46%) held a PharmD degree, 13 (46%) held a PhD degree, 2 (7%) held a dual PhD and master’s degree, and 4 (14%) held master’s degrees. More basic-sciences faculty members than clinical faculty members held a PhD degree. A series of faculty development workshops in grant writing was offered to junior faculty members prior to capstone deployment, along with initial training on standardization of procedures. The training was offered by the capstone director and involved group and individual face-to-face meetings.

EVALUATION AND ASSESSMENT

The capstone project was an individual project graded by the student’s own faculty advisor. The final proposal accounted for 80% of the final grade. Given that the objectives for the capstone research experience were SMART (specific, measurable, attainable, realistic, and timely),³² the grading instrument chosen was one that allowed a uniform and rigorous quantification of the research methodology. Thus, grading was based on a detailed, validated assessment instrument that was specifically developed for capstone research projects in biomedical sciences and showed high reliability properties.³³ The original instrument was minimally modified by removing the items referring to stylistic criteria, which were not relevant to the capstone requirements. The instrument consisted of 23 items to assess the quality of the student’s work in describing specific aims, background and significance, research design and methods/experimental procedures, summary and recommendations/discussion of anticipated results, and plan for the dissemination of results. All items were rated on a 4-point Likert scale.

Grading of the poster presentation, which was valued at 20% of the final grade, was based on an ad hoc, specifically designed rubric to capture the quality of the work on 3 main categories: scientific content of presented material (most of the rubric’s weight), visual appeal, and oral presentation of the poster. The final grade assigned was pass/fail with a threshold of at least 75% for passing. A limited number of student awards, such as certificates of excellence, books, and recognition in the school’s brochure and in the local media were also instituted.

The capstone experience was assessed by process and outcomes evaluations. The process evaluation, as a means of quality assurance, consisted of interim monthly student evaluations by faculty advisors, as well as midpoint- and final-quality survey instruments administered to students and faculty members. The interim monthly student evaluations were brief paper-based survey instruments addressing student progress, sense of accountability, professionalism, and compliance with intermediate deadlines and advisor meetings. If a student received an unsatisfactory compliance evaluation more than 2 consecutive times, the student was referred to the Office of Academic Affairs to discuss the reasons for noncompliance and to establish a remediation plan. Unsatisfactory academic performance was handled by the student’s faculty advisor and, according to the pedagogy of student-centered learning, consisted of additional meetings with the student, setting incremental goals, intermediate deadlines, and extra readings targeting the student’s specific needs.

Final outcome assessment consisted of 2 parts. The first was student performance, as assessed by the research proposal assessment instrument described above. Faculty advisors were requested to discuss but not to negotiate this research proposal assessment with the student upon completion of the project. This aspect was in line with the student-centered philosophy of involving students in the process by requiring them to reflect on their performance. The other key aspect involved final online survey instruments soliciting student and faculty perceptions and reflections on the capstone experience. The questions for students addressed their confidence level in the skills listed as educational objectives, overall confidence in their research skills (on a continuous scale from 1 to 10), perceived gain in specific skills, and perceptions on the expected utility of the capstone experience (both on a 5-point scale). A few open-ended questions were asked regarding the most important concepts learned in this experience along with suggestions for future capstone classes. No data were collected about students’ capstone grades. Questions for faculty addressed satisfaction with the quality of the capstone work of each student, student skills gained, perception on the expected utility, and evaluation of the administrative implementation of the capstone experience. The inaugural class of 2012 consisted of 65 students, of which 31% were men, 96% held a bachelor degree or higher, and 51% had plans to pursue additional training after graduation. Forty-nine students (75%) selected research projects in the field of clinical pharmacy, health outcomes and public health, while 16 students (25%) selected research projects in the basic sciences fields.

Forty-four students responded to the capstone survey instrument, yielding an overall response rate of 68%. Prior to this capstone project, students had limited exposure to research: 27 students (61%) had no research experience at all, 9 students (21%) had less than 1 year of research experience, 7 students (16%) had 1 to 2 years, and 1 student (2%) had more than 2 years. The 65 students worked on individual capstone research projects. Twenty-eight faculty members served as capstone advisors, yielding a student-to-faculty ratio of 2.3:1. Based on students’ evaluations, the inaugural capstone experience was successful. Overall student satisfaction with the capstone experience was rated as excellent or very good by 15 (34%) students, good by 15 (34%), neutral by 9 (21%), and poor or very poor by 5 (11%). Students with positive opinions tended to characterize the experience as “instructive” and as “learning through the challenge.” The 5 students who expressed overall dissatisfaction were among the 8 responders who had more than 1 year of research experience prior to their enrollment in our college, and they characterized the experience as “not challenging enough.”

In the pre- and post-experience evaluation of students’ confidence in performing the set of skills and educational objectives of the capstone project, students rated their confidence in performing all skills consistently higher in the post-experience evaluation than in the pre-experience evaluation. The rating of overall confidence in research skills increased from an average of 6.8 ± 1.4 (SD) points before the capstone experience to 7.6 ± 1.3 points after the capstone experience (two-tailed paired Student t test, p=0.04). When specifically asked to retrospectively rate their skill gain in 4 main areas, most students rated the capstone experience highly (Table 2).

Table 2.

Survey Responses of Students (N=44) Regarding Skill-Gain Achieved During Their Capstone Experience

The perceived utility of the capstone experience was high for all capstone features (Table 3). Notably, 61% to 70% of students agreed or strongly agreed that the experience was useful for their future careers, irrespective of their future work settings. Beyond the research skills, students frequently identified several benefits of the capstone experience, including learning time management skills, patience, diligence, critical-thinking skills, and problem-solving skills. Students’ opinions on requiring future students to complete a capstone project were positive, with the majority indicating they favored (71%) or highly favored (18%) keeping the capstone research project in its current or a similar alternate format; 11% felt that the capstone project should not be a graduation requirement. At the end of the capstone experience, 65% of faculty members were satisfied with their student’s performance, while 35% were neutral or expressed some level of dissatisfaction. The most common reasons for faculty dissatisfaction were students’ lack of/poor time-management skills, low compliance with the intermediate project deadlines, and insufficient medical-writing skills. Eighty-four percent of faculty members agreed or strongly agreed that the capstone experience would help the students in their future career, regardless of their workplace.

Table 3.

Survey Responses of Students (N=44) Regarding the Relevance and Applicability of Their Capstone Experience

DISCUSSION

Based on student and faculty member perspectives, our capstone experience was successful. Most (89%) of the students considered that such a capstone experience should be a requirement for graduation despite the fact that only 25% considered pursuing some form of research in their careers. This figure is higher than other estimates,^34-36 but comparisons are difficult because of the high heterogeneity among academic settings.

Several key factors contributed to our success. Our choice of a hypothesis-driven research proposal format was beneficial. Some previously published experiences on required student research have used a mix of research project formats²⁶ or have chosen a focus on pharmaceutical care.²⁷ Our focus on a grant proposal brought a unifying approach across all disciplines and allowed for uniform expectations and standardized grading. Other strengths and factors contributing to the success of our capstone experience were the curricular integration, the standardization of expectations and process (grading, PICO model, standard operations manual), the choice of a cost-efficient design, the commitment of the vast majority of the faculty members to the program, and the enthusiasm of our inaugural class of pharmacy students.

Resources and logistics burden have been cited as obstacles to the implementation of required research in some settings.¹⁰ Our experience demonstrated that a relatively long research experience embedded into a PharmD curriculum, can attain its educational objectives with minimal administrative and financial resources.

An important key factor for success was the commitment of the majority of the faculty members to support the required capstone research program and its guidelines. Deviations from the protocol were few and were a result of initial lack of understanding of the school-wide endeavor. We may not have clearly differentiated between student research as independent study and a school-wide capstone experience to the faculty members. By design, we did not require data collection, the advantage of which was elimination of pressure to gather data in a context of limited control over its quality. The disadvantage was that some students (albeit few) perceived the proposal as an incomplete and less-satisfying research experience.

Unlike a graduate program (PhD) or an elective course, requiring data for a capstone experience is a major institutional decision because it involves careful consideration of the impact on resources, logistics, IRB approvals, and continuous sites commitments, along with necessary data quality-control measures for all projects. The scholarly value and the resources involved need to be balanced, even in institutions with strong research-dedicated infrastructure and resources.²⁸ Another potential issue was the requirement that students work individually rather than in teams. In the published literature, some programs used individual projects²⁶ while others used student teams.²⁷ While teamwork skills are crucial, in both research and nonresearch settings, our goal was that the student master and take full responsibility for the research project. Moreover, given that the timeframe for the capstone experience was embedded in the practice-experiences schedule, student group-work would impose additional logistics and scheduling challenges.

Inherently, the first year in which the capstone experience was required had some limitations and revealed the need to refine features pertaining to response rate, compliance, grading, and preliminary student preparation. While less than desirable response rates are common in college and school survey instruments, several factors may have contributed to our survey response rate, such as student fatigue regarding end-of-school survey instruments and preparation for the NAPLEX.

The highly individualized nature of the capstone research experience makes student assessment difficult. By instituting an assessment tool resembling the criteria used in competitive grant applications, we achieved a consistent grading system, irrespective of the research field or institutional departments. However, by dichotomizing the final grade into pass/fail, the grades did not reflect the full range in the quality of the proposals submitted. Additionally, the most common reason for faculty member dissatisfaction was the lack of student compliance to intermediate deadlines. As such, for the next class, we are abandoning the pass/fail grading system and adopting a letter grade system, the advantage of which is twofold: professionalism and compliance will be reflected in the student’s grade, and the wide range of depth, and quality of student research proposals will be better stimulated and captured. This evolution in grading seems to be historically natural, as some universities have witnessed inherent grade inflation during the first years of implementing a required research experience.²² Because of the way grades were reported, we did not capture the actual scores on the proposal performance, which would have provided a more nuanced insight into a student’s performance. Future course offerings will need to address this issue. In our capstone program, the student’s faculty advisor graded the project. Using multiple graders might have increased the consistency of assessments, but the benefits of this approach needs to be weighed against increasing faculty workload.

The assessment of student performance in a research project is challenging.² Student perceptions are important, but they may be weak surrogates for the actual skill.³⁷ Grading rubrics, by their nature, have conceptual flaws^38,39 and do not fully capture creativity, which should be treasured in a research-minded professional. Other measures of success may include subsequent impact on career and scholarly work.² While justifiable in a PhD program, the expectation for a publishable paper is untenable in a required capstone experience, even if data are available. Most students or trainees, even in research-intensive environments, do not have a publication at the end of 12 months⁴⁰ or even after more than 1 year of research.⁴¹ Moreover, the estimated investment of faculty time per student publication is considerable and suggests a limited institutional return for investment.²⁸ To truly assess the success of a capstone experience, innovative assessment methods will be needed.

The most common reason for both faculty and student dissatisfaction in this study was the need for more time allocated to preparatory sessions at the beginning of the capstone. Thus, we are planning preparatory workshops for the beginning of the next experience, to include reviews of the second-year core course in research methods and biostatistics, workshops in advanced literature search skills, and medical writing.

Any student capstone research concentrations at institutions of higher education require approval of the project at inception, with various levels of administrative complexity, multitier levels of approval, quality control checks, and process monitoring to ensure that students make suitable progress and attain the desired educational objective.^25,26,28,33 While we instituted approval of the student’s declaration of research intent, this level of approval and monitoring was considerably lower compared with that of other universities. Depending on other design and institutional factors, this issue may deserve further considerations in the design of a required capstone experience. Adherence to protocol translates into a steady level of expectations for all projects, which, in turn, may have long-term program benefits. For example, an extensive faculty oversight of student work was credited with an estimated increase of 85% in the number of funded student research proposals at 1 medical university only 5 years after a student research program was instituted.²² Consistent oversight may be the critical ingredient for a successful capstone experience.

CONCLUSION

A school-wide pharmacy research program can successfully achieve its learning objectives in a highly favorable, cost-benefit manner. This capstone model had several strengths and unique features that contributed to its success: cross-disciplinary applicability, a hypothesis–driven research proposal, and use of standardized assessment tools. The authors hope that the findings presented here and our experience will encourage other institutions to implement and further refine a student-driven research program in the PharmD curriculum.