Conducting a Successful Residency Research Project

Abstract

The residency research project can be a challenging endeavor for pharmacy residents since they typically have limited experience in this area. Furthermore, as the number of accredited residency programs has increased, so has the demand for preceptors with research experience. This review is intended to assist the resident and preceptor by providing steps and guidance with conducting a successful residency research project. Items such as idea generation, proposing the right type of project, departmental review, and project management skills are discussed and guidance with writing the research protocol is provided. Items that must be addressed in every research protocol are described and a generalized protocol template is presented. In addition, the institutional review board review process is described and tips and pointers for obtaining approval are included. Finally, useful tools and resources are provided that can be used up front or throughout each phase of the research project.

The purpose of the residency research project is to provide the resident with the skills necessary to conduct and manage a major project over the course of 1 year. Additionally, it allows residents to prepare and present a major presentation at the regional level and improve their communication skills. Many college curriculums do not require pharmacy students to complete a major project prior to graduation; thus, the research project may be the resident's first experience with such a task and one of the more challenging aspects of completing a pharmacy residency program.

In 2006, over 1000 students successfully matched for an American Society of Health-System Pharmacists (ASHP) accredited residency,¹ a substantial increase from the 800 students matched in 2004 and 600 in 2001. The increasing number of pharmacy graduates seeking residency training has led to an increase in the number of residency programs. As of 2006, 853 ASHP-accredited programs existed, offering 1,900 positions, with 1,482 representing postgraduate year 1 (PGY-1) residency positions. This has translated into an increasing need for preceptors. While this need may be well matched with the number of qualified preceptors with regard to clinical activities, the volume of preceptors experienced in research may be limited, especially as outgoing residents transition into preceptor positions themselves. Additionally, preceptors with vast clinical experience may not have extensive research experience, as this may not be part of the job description for many pharmacists. One survey reported that only 46% of critical care practitioners were involved in research.²

There are many varieties of major projects that exist for residents ranging from research to “non-research” ideas. For example, a non-research idea could be the development and implementation of a new service or a project related to quality improvement. These projects could be practical for a 1-year experience. Most residency projects, however, stem from a research idea or evaluation of “best-practice” at one's institution. This review is intended to assist residents and preceptors with conducting a successful residency project with an emphasis on research (as opposed to non-research projects). Procedural steps are offered along with clinical pearls and advice based on the experience of this author. Finally, useful resources are provided that can assist the pharmacist prior to and throughout the study to complete a successful research project.

IDEA GENERATION

Idea generation is the first step in conducting a research project and often a major barrier for new practitioners (Table 1). Potential research projects could be generated from questions that arise in everyday practice or in clinical rounds. Clinicians should consider areas where existing literature or recommendations are unavailable. If literature is available, several questions must be raised to determine whether an additional study is warranted. For example, were there inconsistencies among the results of various clinical trials? Can the methodology of these studies be improved? Was a study conducted in the population in question, and if not, would it be appropriate to extrapolate the findings from that study? Clinicians could also consult with medical/surgical attending physicians or other hospital staff members for potential research ideas. This could lead to a good research idea (and project) as well as the development of new professional relationships and the potential for future collaborations. Research ideas can also be generated from information obtained within cycles of the plan-do-study-act (PDSA) model. When developing research questions, clinicians should ultimately consider: (1) Is the study feasible? (2) Is the topic interesting (to the investigator)? (3) Is the idea new and/or novel? (4) Is the study ethical? (5) Will the results be relevant?³ Finally, all research ideas should be consistent with the vision of the institution and yield information that is useful to the department, school, and/or college.

Table 1.

Steps for Conducting a Pharmacy Residency Research Project

PROPOSING THE RIGHT PROJECT

A crucial aspect of the residency research project is proposing the right type of project and it is ultimately the responsibility of the preceptor to make this determination. Most unsuccessful projects are those not suitable for a 1-year residency. While prospective, randomized, controlled trials are the best methods for answering a research question (and have the highest score in evidence-based ranking systems), they typically take several years to complete and are beyond the scope of a 1-year residency. In addition, residents may find it difficult to enroll patients given the time-sensitive nature of obtaining informed consent when they are on rotations that may not be flexible or have scheduling conflicts (eg, missing patient rounds to enroll a study patient). Therefore, priority should be given to projects where the resident can be involved with all aspects of the research study. This includes protocol development, institutional review board (IRB) submission, data collection, database development, data analysis, presentation of study results, and manuscript preparation (if required). Projects that are “handed down” from one year's resident to the next should be discouraged because they disallow exposure to all parts of a research project. If this is not possible, a special effort should be made to provide the resident with the “missing” experience by working on another project.

Given the barriers encountered in conducting a prospective trial, most residency projects should be retrospective or non-interventional (ie, observational) in nature. While this may rank lower than a randomized, controlled trial in the hierarchy of evidence-based medicine, it will yield the best learning experience for the resident in a 1-year timeframe. Even when conducting retrospective research, there are several factors that must be considered once the research question has been determined that can contribute substantially to the success of the project. The first factor is how patients will be identified. Will patient information come from a registry? ICD-9 codes? a centralized pharmacy database? Each method has its limitations, which must be carefully considered to avoid selection bias when choosing the study population. The second factor to consider is whether the required data are readily available. Are they available from a computerized database (eg, an electronic medical record) or must they be obtained from a paper-based patient record (ie, medical chart). It is important to consider not only what data are available but whether it is searchable and retrievable. A common pitfall is to assume one can obtain data that is actually not accessible. The third factor to consider is whether or not the necessary data points are clearly documented. Clinicians should keep in mind that many outcome measures may be difficult to determine in retrospect if an adequate documentation system was not in place up front. For example, early goal-directed therapy (EGDT) is a strategy for treating patients with severe sepsis/septic shock using definitive endpoints for resuscitation, hemodynamics, and oxygenation. These endpoints are to be achieved within 6 hours from the onset of severe sepsis/septic shock.⁴ Early goal-directed therapy has demonstrated improvements in survival; thus, many institutions are working to implement this intervention in their emergency departments and intensive care units. If one were to study compliance with EGDT it would be necessary to assess whether specific goals were reached within 6 hours of the onset of severe sepsis/septic shock. If the onset of severe sepsis/septic shock was not clearly recorded at the time of presentation, it would be difficult to assess these endpoints.

Finally, careful consideration should be given to the sample size required to yield meaningful results and whether that sample size can be met at the given institution. For example, it would be difficult to evaluate the relationship between duration of antimicrobial prophylaxis and clinical outcomes in patients with penetrating abdominal trauma if patients with gunshot wounds were rarely admitted at one's institution. A sample size analysis should be performed to decrease the probability of type II error occurring or inappropriately accepting the null hypothesis. If a sample size is too large it may be impractical for a 1-year research project. The actual number that may be considered impractical will vary based on the type, volume, and availability of data collected. Several computer programs exist for calculating sample size. If these are not available, alternative methods (using direct mathematical calculations) do exist.⁵

With any major project, it is important to plan in advance and the research project is no exception. Residents and preceptors should anticipate possible barriers such as the inability to access data, obtain patient lists, or accurately determine clinical outcomes. Additionally, a basic understanding of statistics is required so the data can be accurately interpreted upon collection. Residents and preceptors are encouraged to consult with statisticians if they are unsure how to analyze the data. If a statistician is not available at the resident's institution, several published resources are available.^6-11

FUNDING

Although funding opportunities specifically for pharmacy residents exist, the grant-writing process can be somewhat overwhelming for a PGY-1 resident. In addition, the entire process (from grant writing to acceptance of the grant) can be extremely time consuming and not fit with the goal of completing the project within 1 year. Potential barriers include delays in writing the grant, obtaining approval from the IRB, obtaining letters of support, review by the institution's grants administration department, and review by the funding agency. Residents and preceptors must consider alternative options if the grant is not funded and the project can not be started. While grant writing can be a useful experience for a resident, it should be done as an additional exercise independent of the residency research project.

DEPARTMENTAL REVIEW

Many residency programs have a residency advisory committee (RAC) that meets regularly throughout the year to discuss the residents' progress and the status of their program. The size and structure of the RAC will be specific to the program but generally consists of the residency program director, residency preceptors, and management. Preceptors (or residents) should forward their potential ideas/projects to the RAC, which is encouraged to provide input regarding the appropriateness of each proposed project. In this setting, an open forum should be provided for multiple preceptors/investigators to discuss the objectives of their project, the relevance to their department or institution, study design, potential barriers, and likelihood the project can be completed. This approach is intended to highlight issues that might not be foreseen originally by the individual generating the idea, correct suboptimal methodology, and potentially improve the efficiency of the research study. It also provides a forum for mentorship of new preceptors. Conflicts among RAC members should ultimately be resolved by the residency program director.

PROJECT SELECTION

Once an approved list of projects has been determined by the RAC, they should be presented to the residents. This list should include at a minimum the title of the project, the study goals, and the names of the preceptor(s). Before a project is selected, each resident is encouraged to conduct a literature review and determine whether the subject is an area of interest. Furthermore, residents should meet with their preceptor to discuss the project in detail and determine the goals and expectations of the preceptor. Preceptors should disclose future plans for publication along with the necessary requirements for authorship.¹² Residents should be informed that data collection alone does not satisfy these criteria, and in order to be considered an author, their commitment to the project is required even after they have graduated from the residency program.

ESTABLISHING A TIMELINE

Once an agreement has been reached between the resident and preceptor, the next step should be to develop a timeline. This timeline should include deadlines for the following components: literature review, protocol development, development of the data collection tool, IRB submission, completion of data collection, database development and data entry, and data analysis. Timelines should also include regularly scheduled meetings with the resident and all preceptors, which can help keep the project on track. This may be particularly useful for multisite programs where the resident may have clinical experiences at another institution. Ample time should be incorporated into these timelines to allow for the necessary revisions (which may actually be required multiple times) throughout the process as well as for presentation development. Time should also be incorporated for any unexpected barriers that may not have been predicted at the start of the process. As with most major projects, barriers almost certainly will be encountered and will be different for each research project (eg, delays in obtaining IRB approval, patient lists, or medical charts).

Preceptors should be in constant contact with the resident during the data collection phase to ensure adequate progress is being made. Otherwise, residents may be faced with trying to collect data on a large percentage of their estimated sample size with a short time remaining before their regional residency conference. The degree of procrastination tends to be greater with research projects than with other residency projects. This may be due to the fact that the final deadline (ie, presentation at the regional residency conference) is approximately 9 months from the start of the project with other residency responsibilities taking priority in the meantime (eg, medication use evaluations, daily topic discussions with preceptors, presentations to medical staff).

WRITING THE PROTOCOL

The protocol is the document that describes the steps that will be taken to answer the research question. A clear and concise protocol is pertinent to the success of any research project. In addition, a clearly written protocol can help facilitate manuscript preparation since many of the components (ie, introduction and methods) can be transcribed. All aspects of the study design and all data collection techniques should be included in the protocol. While there are no set criteria for writing a protocol, the common headings/subheadings are described below (Table 2).

Table 2.

Components of a Research Protocol

Title Page.

The title page should contain the full title of the study along with the names, full titles and affiliations of all investigators involved. In addition, contact information (ie, mailing address, e-mail address, telephone number, and fax number) for the corresponding or principle investigator should be included. Other information might include the name of the study sponsor (if applicable).

Introduction.

The introduction should describe the specific reasons for conducting the study. A brief overview of past related studies should be provided, along with their individual limitations or conflicts with the collective literature. It is important to specify why this study may be different and the impact the results might have in clinical practice. In general, introductions should be concise but comprehensive enough to justify what led to the specific research question. The introduction should conclude with a research question that is clearly stated and easy to comprehend. The introduction will either enthrall or deter the interest of a reader or reviewer.

Objectives/Purpose.

The objectives of the study should relate to the research question that was proposed previously. As such, the variables that will be measured should be clearly identified. For example, rather than stating, “the objective of this study is to compare dexmedetomidine versus propofol in surgical patients,” one should state, “the objective of this study is to compare the incidence of delirium in patients who received dexmedetomidine versus propofol following surgery.” Furthermore, the distinction should be clearly made between the primary and secondary objectives (if applicable).

Methods.

The Methods section is the most important part of the research protocol and should describe the study design, study population (which includes how patients will be identified, inclusion criteria, exclusion criteria) and study procedure. The study population and procedure must be well thought out and defined to avoid confusion or deviations once the study is underway. The study procedure should include a descriptive narrative pertaining to each step of the study process. Outcome measurements along with definitions should be provided.

Statistics.

The statistical tests that will be utilized should be noted along with a sample size analysis. Sufficient detail should be provided with the presumption that the reader would be able to verify the results themselves, given the original data set. The computer software that will be used should also be mentioned.

Data Handling and Record Keeping.

This section should describe how the data will be maintained (eg, a regulatory binder in the locked office of the PI) and if/when the de-identification tool will be destroyed. A statement regarding confidentiality could be included as well.

Miscellaneous.

Some IRBs require additional items such as a budget, data collection tool, timeline, etc. Clinicians should consult their individual IRB for specific instructions in these areas.

References.

Direct references should be provided to the original studies whenever possible. References should be numbered in the order in which they appear in the text. Since journals may vary on the format required for references, a complete listing of all authors of each publication should be provided in the protocol (as opposed to listing the first 3 authors followed by “et al”). The reference format can then be easily conformed to any journal's style.

DEVELOPING THE DATA COLLECTION TOOL

The data collection tool should be a user-friendly instrument that allows the resident to efficiently capture the necessary data. Data collection tools can be either electronic or paper based. Advantages of electronic-based instruments are that, in some instances, they can eliminate the need for transcription into an electronic database (and possibly decrease transcription errors). Disadvantages include that they require computer access, which may not be readily available. The volume of data collected should err on the side of too much rather than too little. It is easier to collect more data up front (with the realization it might not be used) than to re-collect data at a later date, especially given some of the barriers to data retrieval (eg, waiting for medical charts to be pulled, slow functioning computer databases). Nonetheless, emphasis should be placed on collecting the right data through careful planning. Residents should keep in mind that any revisions to the data collection tool after IRB submission may require resubmission and approval by the IRB before the form can be implemented. This could result in delays to project completion. Review by multiple preceptors/investigators upfront is therefore warranted.

SUBMITTING TO THE IRB

All research projects should be approved by the institutions IRB. Most IRBs have different levels of review (eg, full, expedited, or exempt) based on the type of project being proposed (Table 3).^13,14 As most pharmacy residency projects will be either observational or retrospective, they may qualify for an expedited (or non-full board) review. Clinicians should consult with their IRB to determine the appropriate steps to obtain approval.

Table 3.

Examples¹³,¹⁴ of Human Subject Investigations Requiring Full Review, Expedited Review, or Exempt from Review by Institutional Review Boards^a

Some examples adapted from Parvizi J, Tarity TD, Conner K, Smith JB. Institutional review board approval: why it matters. J Bone Joint Surg Am. 2007;89:418-26¹³

^aResearch intended for publication or for presentation outside the institution generally warrants review by an Institutional Review Board Researchers should consult their individual Institutional Review Board's to determine criteria for exempt from, expedited or full board review

Most IRBs will require all investigators to complete some form of training, which can usually be obtained through a web-based program. One example is the Collaborative Institutional Training Initiative Program.¹⁵ This course provides facts and information regarding common concepts and principles relative to human subject protection.

Upon submitting their protocols to the IRB, residents should assure their submissions are complete and meet all of the IRB requirements. The materials needed may vary based on whether or not the study requires full board or expedited approval. The Health Insurance Portability and Accountability Act (HIPAA) requires authorization for the release of protected health information and is usually included as part of the informed consent. Some studies (ie, retrospective studies) may qualify for waiver of authorization and this request must also be submitted to the IRB. If the IRB requires the data collection form to be submitted, careful inspection should occur to assure all information is de-identified.

It is this author's opinion that preceptors, and not residents, should serve as the principle investigator (PI). The PI is ultimately responsible for every aspect of the research study. They are also the individual listed for all correspondence from the IRB (which will typically continue beyond the time the resident may be present). Since many residents will be conducting a research project for the first time, they may not have the skills or knowledge base to assume the many responsibilities of the PI. The PI can transfer authority but not responsibility for the research study; thus, this role should be served by the preceptor. However, residents should not use this as a “safety net” and leave an unfinished project for the preceptor to complete once they have graduated.

DATA COLLECTION

The data collection phase can be a time-consuming process for a pharmacy resident, especially if the project entails retrospective chart review. It is helpful if the preceptor and the resident evaluate the first 1 or 2 patients' charts together so the resident can learn where to find certain information and develop strategies for being more efficient. This can also serve as a “pilot” for the data collection tool. It is pertinent to ensure there is no ambiguity in definitions or procedure. In fact, preceptors and residents should meet periodically to ensure the data are being collected properly. Deadlines should be set not only for 100% completion of the data collection, but for 25%, 50%, and 75% completion as well. It may be useful to conduct a preliminary analysis after approximately 10% of the total sample size has been collected to ensure the data collected will yield meaningful information and can be accurately analyzed. If necessary, alterations can be made (after the initial pilot or the preliminary analysis) to collect additional information sooner, rather than re-reviewing medical records for new data after all patients have been evaluated. If there is any variation to the protocol or data collection form, a revised form/proposal must be submitted to the IRB for approval.

DATA ANALYSIS

Statistical analysis should be done with both the preceptor and resident (and biostatistician if needed). It may be useful to have other investigators present as well. It is important that the data are validated before any statistical testing is performed. Residents and preceptors should search for incorrect data entries (eg, an extra 0 on a data entry), values that simply do not make sense (eg, ICU length of stay that exceeds hospital length of stay) or values that may have been miscoded (eg, mechanical ventilation coded as “no” but a length of mechanical ventilation of “5 days”).

It is this phase of the research study when residents will gain experience with using statistical tests and software. Preceptors (or statisticians) are encouraged to explain which tests are being used to analyze each piece of data and why this may be the most appropriate test.

DATA INTERPRETATION

Once statistical analysis has been completed, the results should be shared with all investigators. Residents should prepare a results summary page listing the values for each parameter evaluated along with a p value and the statistical test that was used. Preceptors should review these data for accuracy, trends, and limitations. In many cases, additional data may be required.

SHARING THE DATA

The final results of the research study should be distributed to the necessary individuals within the department and clinical relevance should be discussed. Action plans (based on the conclusions of the study) should be developed that are consistent with the PDSA model.¹⁶ Once the presentation has been created, practice sessions—first between the resident and co-investigators, and then between the resident and RAC–should be scheduled. This can provide an opportunity for useful critique on the quality/structure of the presentation from clinicians who have a “fresh look” at the project. Final recommendations can then be shared with other clinicians along with the appropriate institutional governing bodies or committees (ie, Pharmacy and Therapeutics Committee, Quality Committee, etc). It is hoped that presenting the study multiple times within the institution prior to the regional residency conference will increase the resident's comfort level with the data and help promote a polished presentation. Multiple presentations will also provide the resident with the opportunity to modify the presentation for different audiences or disciplines (eg, physicians, nurses, administrators, etc).

CONCLUSION

The residency research project may be the resident's first attempt at conducting research. The key to success is proper planning by both the resident and preceptor, along with the use of an organized approach. This review has provided a framework for the resident and preceptor, highlighting the major steps throughout the process, with the goal of accomplishing a successful residency research project.