Establishing a Successful Clinical Research Program

Abstract

Clinical research (CR) is a natural corollary to clinical surgery. It gives an investigator the opportunity to critically review their results and develop new strategies. This article covers the critical factors and the important components of a successful CR program. The first and most important step is to build a dedicated research team to overcome time constraints and enable a surgical practice to make CR a priority. With the research team in place, the next step is to create a program on the basis of an original idea and new clinical hypotheses. This often comes from personal experience supported by a review of the available evidence. Randomized controlled (clinical) trials are the most stringent way of determining whether a cause–effect relationship exists between the intervention and the outcome. In the proper setting, translational research may offer additional avenues allowing clinical application of basic science discoveries.

CME Objectives: After reading this article, the reader should be able to understand the principals of research team building, to appreciate the different levels of scientific evidence, and to generate hypotheses.

Practicing surgeons are usually very driven individuals that invariably end up building a busy clinical practice. Clinical research (CR) is, in our opinion, a natural corollary to clinical surgery. In addition to academic advancement and visibility, CR gives investigators the opportunity to critically review their results and outcomes and potentially develop new strategies. It is our responsibility to encourage residents and fellows to consider getting involved in CR very early in their career.

In this article, we will cover what the authors believe are critical factors and important components of a successful CR program, based in part on the available literature, but predominantly on personal experience.

Success in CR does not come easy in the current economic environment, in striking contrast to the academic world of the 1970s and 1980s. What we consider individual and institutional factors are equally important. Investigators must be extremely motivated, especially in this time of increased demand for productivity and attention to relative value unit generation. An investigator must be organized to allocate time and resources, and to seek adequate training and mentorship to ask meaningful, realistic, and controversial questions. It is critical to understand the volume, complexity, and type of referrals available to the investigator and to start to focus early on very specific questions. Building a manageable and sustainable database can then be used to answer these questions. Investigators should also explore the available resources (cooperative group participation and membership, clinical trials, industry sponsored trials, available databases) and try to become an active participant early in their career.

At the institutional level, several components ought to be present and readily available for a young investigator to be successful. Institutional commitment for protected time, administrative support, and resources are critical factors that young surgeons should be looking for when starting a career that involves a CR component. Without time, resources, and support, it is virtually impossible to develop a successful CR program. Finally, an environment that fosters interdepartmental and intradepartmental collaboration will facilitate mentorship, a critical factor in the development of a junior investigator.

The Clinical Research Team

Depending on the available institutional resources, research infrastructure and a dedicated team may already be in place. Without proper support it is unreasonable to expect any CR to be successful. Lack of time is one of the most common reasons that surgeons give for not participating in clinical trials. The development of a research team can help overcome this barrier and enable a surgical practice to make CR a priority. The composition of a research team varies, but the goal for all is the same: alleviate the burdens of clinical trial participation. Members of the team should include CR associates (CRAs), CR nurses, non-nursing CRAs, data managers, regulatory staff, and administrative support staff. A research coordinator, who is usually broadly experienced with both CR and with the practice's operation, often oversees the team. Matching individuals with suitable tasks ensures that the program's resources are being used appropriately and that staff enjoy and remain challenged by their responsibilities. Depending on a program's organizational structure, the responsibilities assigned to each may differ greatly from one research program to another, both within and outside the United States. Tasks assigned to the staff include screening for potential study candidates, determining eligibility, coordinating the patient calendar, preparing documents for submission to Institutional Review Boards, filing amendments, submitting safety data, conducting patient education, obtaining informed consent, and assessing potential adverse events. No single individual could expect to fulfill all of these tasks. When resources permit, it is also helpful to have a manager or coordinator who supervises the program by overseeing quality assurance, staffing, budgeting, and site audits.¹ This person will work closely with the primary investigator to ensure that all investigator responsibilities are being met. Poor delegation leads to inefficient use of the program's resources and may cause staff to become dissatisfied with their duties. The CR manager can help ensure that tasks are being delegated appropriately.

Effective management requires shared commitment to excellence, mutual respect for each team member's role, and effective communication. Once staff members are trained and data management systems are implemented, this infrastructure has to be maintained and refreshed continually. Ultimately, a leader of an effective research program must acknowledge the value of each of its members while promoting a culture of teamwork and commitment to delivering high-quality care to patients.

Basic Steps to Develop a Research Program

With a research team in place, the next step is to develop a program. Taking an entrepreneurial approach is a successful mechanism when developing a CR program. Maintaining a sustainable program requires fiscal planning, much like a business. Researchers are often frustrated that per-patient reimbursement does not always cover the actual costs of conducting a trial and that reimbursement is usually given after patient enrollment. With National Cancer Institute (NCI) cooperative group trials, inadequate federal funding is a known and accepted fact that unfortunately deters smaller community-based practices from participation in cooperative group trials. A recent study determined that the average cost for each patient on a clinical trial is approximately $6,000 versus a per-patient reimbursement of only $2,000.² It is important to be aware of alternative funding mechanisms or institutional support that may be available to supplement program needs. Some physicians and their staff receive salary support from the institution to participate in CR, a great employment option for individuals dedicated to conducting trials. Physicians who are not salaried can build a similar mechanism into their practice. For example, because research requires additional time that is not reimbursed by insurance and does not typically generate revenue, it may be possible to add physician reimbursement as a cost covered by the study budget. Reimbursing physicians for their time is a reasonable study cost and helps create a research culture within the institution and institutions clearly benefit from offering a broad menu of clinical trials. If the program cannot be adequately funded by federally funded trials, adding industry trials may be an alternative. Though investigators are generally pleased by the higher reimbursement rates provided by industry, some complain that industry trials are less stimulating and provide fewer opportunities for publication. Always be selective before choosing trials and consider the question being investigated as well as patient demographics. It is important to be cognizant that if investigators open a trial that cannot accrue, they tax the program budget by wasting time and resources. Since NCI per-patient reimbursement alone is often insufficient, many additional options are available through NCI and other federal sources.³ Joining a community clinical oncology program (CCOP) is a great option for community sites dedicated to research. CCOPs benefit from having access to numerous phase I, II, and III trials and have significant autonomy. CCOPs also manage their own budgets and receive some funding before patient enrollment, unlike standard cooperative group partnerships. Becoming a CCOP requires a previous record of success. If a program is still in initial stages, consider becoming an affiliate member of a cooperative group instead. This enables a researcher to partner with a member institution and participate in all trials offered through the institution's cooperative group affiliation. In this mechanism, reimbursement is provided after patients are enrolled and is initially given to the member institution, which is then responsible for channeling funds to partner institutions. Joining the NCI Clinical Trials Support Unit is also an option worth pursuing for programs at all levels. Also, NCI has many investigator-initiated funding opportunities, including training grants and administrative supplements, all of which are listed on the NCI Web site. In addition to federal options, enhancing knowledge of funding opportunities offered through philanthropic organizations can be beneficial. From professional societies to advocacy organizations, most offer varying levels of grants, some exceeding several million dollars in annual funding. Many of the grants can be used to supplement the research one is already pursuing, such as ASCO's (American Society of Clinical Oncology) community oncology research grants.⁴ Be clear about the requirements associated with grants funded by nonprofit organizations. Most researchers find these grants helpful, but some are not applicable because of conflicts of interest or inability to meet associated requirements. Also, do not automatically dismiss grants for small amounts of funding; instead, consider realistic ways to incorporate these mechanisms into your program. Smaller grants can be useful to fund feasibility studies or pilot projects. Using supplemental funding mechanisms can greatly enhance a CR program. As federal funding availability decreases, it has become more and more competitive also to be funded through alternative mechanisms.

I Have an Idea. How Do I Formulate a Hypothesis and Pursue the Answer?

Over the last several years, medical and health professionals have begun using evidence-based medicine (EBM) in their practice integrating best available research and their clinical expertise with the specific patient clinical scenario. In the early 1970s, Cochrane⁵ criticized the lack of reliable evidence behind a plethora of health care interventions commonly accepted at the time. Rigorous evaluation of these interventions highlighted the need for an increase in evidence in medicine, planting the seed for EBM. David Sackett of McMaster University used the term “critical appraisal” to describe extracting evidence from systematically examined medical literature in the early 1980s.⁶

The actual term EBM was coined by Dr. Gordon Guyatt of McMaster University in 1990. An initial group of physicians from McMaster University joined forces with specialists from a variety of institutions to create the Evidence-Based Working Group. This group became responsible for adopting the idea of EBM and presented it in the pivotal report announcing it as a new medical paradigm: “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine.”⁷

The availability of systematic reviews, medical databases, the Cochrane Library, and evidence-based journals, for example, focusing on articles of immediate clinical use, has significantly improved research and clinical decision making. For example, in 1997, when the National Library of Medicine announced it was offering free access to the first-line web-based medical databases MEDLINE and PubMed, usage jumped 10-fold to a total of 75 million searches annually.⁸ Availability and accessibility of information have also increased with the advent of second-line databases such as the Cochrane Library, UpToDate, and Best Evidence along with EBM-related journals such as the ACP Journal Club and Evidence-Based Medicine. Furthermore, with electronic peer review and electronic publication, new evidence is rapidly available and disseminated.

The greatest difficulty for some junior investigator is how to generate a hypothesis. It is our personal conviction that the seed of a new clinical hypothesis comes from personal experience corroborated and supported by review of the available evidence.

Specific hypothesis-generating questions could arise from clinical findings, differential diagnoses, manifestations, harm, etiology, therapy, prevention, diagnostic tests, and prognosis (Table 1). However, the last two questions that need to be answered before starting a study are: why is this work needed and who would benefit from it?

Table 1. Common questions.

Etiology: Questions to identify and understand the cause of a condition or disease.

Prevention: Questions related to disease development. This involves identifying and understanding modifiable risk factors associated with the condition as well as early screening techniques and standards.

Diagnostic test: Questions related to selection and interpretation of diagnostic tests and how to confirm or exclude a diagnosis. This involves consideration of a test's specificity, sensitivity, likelihood ratios, cost, risks, and so on.

Therapy: Question related to selecting appropriate treatments, weighing the associated risk/benefits and efforts/costs.

Prognosis: Questions related to estimating the likely clinical course for a given patient over time and any complications.

In CR, studies can be classified into descriptive and analytic studies. Descriptive studies are observational studies and describe general disease characteristics. They include cross-sectional studies, surveillance studies, case reports, and case series.⁹ Analytic studies test a hypothesis about a casual relation between exposure and outcome.¹⁰ They can be observational, such as case–control and cohort studies, or controlled, such as the randomized controlled trial (RCT) (Table 2).

Table 2. Study designs.

Meta-analysis: A combination of all of the results in a systematic review using accepted statistical methodology.

Randomized controlled (clinical) trial: A prospective, analytic, experimental study that uses data generated typically in the clinical environment. A group of similar individuals are divided into two or more groups (one acting as a control and the other[s] receiving the treatment[s]) and the outcomes are compared at follow-up.

Systematic review: On the basis of a specific clinical question, an extensive literature search is conducted identifying studies of sound methodology. These studies are then reviewed, assessed, and summarized according to the predetermined criteria related to the question at hand.

Cohort study: A large population with a specific exposure or treatment is followed over time. The outcomes of this group are compared with a similar but unaffected group. These studies are observational, and they are not as reliable because the two groups may differ for reasons aside from the exposure.

Case–control study: Patients who have a specific outcome or condition are compared with those who do not. This is a retrospective approach used to identify possible exposures. These are often less reliable than RCTs and cohort studies because their findings are often correlational rather than causative.

Case series: Reports on the treatment of an individual patient are reviewed. These have no statistical validity because they use no control group for comparison.

Case report: They have a role for novel and rare presentations because no large population exists in these cases.

In observational studies, the physician in conjunction with the patient recommends the approach/treatment indicated, as it happens in routine clinical practice, and it could be considered more clinically relevant.

Even though a RCT carries the highest level of evidence, it may not be the most appropriate design for all clinical questions due to technical or ethical issues.^{10 11 12} In these situations, observational studies are the second best option to evaluate the efficacy and safety of a specific intervention. These studies often provide initial evidence useful in generating a hypothesis that can be tested further with analytic studies.¹⁰

Cohort studies, case–control studies, and case series are all different types of observational studies. Cohort and case–control studies differ from case series in that they make use of a comparison group. Case series belong to a group of descriptive studies that do not test the hypothesis of treatment efficacy. A case series follows a group of patients who have a similar diagnosis or are undergoing a certain procedure over a specific period of time.

Treatment safety and diagnostic accuracy are the principal outcomes that can be assessed fairly and reliably in a case series. In the assessment of either outcome, no control group is necessary and long-term follow-up can be obtained readily, especially with a retrospective design.

The external validity is also acceptable in case series that includes a diverse range of patients. By including patients with different characteristics and cointerventions, the study sample is more likely to be representative of the population of interest. In a RCT, however, relatively stringent inclusion criteria and selection of only those patients who wish to participate decrease the extent to which the results can be applied to common clinical practice.

When designed and properly conducted, a case series can be a sensible alternative to studies with higher levels of evidence, with the additional advantage of time and cost savings.¹³

A RCT is a study in which subjects are randomly assigned to one of two groups: one (the experimental group) receiving the intervention that is being tested, and the other (the comparison group or control) receiving an alternative (conventional) treatment. The two groups are then followed-up to see if there are any differences in outcome. The results and subsequent analysis of the trial are used to assess the effectiveness of the intervention, which is the extent to which a treatment, procedure, or service does patients more good than harm. RCTs are the most stringent way of determining whether a cause–effect relationship exists between the intervention and the outcome.¹⁴ In the hierarchy of evidence, RCT (Level 1) are followed by cohort studies (Level 2), case–control studies (Level 3), case series (Level 4), and expert opinion (Level 5).¹⁰

RCT is based on a good hypothesis formulated a priori. Having chosen a subject to research and a specific hypothesis to be tested, preparation should be thorough and is best documented in the form of a protocol that will outline the proposed methodology. An appropriate rationale for the study will follow with a relevant literature review, which is focused on any existing evidence relating to the condition or interventions to be studied. The subject to be addressed should be of clinical, technical, or translational significance to afford relevance to the study, and the hypothesis to be evaluated must contain outcomes that can be accurately measured. The subsequent study design (population sampling, randomization, applying the intervention, outcome measures, analysis, etc.) will need to be defined to permit a true evaluation of the hypothesis being tested. In practice, this will be the best compromise between what is ideal and what is practical. Writing a thorough and comprehensive protocol in the planning stages of the research project is essential. Peer review of a written protocol allows others to criticize the methodology constructively at a stage when appropriate modification is possible. Seeking advice from experienced researchers, particularly involving a local research and development support unit, or some other similar advisory center, can be very beneficial. It is far better to identify and correct errors in the protocol at the design phase than to try to adjust for them in the analysis phase. Articles rarely get rejected for publication because of inappropriate analysis, which is remediable, but rather because of design flaws. There are several steps in performing an RCT, all of which need to be considered while developing a protocol. The first is to choose an appropriate (representative) sample of the population from which to recruit. Having measured relevant baseline variables, the next task is to randomize subjects into one of two (or more) groups, and subsequently to perform the intervention as appropriate to the assignment of the subject.

Choosing the right population is crucial because poor sampling will undermine the generalizability of the study or, even worse, reduce the validity if sampling bias is introduced.¹⁵ The task begins with deciding what kind of subjects to study and how to go about recruiting them. The target population is that population to which it is intended to apply the results. It is important to set inclusion and exclusion criteria defining target populations that are appropriate to the research hypothesis. These criteria are also typically set to make the researchers' task realistic, for within the target population there must be an accessible/appropriate sample to recruit. For the findings of the study to be generalizable to the population as a whole, the sample must be representative of the population from which it is drawn. The best design is consecutive sampling from the accessible population.

If the inclusion criteria are broad, it will be easy to recruit study subjects and the findings will be generalizable to a comparatively large population. Exclusion criteria need to be defined and will include such subjects who have conditions which may contraindicate the intervention to be tested, subjects who will have difficulty complying with the required regimens, those who cannot provide informed consent, etc. Then having determined an appropriate sample to recruit, it is necessary to estimate the size of the sample required to allow the study to detect a clinically important difference between the groups being compared. After deciding on the population to be studied and the sample size required, it will now be possible to plan the appropriate amount of time (and money) required to collect the data necessary.

It will be important at the analysis stage to show that these potential confounding variables are equally distributed between the two groups; indeed, it is usual practice when reporting an RCT to demonstrate the integrity of the randomization process by showing that there is no significant difference between baseline variables (following CONSORT guidelines).¹⁶ Randomization should equally distribute any confounding variables between the two groups, although it is important to be aware that differences in confounding variables may arise through chance. It is also essential that treatment allocations are concealed from the investigator until recruitment is irrevocable, so that bias (intentional or otherwise) cannot be introduced at the stage of assigning subjects to their groups.¹⁷

Ideally, neither the study subjects nor anybody performing subsequent measurements and data collection should be aware of the study group assignment. Effective randomization will eliminate confounding by variables that exist at the time of accrual. Without effective blinding, if subject assignment is known by the investigator, bias can be introduced because extra attention may be given to the intervention group (intended or otherwise).¹⁷

Once the intervention has been applied, the groups will be followed up and various outcome measures will be performed to evaluate the effect or otherwise of that intervention. The outcome measures to be assessed should be appropriate to the research question, and must be ones that can be measured accurately and precisely. Even if it has not been possible to blind the administration of the intervention, it should be possible to design the study so that outcome measurement is performed by someone who is blinded to the original treatment assignment. Probably, the most important prerequisite for conducting an RCT is one's commitment. Commitment relates to not only being involved with respect to one's role but more importantly being committed to the question rather than finding the answer. This commitment is compounded by the fact that, in every respect, conducting a surgical trial takes more time and effort compared with running a clinical practice. A meaningful trial may take anywhere from 5 to 10 years to complete. During this time, the surgical world has moved forward, techniques have been modified, case reports may have suggested complications with a particular procedure, and so on.¹⁸ Successful surgical “trialists” are just as passionate about the research as any aspect of their clinical practice. They likely spent additional time learning how to perform research in the fields of clinical epidemiology, methodology, or public health in addition to their clinical fellowships.

A critical aspect of CR is quality control. Essentially, quality control issues occur in clinical procedures, measuring outcomes, and handling data. Ideally, any outcome measurement taken on a patient should be precise and reproducible; it should not depend on the observer who took the measurement.¹⁵

However, it is often necessary to use multiple observers, especially in multicenter trials. Inevitably, there will be a principal investigator; this person will be responsible for assuring the quality of data measurement through motivation, appropriate delegation of responsibility, and supervision. An investigators' meeting before the study starts and regular visits to the team members or centers by the principal investigator during data collection allow for communication, supervision, early detection of problems, and feedback are good for motivation.

Data will subsequently need to be transcribed onto a computer database from these forms. The database should also be set up so that it is similar in format to the forms, allowing for easy transcription of information. The database can be preprepared to accept only variables within given permissible ranges and that are consistent with previous entries, along with alerts to the user for missing values. Ideally, data should be entered in duplicate, with the database only accepting data that are concordant with the first entry; this, however, is time consuming, and it may be adequate to check randomly selected forms with a printout of the corresponding datasheet to ensure transcription error is minimal, acting appropriately if an unacceptably high number of mistakes are discovered.

In conclusion, a well-designed, methodologically sound RCT evaluating an intervention provides strong evidence of a cause–effect relationship if one exists; it is therefore powerful in changing practice to improve patient outcome, this being the ultimate goal of research on therapeutic effectiveness.

More and more emphasis has been placed on translation science (or translational research) as a very appealing clinical application of basic science discoveries. The dominant view of translation science overly emphasizes the translation of the results of “basic,” “bench,” or discovery research into clinical application through the conduct of clinical trials. We believe that translation is much more than the conduct of clinical trials to test discoveries. It begins with translating the questions that arise out of the need for knowledge in the “real world” into discovery research; translating the findings of discovery research into clinical or policy application through clinical or policy research; translating the findings of clinical or policy research into action at the clinical or policy level. Integrating these three translation gaps into a model of evidence-based health could lead to improvement of health outcomes through translating knowledge into action.¹⁹

Surgeons are the health care providers with access to tissue samples, the key link between the bench and the bedside. For that reason we, as surgeons, are often approached by bench researchers with questions and hypotheses that need clinical confirmation. For junior investigators, in our opinion, this is a very appealing and rewarding way to get involved in research, establishing relationships with potential mentors in the basic science world to obtain academic visibility. Furthermore, correlative science and translational components are often part of multidisciplinary federally funded Research Translation Core (RTC). Tissue samples are often archived in these large studies and are available to study participants with the appropriate questions, expertise and support, offering great opportunities for clinical investigators with an interest in translational research.

Conclusion

The available resources at the institutional, regional, and national level may seem overwhelming and it is easy for junior investigators to get confused, discouraged, and slowly withdraw from research. To be successful in CR, it is critical to be at the right place (adequate institutional support, resources, and protected time) at the right time (stable department, availability of knowledgeable mentors willing to collaborate and support) and ask original questions. While it is appropriate initially to follow in the footsteps of a mentor, it is very important to branch out and develop personal research directions.

There are several common mistakes that are made by young investigators. Lack of focus and specificity seems to be a very common one. Reporting large retrospective clinical reviews, while worthwhile several decades ago, does not serve any purpose in today's environment and is unlikely to be published in a quality journal. As clinical practice tends to super specialize, so should CR. Focusing early on a specific question does not limit the scope of research, but rather makes it more appealing. There are questions that may require several generations of investigators to be answered. As the surgeon develops into a more experienced researcher, several different research avenues may be present. It is even more important at this stage to remain focused on a specific topic. The most successful clinical, translational, and basic researchers are the ones who have persisted and remain faithful to their main research path.