A Guide on Organizing a Multicenter Clinical Trial: the WRIST study group

Abstract

Multicenter clinical trials (MCCTs) are an important research tool. Planning a MCCT is a long and arduous task that requires substantial preparation time. In this guide, we discuss the steps to plan a MCCT. A pre-planning phase, which involves formulating and refining a research question and conducting pilot studies, is detailed as well as the planning phase, which involves the acquisition of funding to support the coordination and preparation of a MCCT, culminating in the submission of an R01 grant. An essential asset to planning a MCCT is the fluidity with which all collaborators work together towards a common vision. The philosophy among collaborators should be consensus and commitment and is emphasized by the development of a consensus-assisted study protocol and the recruitment of centers and co-investigators who are dedicated, collaborative and selfless in this team effort to achieve goals that cannot be reached by a single center effort.

The national healthcare agenda has been “stimulated” under the American Recovery and Reinvestment Act of 2009, with the allocation of over $1 billion dollars to support clinical research. This act funds specific challenge areas such as comparative-effectiveness and clinical research that reiterate the need to improve the quality of clinical research. Although high-quality single-center randomized controlled trials (RCTs) are considered level I evidence,^{1, 2} the results are vulnerable to biases and methodological pitfalls, which may result in conflicting or inconclusive results.^{3, 4} These results may reflect the shortcomings of single-center RCTs, such as small sample sizes resulting in low power, the lack of inclusion of minority groups, and unique practice patterns. To obtain valid conclusions it is necessary to have carefully designed multicenter clinical trials (MCCT). One primary advantage of MCCTs includes having a more heterogeneous sample of subjects. This strengthens the generalizability (external validity) of the investigation. Concerned about the disparity of women and minorities as subjects in clinical research, the NIH Revitalization Act of 1993 was mandated in an attempt to adequately represent these groups in clinical studies.⁵ In this regard, MCCTs have advantages over single-center RCTs, particularly if other centers are located in ethnically diverse geographic regions. A second advantage is having a larger sample size to provide sufficient power to detect smaller treatment effects. Third, with the concerted efforts of multiple leaders from varying backgrounds, a diverse expertise serves as a platform upon which the study protocol and conclusions are thoroughly investigated for pitfalls. In this review guide, we focus on key aspects of planning a MCCT and exemplify these steps with the current collaborative group effort of the WRIST (Wrist and Radius Injury Surgical Trial) study group.

Efforts of the WRIST study group

Distal radius fractures (DRFs) represent the second most common fracture suffered by the elderly population, after hip fractures.⁶ The incidence of distal radius fractures in the US population over the age of 65 is reported to be as high as 80 to 100 per 10,000,⁶ and in the setting of an aging American population, the economic costs of treating this injury have been estimated to range from $385 million to $535 million.⁷ Currently there are four primary treatment options for DRFs: closed reduction and casting, percutaneous pin fixation, external fixation with and without pin fixation, and internal fixation. Recent advances in internal fixation techniques, mainly the volar plates with locking screws, have made this operative treatment increasingly popular. However, treatment choices are heavily biased by surgeons’ preferences.⁸ Thus the WRIST study group (Table 1), comprised of 19 centers across North America with participation from plastic and orthopaedic hand surgeons, was assembled to initiate a MCCT to investigate this question.

Table 1.

WRIST study group institutions and investigators

Institutions	Investigators
University of Michigan (Coordinating Center)	Kevin C. Chung, MD, MS Steven C. Haase, MD Neal Chen, MD Melissa Shauver, MPH (study coordinator) H. Myra Kim, ScD (statistician)
Beth Israel Deaconess Medical Center	Tamara Rozental, MD
Brigham and Women’s Hospital	Phillip Blazar, MD
Cleveland Clinic	Jeffrey Lawton, MD
Indiana Hand Center	Greg Merrell, MD
Massachusetts General Hospital	David Ring, MD, PhD
Mayo Clinic – Owatonna	Scott Duncan, MD, MPH
Mayo Clinic – Rochester	David Dennison, MD
Mount Sinai Medical Center	Michael Hausman, MD
New York University	Steve K. Lee, MD
Stanford University	Jeffrey Yao, MD
University of Cincinnati	Peter Stern, MD
University of Colorado – Denver	Jennifer Moriatis Wolf, MD
University of Rochester	Warren Hammert, MD, DDS
University of Washington	Jeffrey Friedrich, MD
University of Western Ontario	Ruby Grewal, MD, MS Joy C. MacDermid, PhD (epidemiologist)
Wake Forest University	Andrew Koman, MD
Washington University	Doug Sammer, MD
West Penn Allegheny Health System	Thomas Hughes, MD

History of MCCTs

Historically, the first MCCT is a lesser known trial investigating a cure for the common cold. In a controversial series of publications in the 1940s, Lancet published case series and a controlled single-center trial about the promising effects of patulin, a derivative of Penicillin Patulum, against the common cold.^{9, 10} Given the scarce resources during World War II, the Therapeutic Research Corporation approached the Medical Research Council to direct a trial on a larger scale to investigate the drug’s true effectiveness before devoting substantial effort for mass-production of patulin.⁹ This effort led to the carefully conducted Patulin Clinical Trials.¹¹ Although patulin demonstrated no benefit in treating the common cold and threw doubt on the validity of the results of prior less well-designed studies, it served as a model for the better-known subsequent MCCT investigating the use of streptomycin for tuberculosis.^9,12 It is speculated that because patulin demonstrated no beneficial effects, this first MCCT has fallen into historical obscurity. Nonetheless, at least two salient features of this historical case are worth discussing. First, findings in case series and small single-center controlled trials must be investigated in sufficiently powered and properly controlled trials to validate results. Second, the success and remarkable efficiency with which the Patulin Clinical Trials were directed is underscored by the collaborative nature and cooperation of all involved.⁹ These important characteristics will be discussed individually as we elaborate upon how to plan a successful MCCT.

I. Pre-Planning Phase

Research Question

The first methodological step is the formulation of a research question. Decisions made about the research question will dictate future decisions on research design. Both clinical experience and a thorough understanding of the current state of evidence are required. To do this, many investigators will start by mastering the literature, most commonly by performing systematic literature reviews and meta-analyses, and attending conferences to comprehend current issues. These activities naturally lead to a research question. When assessing a good research question, the investigator should remember that it should be feasible to carry out, be an interesting topic, be a novel idea to confirm and/or contribute information that will further the field, be ethical to carry out, and most importantly, have relevance to the scientific community such that it may influence clinical management, health policy, or provide new ideas for research. These 5 criteria can be remembered by the mnemonic, FINER (Table 2).¹³

Table 2.

Criteria for a Good Research Question: FINER

F EASIBLE

Adequate number of subjects

Adequate technical expertise

Affordable in time and money

Manageable in scope

I NTERESTING

To the investigator

N OVEL

Confirms or refutes previous findings

Extends previous findings

Provides new findings

E THICAL

R ELEVANT

To scientific knowledge

To clinical and health policy

To future research directions

Adapted from REF 13.

Careful consideration of the research question with a thorough knowledge of the literature should naturally lead to a hypothesis.¹⁴ The WRIST study group used this method to develop the research question. Since the first description of DRF in 1814 by Abraham Colles,¹⁵ over 5,000 articles have been published related to this injury. To comb through this large volume of work, a meta-analysis was performed on the treating DRFs by external fixation or open reduction and internal fixation.⁸ At the time of this work, 5,340 citations were searched, of which only 5 prospective, randomized controlled trials were identified. Two of these RCTs were hampered by small sample sizes,^{16, 17} and all of which only compared external fixation with casting or variations of external fixation. From the 46 included articles, it was concluded that many of the treatment recommendations relied on expert opinions and were not evidence-based.⁸ This finding was in agreement with several systematic reviews performed by the Cochrane collaboration,^18-21 indicating the need for better comparative investigations. As such, the WRIST study group’s question compares the volar locking plate system to closed reduction/percutaneous pinning and external fixation ± pin fixation in treating unstable DRFs in the elderly population. As a historical parallel, the controversial, conflicting, and ultimately inconclusive published reports preceding the historical Patulin Clinical Trials set the stage for formulating an important research question for a MCCT.

Pilot Studies

A basic principle of the experimental method is that scientific knowledge progresses incrementally. The process of planning a MCCT can be thought of analogously. Pilot studies are smaller-scale studies undertaken to augment the conduct of future related investigations in a stepwise fashion.

There are two types of pilot studies: external and internal pilot studies (Figure 1, Table 3).^{22, 23} External pilot studies, also known as “feasibility studies,” are defined as “stand-alone pieces of work” because they are separate and independent of the main intended study (Figure 1).²³ These studies are typically smaller scale test-pilot runs of the planned larger scale study. Lancaster and colleagues list seven methodological objectives of an external pilot study listed in Table 3.²³ External pilot studies contrast with internal pilot studies by acting as a separate study from the intended larger-scale study. Many granting agencies require this proof of study feasibility and error margins prior to investing funds in the full trial.

Figure 1.

Types of Pilot Studies: External and Internal Pilot Studies

Table 3.

Characteristics of External and Internal Pilot Studies

EXTERNAL PILOT STUDY

Pros

Prelude to larger study of same type

Allows for trouble-shooting the mechanics of the study protocol and design on a smaller scale

Cons

Time-consuming and expensive

Data from external pilot study not used in main study

Short follow-up of patients

Parameter estimates based on smaller sample size

Objectives

Collect data for primary outcome measure to perform a sample size calculation

Examine feasibility of study protocol

Pilot data collection forms or questionnaires

Determine recruitment rate

Test feasibility of randomization method

Ensure interventions of interest are appealing to subjects

Select appropriate primary outcome measure

Train research study coordinators

INTERNAL PILOT STUDY

Pros

A part of the main study

Large sample size; more accurate sample size calculation

No increase in time or money to conduct internal pilot study

No data discarded

Cons

Potential for biased results if study design/protocol changed after internal pilot study or  if sample size adjusted on the basis of internal pilot study data that will estimate  parameters related to treatment variables

No pre-test run of study protocol

Objectives

Interim sample size calculation

Refine statistical estimates of parameters

An internal pilot study is integrated into the larger scale main study (Figure 1) and is not intended to be a test-run of study protocols or training personnel.²² Instead the objective is to recalculate required sample size calculations with realistic recruitment rates using the intended study protocol for the large-scale study. The goal is to provide more accurate sample size calculations and estimates of the time required to complete the study. The investigation then proceeds with adjusted modifications, and in the final analyses of the results, all data, including data collected during the internal pilot study, are incorporated into the dataset. Data from internal pilot studies, however, are used with caution due to the potential of introducing bias into the study, particularly if the study protocol was modified.^{22, 23}

For the WRIST study group MCCT, several external pilot studies were conducted. Data from a prospective cohort of patients were examined to (1) define outcomes of the volar locking plate system²⁴; (2) establish the responsiveness of Michigan Hand Outcomes Questionnaire (MHQ) for DRF²⁵; (3) identify predictors of functional outcomes^26,27; and (4) compare outcomes in the elderly with a younger cohort.²⁸ Conducting these five studies allowed us to better understand the condition, compare different types of interventions, select the most appropriate primary outcome measures, examine the integrity of the study protocol, and pilot test the data collection questionnaire.

II. Planning Phase

Planning Grants

Planning a MCCT requires extensive resources and coordination among the multiple centers and is costly. Grants specifically designated to finance the planning phase of a MCCT have been announced by the National Institutes of Health (e.g. NIH Clinical Trial Planning Grant Program, R34 mechanism) and Plastic Surgery Education Foundation. The purpose of these planning grants is to prepare for a phase III clinical trial (effectiveness study) over the course of 1-year, ultimately culminating in the submission of an R01 grant mechanism to fund a MCCT. A phase III clinical trial is a randomized, controlled and blinded trial with a sample size large enough to test a hypothesis about the effect of a treatment or therapy on clinical outcomes (Table 4).¹³ During this preparatory period, research teams are established, additional collaborative centers are recruited, IRB approvals at each institution obtained, and protocols and study designs, including the manual of operating procedures (MOP), finalized. The MOP is a document meant to facilitate consistency among investigators in protocol implementation and data collection across study sites. For example, to maintain consistency among surgeons, the steps for each surgical procedure are detailed in the MOP and reviewed by each participating surgeon. This step is important to reduce variability among surgeons. This document is drafted during the planning phase and must be reviewed by the Institutional Review Board and Data Safety Monitoring Board. Finally, because the strength of the research team is deemed critical by reviewers, using the recruitment phase to build a diverse research team is the key to securing a successful R01 proposal application.²⁹ Reviewers will examine the team’s expertise, their commitment as demonstrated by letters of support, and the available resources to complete the proposed research project.

Table 4.

Phases of Clinical Trials

Phase I Trial

Initial studies to determine the safety, dosage levels, tolerability, and response  to a new treatment. Typically involves a few dozen subjects.

Phase II Trial

Controlled clinical studies conducted to evaluate the efficacy of the treatment,  dosing requirements, and toxicity (i.e. short term side effects and risks).

Phase III Trial

Expanded comparative controlled and uncontrolled trials (i.e. multicenter trials)  to evaluate the effectiveness of the treatment. Typically involves hundreds to  thousands of subjects.

Phase IV Trial

These trials are also known as Post Marketing Surveillance Trials. After the treatment has been approved, phase IV involves examining additional information (e.g., drug’s risks, benefits, and optimal use). Typically involves thousands of subjects.

Protocol Development

Protocol development is the next critical step during the planning phase. Caution must be taken to avoid methodological pitfalls, which was noted to be quite common in surgical RCTs of the upper extremity.³ The expertise among members of the collaborating research teams is garnered to devote careful attention to the design of the study protocol. In addition, the specialized expertise of an epidemiologist and statistician during this planning phase is critical to avoid pitfalls in methods of data collection and methodological biases in the study design, respectively. To accomplish this, other investigators have developed a consensus-assisted protocol to reflect the philosophy of all study centers and of the general scientific community.³⁰ From the experience of the Southwest Pediatric Nephrology Study Group (SPNSG) MCCT, it has been emphasized that an important factor in facilitating a consensus is the willingness of all team members to commit, participate, and compromise on the final protocol.³¹ Problem-oriented discussions with the intent of resolution are emphasized. To achieve this type of progress, the collaborators participate in face-to-face meetings as an effective and necessary team building effort. Separate dedicated meetings specifically for the MCCT are necessary in two ways. First, it serves as an opportunity to focus entirely on the MCCT issues instead of squeezing in meeting time appended onto a national or regional meeting. Second, it allows collaborators to focus on reaching consensus in an environment of critical peer review.³¹ The moderator of these meetings must be able to command the attention and respect of the collaborators to successfully steer the group towards consensus. In addition, the project leader should also be experienced in the area of investigation and be dedicated to prevail through multiple recommendations and modifications of the project design.³¹

Consensus Building Method Overview

The word consensus is derived from the Latin terminology, cum, meaning “with” or “together with” and sentire, meaning “to think” or “feel.” Thus the word consensus expresses a collaborative effort or, etymologically, “to think together.” A consensus conference is a recognized methodology for use in areas in which conclusions are often difficult to obtain or to synthesize an agreement among experts of various disciplines.²⁹ As a decision-making process, consensus strives to be participatory, cooperative, and solution-oriented. Reaching a consensus, however, can be difficult. Three popular and well-recognized methodologies have been established to accomplish this: focus group discussion,³² nominal group technique,³³ and the Delphi method.³⁴ Below, we discuss each of these techniques used in the WRIST study group and their respective advantages and disadvantages.

Focus Group Discussion (FGD)

This round-table discussion (Figure 2A) is exploratory and interactive, involving proactive members of a group to collect a wide variety of expertise and perspectives. Focus groups provide depth over the topic and can generate rich, complex ideas. Rather than leading the members, a moderator first poses the issue to the panel and allows participants to openly discuss. The moderator then facilitates the discussion by remaining neutral and being information seeking (e.g. “Has anyone had a different experience?”), clarifying (e.g. “Can you give an example?”) and acknowledging (“I understand what you are saying.”) A wide variety of ideas can emerge. However, it runs the risk of a dominant individual leading the discussion, does not always lead to a consensus, and is easily susceptible to digressions. The qualitative results from a FGD are an excellent segue mechanism for more consensus synthesizing mechanisms, such as the nominal group technique.³²

Figure 2.

Focused Group Discussion and Nominal Group Technique

Nominal Group Technique (NGT)

The NGT is also an exploratory and interactive face-to-face meeting (Figure 2B). It is more structured than a FGD, requiring participation from all members. It is also effective by increasing the perceived satisfaction of each individual.³⁵ This methodology was first described in 1968 by Andre L. Delbecq and Andrew H. Van de Ven. It was first presented as a pilot research technique and has since been used extensively in the fields of health management,^{35, 36} social services, education, and industry.

In the NGT, each participant generates his or her thoughts in response to a specific question, which are collected and compiled in a list written on a flip-chart or overhead. These ideas are then voted and ranked to produce a graded consensual list of prioritized items. The advantage of this technique is that it aggregates individual thoughts, but allows the whole group to identify, rank, and rate the critical problems. It avoids dysfunctional dynamics and elicits participation from every member.³³ One disadvantage is that all members must come together, which may be time-consuming and costly. An alternative to the nominal group technique is the Delphi method.

The Delphi Method

Norman Dalkey originated the Delphi method, affectionately classifying it into an emerging field named “opinion technology” in the 1960s.³⁷ The Delphi method solicits and refines group judgment. It is based on two key features: (1) anonymous response to a formal survey or questionnaire from a group of experts; (2) iteration and controlled feedback via multiple rounds by an unbiased facilitator who collates all responses, until finally a consensus of the majority is reached (Figure 3). This method minimizes biasing effects of dominant members, or irrelevant communication (deemed “noise,”³⁴) and avoids group pressure towards conformity. Moreover, if members of the panel are physically distanced, it is more cost-effective than gathering all members in one physical location. A disadvantage of this method is that it is lengthy and time-consuming for the facilitator.

Figure 3.

Delphi Technique

First Consensus Meeting of the WRIST study group

On July 11, 2009 the first consensus meeting commenced at the University of Michigan for the WRIST study group. Prior to meeting, an online discussion forum was generated (www.drftrial.org) to help facilitate discussions among the many collaborators in the manner of a FGD. The online discussion forum was advantageous both time and cost-wise for physically distanced collaborators situated across North America. Ongoing communication among members via the blog helped keep everyone in touch until the consensus meeting. At the meeting, key components of the grant proposal were discussed in a FDG format. An area of rich discussion was the patient inclusion and exclusion criteria. First, using the NGT, a list of inclusion criteria were generated by each member of the study group. As described above, these criteria were listed on a large flip-pad and voted and ranked in order of importance. In this collaborative fashion, a consensus was successfully reached. This proved to be a very effective method for problem-oriented discussions.

Pitfalls

Location

Financial considerations are always important to address, given the substantial resources, time, and effort required to plan an MCCT. Some cost-cutting measures include choosing a geographic area centrally positioned as a mid-point for all collaborating centers, thereby reducing travel and time costs.³¹

Maintaining Group Commitment

Ultimately the most important asset necessary to carry out a productive MCCT investigation is collaboration and sustaining group effort. Throughout this arduous process, willingness and commitment are irreplaceable attributes. Because it is common for clinical trials to have difficulty with recruitment, investigators must be willing to continue the process often over multiple years. Strategies that maintain group commitment include continuous communication and preparatory activities such as getting early Institutional Review Board approvals and pilot testing data forms and outcome measures to assess center readiness and maintain commitment while awaiting appropriate funding.

III. Initiation of the Investigation, the Execution Phase

The successful submission of an R01 grant marks the beginning of the execution phase. Although the planning grant covered a 1-year period, this intense period was preceded by years of researching for the right research question and subsequent years of pilot testing, as described above. These rigorous methodological steps are indispensable in the planning of a MCCT (Figure 4).

Figure 4.

The Pre-planning and Planning Phases of a Multicenter Clinical Trial

Conclusion

Plastic surgery has been lagging behind in partaking in the outcomes movement currently igniting the US healthcare system.³⁸ The necessity of practicing evidence-based medicine has already been kindled among healthcare advocates, accentuating the need for level I evidence. Although conducting an MCCT may seem ambitious, in this guide, we show that the combination of careful planning combined with energy fueled by a dedicated group enables us to accomplish such goals. In fact, leaders in Plastic Surgery are currently paving the path to these meet such achievements.

The experience of the WRIST study group throughout this structured process is illuminated in this guide to planning a MCCT. The approach of a consensus-assisted protocol design serves to represent the knowledge base and experience of all participating members. The energy, enthusiasm, and unified direction of all collaborating members at the first consensus meeting speaks volumes of the potential for this project. We believe this type of infrastructure will ultimately lead to sustainability. On October 5, 2009, the WRIST study group submitted an R01 grant to the National Institute on Aging. With steadfast commitment, we aspire to generate substantial contributions to improve care for our patients with the coordination of one of the largest MCCTs in Plastic Surgery.