Successful Coordination and Execution of Non-Therapeutic Studies in a Cooperative Group Setting: Lessons Learned from Children’s Oncology Group Studies

Abstract

The immense progress made in childhood cancer survival has been due to the systematic and efficient conduct of large multicenter therapeutic trials, utilizing the infrastructure developed by national cooperative groups. These therapeutic trials have been successful, in part due to the high participation rates by the participating member institutions. However, participation in non-therapeutic trials in the cooperative group setting has lagged behind that of therapeutic trials for a variety of reasons, such as lack of institutional resources, leading to low priority given to such activities. The purpose of this report is to share some of the methods developed and successfully implemented by a Coordinating Center (City of Hope National Medical Center) to maximize institutional participation and patient enrollment and to standardize data collection and quality control, in order to ensure successful execution of two large, extramurally funded, cooperative group non-therapeutic studies. To date, over 175 institutions have obtained regulatory approval for the protocols showcased here, accrual has been on target, and completeness and quality of the collected data has been excellent. The successful execution of these non-therapeutic studies demonstrates the advantages of diverse study publicity techniques, detailed standardized operating procedures, and effective utilization of technological resources.

INTRODUCTION

For several decades, survival rates for childhood cancer have improved at a remarkable pace, such that pediatric malignancies are now curable in almost 80% of affected children (1). This triumph has occurred through the efforts of clinical investigators and laboratory scientists working in the setting of national cooperative clinical trial groups. Approximately 94% of all children with cancer diagnosed within the U.S. are treated at institutions affiliated with pediatric cooperative clinical trials groups (2). The Children’s Oncology Group (COG) was formed by merging four legacy children’s cancer cooperative groups (Children’s Cancer Group, Pediatric Oncology Group, Intergroup Rhabdomyosarcoma Group, and National Wilms Tumor Study Group) in order to accelerate progress in research and treatment for children and adolescents with cancer. COG develops and coordinates cancer clinical trials conducted at its 220 member institutions, which include pediatric cancer centers of all major universities and teaching hospitals throughout the U.S. and Canada, as well as sites in Europe, Australia, and New Zealand.

With increasing survival rates has come a heightened recognition of the need to reduce long-term sequelae and improve quality of life in childhood cancer survivors, as well as to reduce the disparities in survival rates by race and ethnicity. Several reports have demonstrated that long-term survivors are at risk for developing adverse treatment-related outcomes including early death, second neoplasms, organ dysfunction, impaired growth and development, decreased fertility, impaired intellectual function, and overall reduced quality of life (3-19). Increasing efforts are underway to identify high risk sub-populations that would benefit from primary (individualized therapies) as well as secondary (screening and chemoprevention) prevention strategies.

Furthermore, while childhood cancer survival rates have improved overall, the observed racial and ethnic differences in survival in children with acute lymphoblastic leukemia (ALL) (20, 21) have resulted in a critical need to understand the underlying cause(s) of these differences, so that directed interventional trials can focus on reducing the survival gap. The purpose of the two non-therapeutic trials under discussion here were to (a) identify populations at increased risk for late-occurring sequelae; and (b) to understand the causes of differences in survival by race and ethnicity in children treated for ALL.

Historically, research in pediatric oncology has relied heavily on participation by multiple institutions, mainly due to the rarity of pediatric cancers, making single institution studies impossible. The advantages of multi-center studies include obtaining the desired sample to answer the study questions in a limited timeframe, and improving the external validity and generalizability of the results. However, significant logistical and methodological effort and coordination is required in order to ensure that eligible study subjects are recruited and the necessary study requirements are executed at diverse clinical facilities (22). In the cooperative group setting, non-therapeutic protocols typically present unique challenges. Given limited staff and funding, member institutions may give such studies lower priority than therapeutic protocols, possibly leading to delayed attainment of institutional review board or ethics committee approval and low accrual.

The COG leadership has been tremendously supportive of the conduct of non-therapeutic trials. Efforts on the part of COG to increase accruals for these trials have included monetary reimbursement for cases enrolled on such trials across all COG institutions, and the creation of a consortium of 10 to 15 institutions to facilitate the conduct of non-therapeutic studies by providing funding to hire dedicated research staff. In addition to these methods, there is a need for coordinating centers for non-therapeutic cooperative group studies, which serve a critical but complementary role in the successful execution of these trials. The aim of the current report is to share methods utilized by a Coordinating Center (City of Hope National Medical Center) to maximize institutional participation and patient enrollment and to standardize data collection and quality control for two large, extramurally funded cooperative group non-therapeutic studies. The first study uses a case-control study design, while the second relies on a prospective, longitudinal methodology in order to test study-specific hypotheses.

MATERIALS AND METHODS

Case-Control Study: Identification of populations at increased risk for late-occurring sequelae

This study utilized a case-control design, as shown in Figure 1. Eligible cases were diagnosed with a primary cancer at age 21 or younger, who subsequently developed one of the qualifying events (late-occurring treatment-related complications). For each case, up to four controls were randomly selected by the COG Statistics and Data Center (SDC), using predefined matching criteria. All participants provided a biological specimen and completed a brief questionnaire documenting current health status. Institutions submitted clinical reports confirming the event of interest (cases), a brief questionnaire regarding the patient’s current status, and a summary of therapeutic exposures (cases and controls).

Figure 1.

Study Flow Chart – Case-Control Study

In the initial phases of the study, priority was given to securing regulatory approval at member institutions. Promotion and publicity techniques utilized by the Coordinating Center included mass e-mails, personal telephone calls to institutional principal investigators (PIs) and key research personnel, announcements in group newsletters, and presentations at group conferences. The Coordinating Center also offered to complete institutional review board or ethics committee study applications, and regularly fielded and helped create responses for review board queries. A master document of regulatory board questions fielded by the Coordinating Center was used to maintain consistency in responses and study techniques.

During the data collection phase, a ‘Frequently Asked Questions’ document was used by the Coordinating Center to provide uniform responses regarding details of study methodology. Instructional tools and documents (see Table 1) and announcements regarding new document postings, amendments, and procedural clarifications were made available by the Coordinating Center to participating institutions via the COG member-accessible website and mass e-mail in order to facilitate patient enrollment and data collection.

Table 1.

Tools and Documents to Facilitate Patient Enrollment and Data Collection

Item	Purpose/Description	Case-Control Study	Longitudinal Study
Training Module	Instructional Microsoft PowerPoint® presentation providing institutional staff with study overview, eligibility requirements, enrollment procedures, and instructions regarding all required data and biospecimen submissions	X	X
Case Identification Form/Eligibility Screening Tool	Checklist assists CRA in identifying eligible patients	X	X
Submission Requirements Checklist	Checklist delineates all study data collection and biospecimen submission requirements	X	X
Specimen Collection Kit	Specimen collection tubes, ice pack, shipping container, and instructions detailing specimen collection and shipment procedures	X	X
Dose Calculation Worksheet	Standardized instructions for calculating cumulative chemotherapy dosages for use in completing therapeutic summary form	X
Study Information Booklet	Illustrated color booklet provides abbreviated information for institutions regarding study purpose, eligibility and data collection, designed to provide an introduction to the study		X
Eligibility Flyer and Clinic Reminder	Assists clinical staff in identifying eligible patients and serves as a communication tool for obtaining and tracking required study blood specimens and questionnaires		X
Patient Tracking Worksheet	Assists CRA with tracking biospecimen and study data submission due dates by study timepoint		X

A variety of methods were used by the Coordinating Center to establish rapport with staff at participating institutions, promote study participation, and disseminate instructions for standard data collection. To assist in providing consistent and timely communication with participating institutions, the Coordinating Center team established a group email address, which was particularly helpful in situations requiring an immediate response. Periodic e-mails were sent by the Coordinating Center team to participating sites to express gratitude for institutional participation, provide updates on study progress, and target specific study goals. In addition, regular phone contact was made with participating institutions to offer individualized assistance and advice regarding study methodology. The research team at the Coordinating Center also met weekly to appraise study accrual status and to troubleshoot issues relating to patient recruitment, data collection, and institutional participation.

A comprehensive Microsoft Access^® relational tracking database was created by the Coordinating Center staff to efficiently track institutional approval status, patient enrollment, specimen and data submission, and shipment of specimen collection supplies to participating institutions. Information regarding patient enrollment and completed data collection forms were submitted by participating institutions via the HTML-based COG electronic Remote Data-Entry System (eRDEs). Data were subsequently exported periodically into the Coordinating Center tracking database. All features of this database were outlined in a comprehensive Standardized Operating Procedure (SOP), documenting all study-related tasks.

The Coordinating Center staff provided participating institutions with specimen collection kits and maintained requests for study supplies in a shipping database. Institutions were encouraged to request study supplies in advance of patient enrollment to prevent delays in obtaining specimens. For participants who were not scheduled to return to their home institution in the near future, samples were collected via a residential blood drawing service or using mailed buccal cell collection kits. Institutions requesting a residential blood draw forwarded the patient’s contact information and consent form to the Coordinating Center, which then worked with the residential blood drawing service to schedule home appointments. This service enabled those living in remote areas to participate with no cost to the patient or to their home institution. Buccal cell collection kits were mailed to patients who were not geographically accessible to the residential blood draw service (residents of Hawaii, Alaska, Canada and those residing outside of North America) as well as those unable or unwilling to give blood samples. Patients mailed the buccal cell samples directly to a centralized specimen-processing laboratory.

Parents or patients (12 years of age and older) were asked to complete brief paper-based questionnaires detailing current health habits and family medical history. Institutions were instructed to review returned questionnaires for completeness prior to submission to the Coordinating Center. Respondents were asked to clarify if they ‘Did not know’ or ‘Did not wish to reply’ to questions with missing data. Upon receipt by the Coordinating Center, questionnaires were similarly reviewed and the participating institutions were contacted for clarification regarding any incomplete or inconsistent data. Completed questionnaires were entered into a custom Microsoft Access^® database. The study SOP documented data-entry procedures and methods for handling questionable responses (e.g., responses with a range of values instead of a specific digit). In addition, the Coordinating Center database was programmed to generate warnings if attempts were made to enter values outside of expected ranges.

In order to maximize completeness of data submission, the Coordinating Center tracking database was programmed to generate automated monthly e-mails to participating institutions regarding delinquent data submission. In addition, the database was programmed to generate automated e-mails notifying sites of newly identified control patients selected to match cases. Participating institutions were asked to fax or mail clinical reports validating events of interest to the Coordinating Center; these were subsequently logged as “received” in the study database. Reports were then subjected to a multi-tiered review by a research assistant and study clinician to confirm patient eligibility. At the time of biologic specimen collection, participating institutions completed a laboratory slip documenting collection date/time and patient identifiers. Laboratory slips were faxed to the Coordinating Center and logged into the study tracking database. Records of received specimens maintained by the Coordinating Center were cross-checked against those maintained by the off-site specimen processing laboratory at regular intervals.

Prospective Longitudinal Study: Differences in survival by race/ethnicity in ALL

This study utilized a prospective, longitudinal design (Figure 2). Eligible participants were diagnosed with ALL at age 21 or younger and belonged to one of four ethnic/racial groups. Eligible patients were prospectively identified by participating COG institutions, and study participation lasted for 6 months during the maintenance/continuation phase of ALL therapy. Participants provided blood specimens at monthly intervals and completed self-administered questionnaires at five time-points. Institutions submitted therapeutic summaries and completed monthly worksheets documenting each participant’s therapy and health status. In addition, follow-up data were collected every 6 months following completion of active study participation. All relevant data and questionnaires were submitted directly to the Coordinating Center.

Figure 2.

Study Flow Chart – Prospective Longitudinal Study

As in the case-control study, a comprehensive Microsoft Access^® relational tracking database was created by the Coordinating Center to track specimen and data submission, patient enrollment, and shipment of specimen collection supplies to participating institutions. In addition, due to the prospective nature of this study, the study database was used to generate a projected study timeline for each patient based on their date of enrollment. The projected data collection due dates were then used to track upcoming patient appointments, data submission for each time point, and overdue data.

The electronically-generated study timeline was also used to facilitate patient tracking. After receiving notification of patient enrollment, an automated e-mail was sent to the enrolling institution with projected data submission due dates individualized for each patient. One week prior to a projected due date, an e-mail was sent to the institution reminding the staff of the patient’s upcoming appointment and providing a list of study procedures requiring completion on that study day. The same reminder e-mail was sent again two days prior to the projected appointment date. Participating institutions were asked to notify the Coordinating Center of delays in patient therapy so that adjustments to the projected timeline could be made. Laboratory slips and questionnaires were faxed to the Coordinating Center by the participating institutions and logged into the study tracking database in order to provide documentation of completed study procedures at each time point. Institutions were asked to immediately notify the Coordinating Center if a study appointment was missed. Missed appointments were documented in the study database. Blood specimen collection was tracked at each time point by logging faxed laboratory slips received from the participating sites into the study database and cross-checking against records maintained by the off-site specimen processing laboratory at regular intervals. During the course of the study, parents and patients (12 years of age and older) were asked to complete five questionnaires. Due to the time-sensitive nature of the questionnaires, institutions were instructed to have the patient/parent complete these within one week of a study blood draw.

In order to assist participating institutions with identification of eligible patients, the Coordinating Center obtained a de-identified list of patients enrolled on COG therapeutic protocols for ALL from the COG SDC. Information provided in this list included patient registration numbers, date of enrollment on the therapeutic protocol, protocol identification number, and enrolling institution. From this information, the Coordinating Center projected the date each patient would become eligible to participate in the study. The Coordinating Center tracking database was programmed to generate automated e-mails to participating institutions when patients were approaching eligibility for the study. Additionally, e-mails were programmed to notify institutions of delinquent data submission.

In addition to maintaining a “Frequently Asked Questions” document (see Case-control Study Materials and Methods), the Coordinating Center provided institutions with various study tools to assist with patient recruitment, patient tracking, and data collection (see Table 1). Announcements regarding new document postings, amendments, and procedural clarifications were posted via memo on the COG member-accessible website, and were also disseminated via mass e-mail.

Coordinating and Statistical Centers: Role Delineation

The Coordinating Center and the COG Statistics and Data Center have played distinct, yet complementary roles in order to ensure the success of these two studies. The COG SDC has been responsible for assistance with the creation of the electronic remote data entry (eRDE) forms and data export to the Coordinating Center; central registration of the enrolled patients; electronic validation of eligibility criteria; provision of baseline data on clinical characteristics of the enrolled patients; random selection of the controls from the COG database for the case-control study; and statistical guidance and support for both studies. On the other hand, the Coordinating Center at City of Hope has been responsible for providing assistance, advice, and study tools to COG member institutions enabling them to open the studies quickly, to identify and enroll eligible patients in a timely fashion; to ensure the timely and accurate submission of data; to ensure submission of biospecimens using appropriate packaging and shipping instructions to maintain viability of the specimens; and, for institutions with limited availability of interpreters, to administer study questionnaires to Spanish-speaking patients via phone interview. The Children’s Oncology Group, although intensely interested in non-therapeutic research, is funded exclusively to perform clinical trials. Nevertheless, the COG leadership has championed the activities of the Coordinating Center by providing access to COG centralized resources, including the SDC and the Centralized IRB (CIRB). The CIRB offers a facilitated review mechanism to expedite IRB approval and reduce administrative burden for local IRBs. Thus, both the Coordinating Center and the COG centralized infrastructure have contributed to the successful execution of the two studies described here.

RESULTS

Of a total of 220 COG member institutions, 145 institutions (66%) participated in the case-control study and a total of 136 institutions (61.8%) participated in the longitudinal study. In aggregate, a total of 177 institutions (80.5%) participated in at least one of the two studies. An additional 9 institutions (5%) began the IRB approval process and never completed it. The median time between the official opening of the study at COG and local institutional review board approval ranged from 5.6 months (0.5 to 31.4 months) for the longitudinal study to 8.1 months (0.8 months to 40 months) for the case-control study. No monetary incentive was provided to the institutions for getting studies approved at the respective sites. The costs to get the studies approved at the 177 sites were therefore those incurred as part of the funds received from the extramural grants detailed below, supporting the Coordinating Center personnel facilitating the completion of application process by the member sites. Each center received monetary incentives from the COG Operations Office for enrollment of patients – ranging from $400 per patient enrolled to the case-control study, to $1000 per patient enrolled to the longitudinal study. This incentive was offered to all COG member institutions participating in these studies for the purpose of supporting personnel costs associated with patient enrollment and data submission.

Of the institutions that participated in the studies highlighted in this manuscript (i.e., the case-control and longitudinal study), the median number of institutional therapeutic and non-therapeutic enrollments for the year 2006 were 9 (range: 0-58) and 14 (range: 0-189) respectively.

The target number of patients needed to address the study hypotheses, and the number accrued at the time of this report are presented in Table 2. Comparison is also made with three previous COG non-therapeutic studies (labeled Studies A, B, and C) that were not similarly supported, demonstrating a significant difference in accrual between studies with and without a Coordinating Center (p<0.001), and confirming the impact of a Coordinating Center on the efficiency of accrual.

Table 2.

Targeted and Actual Accrual on COG Non-therapeutic Studies, without and with Support from a Coordinating Center

	COG studies with no support from a Coordinating Center*			COG studies supported by an extramurally funded Coordinating Center*
Number	Study A	Study B	Study C	Case-Control Study	Longitudinal Study
Target	90	510	850	1725	720
Accrued§	66	206	132	961	454
Years of accrual	5 years	6 years	3 years	4 years	2.75 years
Accrual status	Closed	Closed	Closed	Open	Open

^*Reference to Coordinating Center here denotes functions described in detail in the Methodology section

^§Differences in accrual between studies with and without a Coordinating Center are significant (p<0.001)

Case-Control Study

Accrual has been higher than projected for case patients with events (Figure 3a). Completeness of data collection has been excellent, with good quality specimens procured from 93% of the patients enrolled; complete submission of data abstraction forms documenting adverse events for case patients for 97% of the patients enrolled; complete submission of patient/parent questionnaires by 96% of the patients; and complete submission of therapeutic summary forms for 94% of the patients enrolled to date. In addition, staff at many participating institutions has provided positive feedback regarding instructional tools and personal communication provided by the Coordinating Center. The study continues to enroll cases and controls to reach its target accrual.

Figure 3.

Target vs. Actual Accrual: (a) Case-Control Study, (b) Longitudinal Study

The Lance Armstrong Foundation (PI: Bhatia) has supported and the Leukemia Lymphoma Society (PI: Bhatia) is now supporting the case-control study. Specifically, the funding has supported the activities of the Coordinating Center (one full-time clinical research associate and 30% effort for a research coordinator), the processing and storage of biospecimens, and statistical support. The General Clinical Research Center grant from the National Cancer Institute (NCI: M01 RR00043, Institutional PI: J Zaia) has provided support for the specialized specimen tubes for nucleic acid collection and banking, as well as the shipping costs incurred due to the shipment of specimens from all the geographically diverse institutions to a central laboratory. The U10 CA098543 (PI: G Reaman) provided funds for statistical support at the COG SDC.

Prospective Longitudinal Study

Accrual is progressing as planned (Figure 3b). Completeness of data collection has been excellent, with 92% specimen collection success for the 7 time points of specimen collection required for each patient; complete submission of patient/parent questionnaires for 90% of the enrolled patients. Furthermore, institutional CRAs have completed therapeutic/health status worksheets for 91% of the patients, and have completed 88% of the monthly worksheets to date. The Coordinating Center continues ongoing efforts to encourage submission of the remaining delinquent materials, and to enroll additional eligible study participants to reach its target accrual.

Funding from the NCI (R01 CA096670, PI: Bhatia) is supporting the longitudinal study in its entirety. Specifically, the grant funding supports the activities of the Coordinating Center (one full-term clinical research associate and 30% effort for a research coordinator), processing of the biospecimens and genotyping, and statistical support. The General Clinical Research Center grant from the NCI (M01 RR00043, Institutional PI: J Zaia) has provided support for the specialized specimen tubes for nucleic acid collection and banking, and electronic monitoring devices for determining adherence, as well as the shipping costs incurred due to the shipment of specimens from all the geographically diverse institutions to a central laboratory. Finally, the U10 CA098543 (PI: G Reaman) provides funds for statistical support at the COG SDC.

DISCUSSION

Cooperative group studies conducted at multiple institutions provide the advantage of obtaining large samples in a timely fashion in order to produce generalizable study results. This is particularly true of studies focusing on childhood cancer, due to the small number of patients treated at individual institutions. However, the heterogeneity of the participating institutions in terms of resource availability and protocol prioritization presents unique challenges to completeness and quality of data collection when using a standard “one approach fits all” methodology. This is particularly true with non-therapeutic studies that are placed at a lower priority by many member institutions, as compared with therapeutic clinical trials. Publications regarding multi-center research studies do not generally provide a detailed description of quality-control methodology (23). However, this is a matter of great practical importance that should be borne in mind in all phases of developing any project, and even more so in the case of a multi-center project (22).

In the context of multi-center studies, the role of a Coordinating Center is particularly crucial (22),(24). This center must ensure the validity of the study results by organizing and training research staff, communicating effectively with collaborating institutions, implementing quality control measures, and performing appropriate data management. In order to be able to perform these functions, mechanisms for coordination and feedback between the Coordinating Center and the various organizational levels must be in place (25-27).

Both studies showcased here have been extremely successful in the cooperative group setting in terms of institutional participation, patient accrual, and completeness and quality of data submission. The Coordinating Center determined that publicizing the study among COG member institutions was essential to study success. Diverse methods of study publicity were necessary to effectively raise awareness of the study among COG’s varied institutional membership. While each individual technique may have had moderate success, in concert, efforts to raise awareness were successful overall. Direct, personal communication with staff at participating institutions was often invaluable in accomplishing study goals. In addition, the establishment of detailed standardized operating procedures and consistent documentation of details regarding study methodology were crucial in maintaining uniform, high-quality data collection. Likewise, incoming data were reviewed critically by Coordinating Center staff on an ongoing basis for purposes of quality control and monitoring. The creation of a tracking database equipped with multiple automated functions allowed Coordinating Center staff to track study progress effectively and efficiently, and, more importantly, this tracking occurred in real time, so that corrective actions could be instituted where necessary. Finally, establishing an effective working relationship with the staff at the COG Statistics and Data Center to ensure data flow in real time, and receiving the ongoing support of the COG leadership, has played a tremendous role in the successful execution of these studies.

In summary, we have demonstrated that there is heterogeneity in the COG member institutions, in terms of the number of patients enrolled onto therapeutic and non-therapeutic protocols, indicating varying levels of resource availability and hence allocation to non-therapeutic trials. This heterogeneity underscores the importance of support staff in facilitating the conduct of the studies, particularly for the smaller institutions that are less able to devote resources to the conduct of non-therapeutic studies. The creation and utilization of a Coordinating Center is a cost-effective mechanism to ensure that all components of large multi-institutional studies are completed efficiently and successfully. The success of these studies highlights important areas of responsibility for the Coordinating Center in multi-center trials, and its relationship with the central operations of the cooperative group. Lessons learned from this process also provide guidance for the administration of future, similar protocols, thus emphasizing the need for procuring extramural funds for staffing a Coordinating Center.