Improving efficiency and reducing costs: Design of an adaptive, seamless, and enriched pragmatic efficacy trial of an online asthma management program

Abstract

Clinical trials are critical for medical decision-making, however, under the current paradigm, clinical trials are fraught with problems including low enrollment and high cost. Promising alternatives to increase trial efficiency and reduce costs include the use of (1) electronic initiatives that permit electronic remote data capture (EDC) for direct data collection at a site (2), electronic medical records (EMR) for patient identification and data collection, and (3) adaptive, enrichment designs with pragmatic approaches. We describe the design of a seamless, multi-site randomized Phase II/III trial to evaluate an asthma management intervention in urban adolescents with asthma. Patients are randomized, asked to access four online sessions of the intervention or control asthma management program, and are then followed for one year. The primary efficacy endpoint is self-reported asthma control as measured by the Asthma Control Test (ACT). Comparative effectiveness parametric approaches are utilized to conduct the trial in a real world setting with reduced costs. Escalated electronic initiatives are implemented for patient identification, assent, enrollment and tracking. Patient enrollment takes place during primary care visits. A centralized database with EDC is used for CRF data collection with integration of EMR data. This Phase II/III trial plans to have a total sample size of 500 patients with an interim look at the completion of Phase II (n = 250), The interim analyses include an assessment of the intervention effect, marker(s) identification and the feasibility study of EMR data as the trial CRF data collection. Patient enrollment has begun and is ongoing.

1. Introduction

Clinical trials are critical for medical decision-making; however, conducting a clinical trial is costly and labor intensive. Implementation of a randomized controlled trial (RCT) involves the establishment of specialized clinical sites and staff for recruitment of patients and delivery of the intervention. For multisite trials, a data coordinating center (DCC) is needed, with infrastructure to handle site coordination in addition to data collection, management and analysis. Although electronic medical records (EMR) have been promoted in health care reform as a front-end strategy to increase quality and efficiency of patient care, the current EMR structure is not designed for clinical research. Despite advances and much progress, challenges remain in the adoption of EMR for use in clinical practice and for RCTs. Currently, 30% to 50% of case report form (CRF) data for an RCT is extracted from medical records, while the remaining fraction is collected for research purposes outside of routine health care [1].

As mentioned above, RCTs are costly. The estimated cost to enroll a single patient at a clinical site is $1000–$1400, and includes obtaining written informed consent, documenting patient enrollment, non-routine care assessments, safety monitoring, and CRF data collection [1]. These costs do not include trial-related tests and procedures or the site and data management necessary by a DCC for a multicenter clinical trial. Contributing to the sustained high cost for conducting clinical trials is a lack of adequate and efficient study designs and initiatives for utilizing EMRs. Clearly, there is a need for more and better evidence on how to best improve the conduct of clinical trials [2–5].

Comparative effectiveness research (CER) offers options for reducing study costs and accelerating the translation of research findings to practice [4,6–9]. Features of CER include the use of pragmatic approaches which incorporate novel study designs to estimate “real world” outcome probabilities, enhance trial efficiency, and increase external validity. This is in contrast to explanatory trials in which investigators seek to maximize the possible effect of an intervention under ideal, highly controlled, optimal conditions [7].

Adaptive designs, also a feature under the umbrella of CER, allow modification of a trial under a rigorous statistical plan using the data observed after trial initiation [10]. A seamless Phase II/III design, as an example, combines the traditional II and III phases. In adaptive designs, the Phase II component is part of the Phase III trial and the treatment/intervention efficiency analysis will include all patients enrolled before and after the Phase II adaptation. This design has many advantages including a reduction in the required sample size and study duration, and a shorter, less costly implementation period. The trial may be stopped at the completion of Phase II if there is a concern about patient risk/safety, no evidence of an intervention effect (i.e., futility analysis), or a significant finding related to the study intervention. The adaptive enrichment design, another aspect of new approaches to the conduct of clinical trials, may contribute to trial efficiency [11]. This design uses a variety of methods to improve study power by identifying patients most likely to benefit from the intervention, thereby increasing the likelihood of an intervention effect. The enrichment approach also allows the use of positive biomarkers as inclusion criteria, even when those markers are not available at randomization, while still controlling for Type I errors. To our knowledge, the enrichment design has not been used in behavioral trials; however, the future of such trials is towards the application of personalized strategies or approaches that are tailored to each patient. The adaptive seamless design has become popular during the last decade due to new demands for a new and faster product and/or intervention development with more certainty and lower cost. Recently, FDA has issued draft guidance documents for both adaptive design [12] and enrichment design methods [13].

In addition to new adaptive and enriched trial designs, electronic initiatives such as the use of EMR data for patient identification and remote data capture for trial CRF completion are revolutionizing the potential for conducting cost-efficient clinical trials.

The Virtual Data Warehouse (VDW) is an example of such an initiative [14], which is currently used for collaborative research through the HMO Research Network (HMORN) of which Henry Ford Health System is a founding member [15]. The VDW is made up of a standardized database stored behind separate security firewalls at each member institution with the same variable identification (name, format, and specification) in SAS code at the patient-level. The VDW converges with census data and almost all EMR data (demographics, encounter, and pharmacy fill data) and provides an ideal electronic health record platform for addressing the newly emerging area of CER. Use of the VDW–EMR data has already been shown to reduce the costs associated with conducting CER for HMO-initiated observational studies [16], but it has not yet been used for clinical trials.

The objective of this paper is to describe the rationale and the study design of a randomized trial, using a CER approach, to evaluate a behavioral intervention directed at urban teens with asthma in response to the NHLBI RFA “Pilot Studies to Develop and Test Novel, Low-Cost Methods for the Conduct of Clinical Trials.” The intervention selected for this trial is the computer-tailored, online asthma management program known as Puff City [17,18].

Asthma continues to be a major public health problem in the US with high economic and social costs [19,20]. Vulnerable ethnic communities are disproportionately affected by asthma as demonstrated by higher morbidity and mortality for these groups [19,21]. These disparities are also observed in urban teens [19,20]. We have completed two school-based RCTs of Puff City [17,18]. Results showed reductions in self-report of symptoms and activity limitations. Puff City has not yet been evaluated as a clinical tool or one that is initiated in a clinical setting. The current paradigm for conducting such a trial would be costly with respect to patient recruitment, intervention delivery, and data collection and management. The features we describe are designed to reduce costs associated with the evaluation of Puff City in the clinical setting. We will describe the use of our EMR to (1) identify potentially eligible patients from our patient population, (2) employ a web-based randomization process, (3) conduct patient assessments, and (4) deliver the intervention. Finally, we will discuss the challenges encountered and the various options available.

2. Methods

2.1. Trial design and patient population

The trial is a pragmatic, adaptive, seamless, enriched, randomized, and controlled Phase II/III trial of the Puff City asthma management program, registered with ClinicalTrials.gov (NCT01757002). The target population is urban teens with a history of asthma that are scheduled for a primary care clinical visit. Two sites, Henry Ford Health System in Detroit, Michigan (HFHS) and Kaiser Permanente in Atlanta, Georgia (KPGA), will be enrolling patients. HFHS serves as the Coordinating Center (CC) for clinical site and data management. The University of Michigan Center for Health Communications Research (UM-CHCR), a collaborator on this project, is responsible for all technical aspects of the online assent and enrollment, delivery of the online intervention and control programs, and self-reported data collection.

The trial has two arms (see Fig. 2); Arm 1 (intervention) standard care + an online, computer-tailored asthma management intervention with behavioral assessments at each of four education sessions designed to be no less than one week apart, and followed by a booster session delivered six months from baseline; Arm 2 (control) standard care + access to existing asthma informational websites that are non-tailored and provide generic asthma education. The control arm will also have behavioral assessments at each of four sessions and at six months.

Fig. 2.

The contents of randomized clinical trial of the asthma management trial with tailored Puff City intervention.

The entire study includes 500 patients, with 250 in each arm. The initial Phase II component (n = 250 or 125 in each arm) of the trial was designed to study feasibility and to assess an early indication of effect from Puff City. At the completion of phase II component, the marker(s) associated with intervention efficacy will be identified and marker-positive status will be defined. Examples of markers (or marker-positive status) include ACT scores less than or equal to 15, at least one ED visit in the last 12 months, or current Medicaid enrollee. We will continue enrollment of another 250 using an enriched design for Phase III in patients found to be marker-positive. Patient safety will be assessed based on the entire cohort of 500 and intervention efficacy will be accessed based on a sub-group of the population (See Fig. 3).

Fig. 3.

The adaptive, seamless and enrichment design of the asthma management efficacy trial.

2.1.1. Inclusion criteria

We include teens meeting modified criteria for persistent asthma used in the Healthcare Effectiveness Data and Information Set (HEDIS) measure “Use of Appropriate Medications for People with Asthma.” [22] Teens must be aged 13–19 years at the time the following criteria are met within a 12-month period:

1. At least one emergency department visit with asthma as the principal reason for visit, OR

2. At least one acute inpatient encounter with asthma as the principal reason for visit, OR

3. At least four outpatient visits with an asthma as the principal reason for visit and at least 2 asthma medications dispensed, OR

4. At least four asthma medications dispensed in the last 12 months.

Patients will be eligible if they are visiting a primary care physician at one of the participating clinics. Minor teens (13–17 years) must be able to provide e-assent with e-consent from a parent/guardian. Adult teens (18 years or older) must be able to provide written or e-consent.

2.1.2. Exclusion criteria

Exclusion criteria include (1) patient does not meet eligibility criteria for asthma; (2) patient is currently or has previously been enrolled in an investigational asthma management/education trial, such as a previous evaluation of Puff City; (3) patient is not proficient in English (Puff City is currently only available in English); or (4) patient or caregiver is unable to complete assent/consent even with help. To accommodate caregivers with limited English proficiency, consent and e-consent forms are available in Spanish and Arabic (two major ethnic groups in Detroit/Atlanta). The HFH IRB also maintains approved copies of a written Short Form Consent in a variety of languages, for use in unanticipated patient enrollment situations that can be used for caregivers speaking other languages. All participating clinics have ready access to translation and interpreter services.

2.2. Blinding

This trial is considered “open label” (meaning patients are aware of which treatment they are receiving, i.e., Intervention options are described as “tailored” versus “existing” asthma education in the consent and assent forms). However, several attempts are considered to reduce bias and ensure study integrity including: 1) physicians and medical staff-providers and supporting medical staff will be blinded to the study treatment until the patient is randomized, and only if the patient reveals program content to the healthcare staff; 2) the study investigators and research staff will be blinded to efficacy results from the interim analysis, but they will be unblinded to safety results.

2.3. Online consent

We give parents and teens the opportunity to use an online consent (e-consent) process prior to the baseline study assessment that takes place during a regular care visit. Few studies have explored online consent, but there are benefits, including standardization of the consent experience, the establishment of an accessible audit trail, and the ability to incorporate videos, animated descriptions of procedures, and voice-over for greater clarity and understanding. Although not a component of the present study, the addition of these audio and visual elements increases patient interest and retention, with patients reporting less stress and feeling more in control with a multimedia informed consent process [23]. In the present study, the patient has a choice to either complete the consent online at home or in the clinic. We expect this process will reduce staff time needed to obtain the informed consent in a busy clinic. During the scheduled visit, clinic staff confirms patient and parent/guardian consent/assent, or initiates the consent process if the participant has not already used the e-consent.

2.4. EMR initiatives

All eligibility criteria can be determined using the EMR back-end encounter and claims data. These tools are used to identify an initial patient roster for the study, and are then updated weekly to identify new patients meeting study entry criteria. Potentially eligible patients are identified prior to a scheduled appointment using our centralized automated appointment scheduling system (EMR front-end) at each research center (HFHS, KPGA). Once a patient is determined eligible, patient eligibility status (e.g., eligibility CRF) is generated for the clinic. A letter introducing the study is mailed to the patient’s home address three weeks before the appointment if there is enough lead-time to do so. An ID and link are provided in the letter to the e-consent form, if the patient wishes to consent prior to the appointment. At the clinic, the physician/study coordinator will confirm patient eligibility and complete clinical assessments before baseline and randomization. The process also accommodates walk-ins and same-day appointments. The trial is designed to take advantage of regularly scheduled clinic visits for patient enrollment and follow-up.

2.5. Randomization

The trial was originally designed as a clustered randomized trial, with the clinic as the unit of randomization as opposed to the patient. Clustered randomization trials are usually preferred for CER due to administrative convenience, ethical considerations, enhancement of patient compliance, and reduced opportunity for trial contamination. The design and analysis of clustered randomized trials remains a challenge due to uncontrolled variation in cluster size and possible imbalance between study arms.

Prior to study initiation, we carefully re-assessed the need for using a cluster design based on a preliminary assessment at HFHS. Considerations included the following:

• Clinic size varied within cluster.

• Patient characteristics varied widely between clinics.

• Probability of contamination was low, considering the intervention is tailored and content is based on patient responses.

• Delivery of an intervention or control session is under the control of the computer.

After considering the above, we decided to use an individual randomization approach rather than the cluster design. Added benefits of this decision included the ability to stratify randomization at the patient level and reduce the sample size required without compromising statistical power (e.g., 80%).

For this trial, the unit of randomization is the patient. Four stratification variables used for randomization include (1) clinical center (HFHS or KPGA), (2) patient gender, (3) computer and Internet access at home (yes/no), and (4) baseline ACT score < 15 (yes/no). The randomization schema was generated using the Urn design [24] which provides an improvement in balance compared to the blocked randomization approach. Prior to randomization, we validated the randomization algorithm in collaboration with UM-CHCR.

2.6. Puff City intervention

Puff City is a web-based, asthma management tool designed to target urban adolescents with asthma. Puff City uses tailoring, defined as the “assessment and provision of feedback based on information that is known or hypothesized to be most relevant for each individual patient of a program”[25]. More simply, computer algorithms are used to assemble theory-driven feedback based on the user’s characteristics, beliefs, and attitudes, creating an extensive array of message permutations. This allows targeted delivery of health messages, and very personalized asthma education [26]. Puff City focuses on controller medication adherence, keeping an inhaler nearby, and smoking reduction or cessation. Patients choose which behavior they want to work on and identify the personal barriers preventing behavior change. The program also includes information on trigger avoidance, instructions on using medication delivery devices and basic asthma physiology. The program consists of a baseline survey, 4 online sessions (≥1 week apart), and a 6-month follow-up survey with a booster (see Fig. 2). For this RCT, the baseline survey and session 1 are delivered in the clinic during the clinic visit. Before leaving the clinic, patients are given a packet containing instructions on how to access the program from any computer with Internet access. The packet also contains information about computer resources in the area. Computer ownership is a variable used to stratify randomization.

Puff City content is clinically-sound and theory-driven with content based on the NHLBI Guidelines for the Diagnosis and Treatment of Asthma [27]. Behavioral theories applied in Puff City include the Transtheoretical Model, the Health Belief Model [28], and aspects of Motivational Interviewing (MI). We assess and derive “scores” for asthma self-regulation using the Asthma Self-Regulation Development Interview [29] developed by Zimmerman et al., which describes sequential “phases” of asthma self-regulation including asthma symptom avoidance, asthma acceptance, asthma compliance, and asthma self-regulation [29]. Submodules for Puff City were created in order to apply a more intense tailoring strategy to “resistant adolescents”, defined as adolescents exhibiting no positive change in behavior after the first of four consecutive sessions. Values-based strategies to reveal dissonance between the behaviors and values reported by students are used to address resistance, as are MI concepts designed to help teens overcome ambivalence to change through empathy and support, as opposed to directives and emotional pulls [30,31]. In addition to the four weekly sessions, a booster session at six months was created to sustain positive change and correct early stages of relapse. Theories applied in the booster include the Self-Determination Theory and Attribution Theory and Marlatt’s Theory of Relapse which address resistance to change, relapse (defined as the abandonment of an abstinence goal) [32], and an individual’s causal attributions (or explanations) for failure to maintain positive change [33].

2.7. Patient follow-up

All patients will be followed for 12 months with assessments at 6 and 12 months post-baseline. Our follow-up schedule is designed to coincide with the expected patient visit timeframe in routine care. Based on preliminary data at HFHS, asthma patients typically have a follow-up visit every 3–6 months. For purposes of the study, a wider study follow-up interval is permissible with windows around each time point (e.g., +/−3 m) to accommodate this scheduling as a means of reducing the need for study-related visits and concomitant study costs. Participating patients will receive reimbursement or co-pays for clinical visits solely related to the study if data cannot be collected during the regular care visit.

All patients will be followed for safety and the potential risks are minimal because the study involves no invasive medical procedures. Psychological, social, or legal risks of this study are also minimal. The caregiver will be told that they have the alternative of not participating and that refusal to do so will not affect their teenager’s future health care at Henry Ford Health System or Kaiser Permanente. Nevertheless, any adverse reaction will be collected up to 6 months at the end of study intervention. A central review system is established to determine the risks related to the study intervention.

To assess the cost of conducting the trial, we are collecting data (eData) on clinic staff time spent implementing certain aspects of the study including obtaining patient informed consent in the clinic, patient safety monitoring, abstraction of medical records data onto the trial specified CRF at the clinical site, and research-related patient enrollment and assessments apart from usual care. Clinics are paid an incentive for participating. A payment process is established based on patient enrollment and follow-up status, as well as data accuracy and completeness.

2.8. Data collection and analysis

Data are collected on case report forms (CRFs) including patient baseline characteristics, and asthma medications, patient asthma management behaviors, smoking status, environmental exposures (eg environmental tobacco smoke) and the study endpoints. The baseline characteristics include demographic information (age, gender and race), symptom frequency and healthcare utilization (symptoms days and nights, previous ED visits), as well as socioeconomic data from patient and/or caregiver (e.g., computer ownership, insurance status, median household income) and quality of life using the Juniper Pediatric Asthma Quality of Life Questionnaire [34]. Any asthma medications from routine care visits up to 6 months are used to tailor the study intervention. Endpoints are collected based on routine clinical care and patient-reported outcomes, as recommended in CER initiatives. The primary endpoint of the trial is asthma control test (ACT) at one-year post randomization. The ACT, a patient self-reported survey tool consists of 5 questions with a total score ranging from 0–25 (where 0 is uncontrolled, has been evaluated and found to have high internal consistency and reliability [30]. The ACT is already implemented as part of routine care as EMR in the Division of Allergy and Clinical Immunology at HFHS. The secondary endpoints are (S1) asthma exacerbations occurring over the 12-month follow-up period (asthma hospitalizations, emergency department visits, or oral corticosteroid dispensing), and ePROs, (S2) symptom-days, (S3) symptom-nights, (S4) days of restricted activity, and (S5) school/work days missed, in the past 30 days at one-year post randomization. The number of asthma-related emergency department visits and hospitalizations within a 12 month period has been used as a clinical measure of asthma control and, according to EP3, is an indicator of the risk of a future acute event [27]. These secondary endpoints were selected based on their relevance to the healthcare decisions being made by patients, providers, and payers as part of adopting a pragmatic approach [7,35], and have been used in previous asthma studies [36].

Data will be evaluated for normality. Data transformation or a nonparametric approach will be considered if data are not normally distributed. Chi-square tests will be used to compare the proportional difference between the two study groups. Propensity scores will be incorporated into the analysis if there are imbalances on patient characteristics between two study groups.

A random effects model or generalized estimating equation (GEE) will be used to test the intervention effect on ACT improvement at 12 months (P1). We will use intention-to-treat (ITT) for P1 and adjust for stratification variables and/or use propensity scores. For the secondary endpoint (S1, asthma exacerbations as a count endpoint), Quasi-likelihood through Log link function GEE will be used to obtain a robust estimation with a less restriction on the underlying passion distribution. Global testing using GEE will be used to study asthma-morbidity reduction as measured from a set of patient-reported outcomes (S2–S5 as the symptom-days, symptom-nights, days of restricted activity, and school/work days missed), followed by testing the intervention effect on each individual outcome [37–39].

The incidence of ED visits and/or hospitalization after study intervention will be collected at Year One (YR1). Chi-square tests will be used to compare the proportional difference between the two study groups.

2.9. Interim analyses and sample size calculation

We plan to enroll 500 patients for the entire trial. The adaptive design permits a first interim look after the first 142 patients have completed the YR1 endpoint and a second interim look at the completion of Phase II when YR1 endpoint data collection has occurred for a total of 250 patients. (See Fig. 3) The second interim analysis is proposed because that is the number of patients to be enrolled under the current NHLBI funding (1R01HL114981-01).

Adequate characterization of the test for marker-positive status is critical (for example, the cut-off-point for baseline ACT score or frequency of ED visits). One difficulty is that relationships between the marker patterns and outcomes are often found only after trial results are known. Due to the post-facto nature of the finding, markers discovered this way should be confirmed in a prospective enrichment study [40]. In applying the adaptive enrichment approach to this study, with marker identification in a targeted subpopulation among all randomized subjects, proposed by Fredlin and Simon [41,42], the first half of the study population (at the completion of Phase II) will be used to identify marker-positive patients. The entire population before and after Phase II will be used to test for overall intervention efficacy using the criterion α = 0.04. The second half of the study population is then used to test for intervention efficacy among the marker-positive patients using more strict criteria (e.g., α = 0.01).

Our sample size is based on existing literature and our previous analyses. The observed effect sizes from previous school-based trials were large in a range of 0.36 to 0.54 for days of functional limitation and 0.33 to 0.47 for quality of life based on teens that had a record of at least one asthma encounter. In adult asthmatic patients, a 3-point difference in mean/median of ACT difference between two groups with the effect sizes in a range of 0.7 to 1.0 is considered to be minimally clinical important difference [43]. In a study conducted among urban teens with low SES, an ACT mean (SD) of 21.1 (3.3) was observed in an untreated group [44]. With alpha as 0.05 or 0.04, two sided testing, and an effect size = 0.3, which is a relatively small effect for a behavioral trial [45] and, we need an independent sample size of 354 (177 per group) to detect the effect with a power of 80%. The O’Brien-Fleming spending function [46,47] is used to control Type I errors. With a total of 354 patients (177 per group), and two interim unequal looks at the first 142 (71 per group) and 250 (125 per group in the Phase II component of the trial),we will have the 80% power to detect a significant intervention effect if the observed test statistic is ±3.3569 or p-value < 0.0008 at first interim look, ±2.4341 or p-value < 0.0149 at the second interim look, or ±2.0017 or p-value < 0.0453 at the final analysis. In addition, assuming 200 patients and 80% marker-positive in the second half of the study population (n = 250),we are able to retain 80% of the power to detect an effect size of 0.48 on the marker-positive patients, and will have more power (over 95%) over if the observed effect size is 0.79, 3-point difference in mean/median of ACT scores between the two groups [43].

2.10. Cost and cost-effectiveness analysis

In preparation for assessing intervention efficacy, we are collecting the data that will be used to compare the cost of care and to conduct a cost effectiveness analysis. This analysis will be completed at the end of the Phase III trial only if the intervention is efficacious. The data used to measure direct costs of asthma care include hospital care costs, outpatient and ED costs, medication costs, and costs associated with follow-up treatment and complications. The primary source for all health resource utilization and costs will be EMR encounter data. Patients will be asked to complete an online follow-up survey at 6 and 12 months post-baseline, which will provide information on secondary care services (specialty, acute or urgent care for asthma), and indirect costs, such as time lost from work/school because of asthma. The unit costs and their sources will be used to estimate total cost per patient.

To measure intervention costs, we will include a fixed monthly cost of hosting the intervention and the cost of personnel time for reminding the patients to complete the intervention sessions. The incremental computing costs per user are expected to be negligible. The time spent solely on trial data collection and intervention development will not be included. Resources associated with the intervention will be recorded at the clinical site including the time spent introducing the teen to the Puff City intervention, and the details of all intervention-related contact with patients, by mail, face-to-face, and by telephone.

To measure cost-effectiveness, we will first compare the intervention costs and ACT (the primary endpoint of the trial) for the control and intervention groups and then calculate the incremental cost-effectiveness ratio (ICER) of the intervention compared with the standard controls using the following formula [48]

$$\text{ICER}=\frac{{\text{TotalCost}}_{\mathrm{I}}-{\text{TotalCost}}_{\mathrm{C}}}{{\text{ACT}}_{\mathrm{I}}-{\text{ACT}}_{\mathrm{C}}}$$ (1)

and its 95% CI for ICER and acceptability curves [49], [50]. Standard errors and the correlation of the difference in cost and effect will be derived by use of a bootstrap procedure. The same analysis approach will be used to assess the cost-effectiveness of the intervention using the ED visits, hospitalization, and other secondary outcomes as appropriate.

2.11. Comparison of EMR and CRF data collection

We will validate the information from EMR data retrievals with that of trial CRF data collection. This evaluation will be conducted upon the completion of the Phase II component of the trial. Clinical data required for the trial is first collected in the Oracle Clinical centralized data management system using electronic data capture (EDC) features. In the meantime, the common electronic VDW formatted EMR data (EMR–VDW data) will be collected as batch eData from two HMORN clinical sites (HFHS and KPGA). The positive predictive value/false discovery rate or inter-correlation coefficient will be calculated to study the reliability of EMR–VDW data versus chart abstraction, especially for clinical endpoint data collection (e.g., asthma related ED visit and hospitalization).

3. Results

The note of award (NOA) was received from NHLBI in August 2012 as a response to RFA-HL-12–019 [51] “Pilot Studies to Develop and Test Novel, Low-Cost Methods for the Conduct of Clinical Trials” for the Phase II component of the trial. There were many challenges to designing a cost-efficient trial. Altering the study design from clustered randomization (clinic level) to individual randomization (subject level) allows us to accommodate frequent relocation of the clinics due to healthcare reform, system financial constraints, and statistical considerations regarding possible imbalanced clusters. Patient incentives were modified to achieve 0% post-randomization attrition at the time of enrollment after a few patients were enrolled. We are actively engaged with the clinical sites by adapting and optimizing recruitment strategies and site per patient payment plans to accommodate the clinics with busy routine clinic schedules and limited time/resources for clinical research. A Data Safety Monitor Plan (DSMP) was discussed at the first DSMB meeting 7 months with enrollment of the first patient 6½ months after NOA To date, 50 patients were enrolled from five of six activated HFHS Clinics. KPGA plans to begin enrollment in March 2014.

4. Conclusion and discussion

A number of electronic initiatives are implemented to maximize the use of electronic resources, including 1) the use of EMR to identify eligible patients and coordinate study visits within routine clinical care; 2) e-consent, confirmed at enrollment; 3) online randomization, study intervention, and patient self-report outcome data collection; 4) the use of patient EMRs and remote data capture (EDC) for trial CRF data collection (eCRF); 5) a web-based tracking system for patient compliance; 6) email, text/phone call reminders to patients regarding study tasks; and, finally, 7) the use of EMR eData through common VDW data structures implemented at both healthcare systems via the HMORN to further validate the usage of a direct EMR data collection for clinical trials.

A major challenge in designing an efficient, low cost clinical trial is the use of electronic initiatives including the use of EMRs (EMR-back end) for trial patient identification, use of centralized appointment scheduling for patient recruitment, enrollment, and follow-up (EMR-front end), and the use of primary care visits in conjunction with patient online self-report data for trial data collection. Unlike the EMR-front end data, the EMR back-end (processing) data is not real-time, and is usually processed 6–12 months after becoming available and “stable”, although the process has been constantly improving in order to reduce the time required for data cleaning. For the current trial, we used the EMR back-end data to retrieve asthma history and confirm eligibility (12 months). A computer algorithm automatically generates a listing of eligible patients with upcoming appointments using a 30-day window, although walk-ins can be accommodated using an eligibility survey. We have also designed a process whereby we can refresh our listing of potentially eligible patients with patients that are new to the health care system and may be candidates for the enrollment.

This trial will directly benefit from EMR–VDW data and will allow an exploration of the validity of EMR/VDW data for the data collection in an RCT. This trial is mainly interested in EMR–VDW encounter data. Currently, EMR–VDW has a total of 13 Tables/CRFs (Fig. 1); however data quality and readiness are varied. For example, the tumor table and encounter tables for diagnosis and procedures have been implemented for over 10 years at HMORN, while some lab results data are still in the validation stage. We remain optimistic that the features described in this and similar trials will promote EMR initiatives in clinical trials research, and reduce the cost of conducting RCTs. If EMR–VDW is as reliable as trial CRF data collection, this tool would save the cost of chart abstraction. Such a trial can be relatively easily conducted and implemented in HMORN with 18 health care organization members.

Fig. 1.

HMO Research Network (HMORN)—Virtual Data Warehouse (VDW) structure with common data elements for a direct EMR data (eData) collection through members of HMORN.

We utilized a seamless adaptive approach with a design of two interim looks before Phase III trial completion. The second interim look is expected at the completion of the Phase II component of the trial to examine evidence of no efficacy (futility analysis) to inform a decision to continue or stop the trial at an early point. Such analyses are very important as they may prevent continued enrollment of patients into an RCT when there is little chance of benefit in terms of patient quality of life and reductions in health care costs. This approach (combination of Phase II and Phase III studies) is a CER initiative to utilize Phase II clinic sites and study patients as part of the progression to a Phase III trial if results of Phase II are positive. This design does not jeopardize the Phase II results.

The adaptive seamless design has been regarded as a paradigm shift in clinical trial development because of its unique features. Considerable time will be saved because there is no pause in patient recruitment when proceeding from phase to phase, the same clinical sites may be used in both phases, and all required approvals for both phases are obtained prior to initiation of the trial. Patients enrolled before and after adaptation will be used in the final efficacy analysis, thereby reducing the required sample size and cost.

A weakness of the adaptive seamless design would be if the Phase II trial is completed and results are promising, but the investigators are not able to continue to Phase III. This situation results in a lag between Phase II and Phase III, which seems to be common in clinical trials and delays translation into practice.

Furthermore, we have considered the enriched design to increase study power. Enrichment designs are based on three principles: (1) decreasing heterogeneity (noise) by choosing an appropriate population (patients who definitely have the disease), (2) finding a population with many outcome events (high risk patients or patients with relatively severe disease— “prognostic enrichment”), and (3) identifying a population capable (or more capable) of responding to the treatment-predictive enrichment (biomarker positive). An example of an enriched design is to randomize all eligible stroke patients at the time when the marker is not available, or yet known, with efficacy evaluation only based upon marker-positive patients, but safety based upon the entire population [40]. This is similar to what we have proposed. While enrichment designs may raise concerns about generalizability, the selected population is one where the treatment makes the most sense and this design will be extremely useful for asthma management interventions. For such enriched designs, special attention must be paid to maintaining blinding. By collaborating with other networks or other biostatisticians outside a network, we should be able to use an independent group to do the marker identification analysis. In his recent publication, Richard Simon has shown that various adaptive-enrichment designs both preserve Type 1 error, and provide a substantial increase in power [52].

In summary, the adaptive, seamless, and enriched Phase II/III design with EMR initiatives facilitates the evaluation of an asthma management intervention by (1) shortening the interval between phases of a traditional trial design, (2) reducing the number of required patients, (3) providing low-cost methods for trial implementation, and increasing study power. The approach results in an efficient and cost-effective design. However, it requires a specific statistical plan to guide the necessary decision-making processes, such as the interim looks, the number of looks, and sequential analysis for controlling Type I errors. This trial was designed using comparative effectiveness research methods and the application of a pragmatic approach to eligible patients in a “real world” setting. The results of the trial will be evidence-based and translational.