IMPROVE trial: A randomized controlled trial of patient controlled analgesia (PCA) for sickle cell painful episodes: rationale, design challenges, initial experience, and recommendations for future studies

INTRODUCTION

The pain of a vaso-occlusive crisis (VOC) is the hallmark of sickle cell disease (SCD). Although ambulatory pain accounts for most days with pain [1–3], pain is the most common cause of hospitalization. Despite the potential of novel pharmacological interventions to modify the clinical course of VOC, opioid analgesics remain the mainstay of VOC management. Pharmacokinetic studies have documented enhanced clearance of intravenous morphine in children and adults with SCD [4, 5], the most commonly used opioid in this population, but these findings have not been translated into specific opioid dosing guidelines developed from well-designed clinical trials.

Patient-controlled analgesia (PCA) frequently is used for delivery of parenteral opioid analgesics, and appears superior to intermittent injections [6]. There are, however, only a few studies to guide optimal dosing of PCA to treat pain [7, 8], mostly from brief efficacy studies in post-operative settings. Many clinicians treat severe VOC pain in SCD using PCA to deliver basal continuous infusion opioids supplemented by demand doses to enhance analgesia [9]. A number of serious methodological issues in previous SCD PCA studies, including a lack of well-defined entry criteria and endpoints or blinding of participants and/or healthcare providers to treatment assignments or assessment results, and non-random allocation of participants to study procedures, have limited our understanding of optimal strategies for opioid delivery that provides analgesia while maintaining safety [10–12].

We sought to conduct a multicenter clinical trial to compare two alternate dosing PCA strategies for management of VOC-related pain. Several trial design issues were addressed during protocol development, which were further informed by the initial trial experience. Numerous trial initiation and implementation challenges also were encountered, resulting in early trial termination for slow accrual. While some of these challenges were unique to SCD or analgesia studies, many reflected the increasing complexity of conducting clinical trials in the inpatient setting. This experience provides important lessons for study design and implementation that could improve future inpatient trials, particularly in SCD.

METHODS

Trial Overview

The Sickle Cell Disease Clinical Research Network (SCDCRN) was established in 2006 to develop and conduct phase III intervention trials. It was funded by the National Heart, Lung, and Blood Institute, National Institutes of Health (U10HL083721), initially with 18 clinical sites but 13 additional clinical sites were selected for participation in the Network in 2009 for a total of 31 clinical sites. New England Research Institutes (NERI) served as the Data and Coordinating Center. The IMPROVE PCA trial (Improving Pain Management and Outcomes with Various Strategies of Patient-Controlled Analgesia) was approved for protocol development by the Network in March 2009 and, after review by the NHLBI Protocol Review Committee and Data Safety Monitoring Board, began study enrollment in January 2010. The SCD CRN IMPROVE Trial was registered with clinicaltrials.gov (#NCT00999245). The trial was closed due to inadequate accrual in June 2010.

Local institutional review board (IRB) approvals were obtained at all participating clinical sites. Informed consent was obtained from all adult enrollees. Parents or guardians provided consent for minor enrollees, who provided assent per local institutional guidelines. Most study consents were obtained in the Emergency Department or shortly after admission.

The study was designed as a randomized, single-blind, two-arm inpatient analgesic clinical trial; randomization was stratified by age, 10-17 years (pediatric), and 18 years and older (adult), and by the PCA opioid used during the trial (morphine, hydromorphone). The two treatment arms consisted of two alternative PCA strategies, higher demand doses with low constant infusion (HDLI) or lower demand doses and higher constant infusion (LDHI), each believed to provide a potentially optimal approach to compensate for the altered opioid pharmacokinetics in SCD. The primary outcome measure was time to significant change (25 mm) in average daily pain intensity using a 100 mm (10 cm) horizontal visual analogue scale (VAS), which was measured for the duration of hospitalization, with a number of other pain measures as secondary efficacy measures. Safety measures included the frequency and severity of opioid-related and disease-related adverse effects. Sample size calculations are provided in the appendix.

Individuals with all genotypes of sickle cell disease, more than 10 years of age with vaso-occlusive crisis, with < 12 hours of parenteral opioid therapy from time of presentation to the hospital, and a 100 mm visual analogue scale (VAS) pain score ≥ 45 mm were eligible for the study. Adult or pediatric patients excluded were receiving chronic moderate to high dose oral opioids such as methadone> 40 mg/day, sustained release morphine> 120 mg /day, or oxycodone> 80 mg/day, or had hypoxia, evidence of acute chest syndrome, or significant renal/hepatic dysfunction based on age-appropriate laboratory values.

Initial Study Design Issues

The IMPROVE trial was designed to correct many of the methodological flaws of previous inpatient VOC studies. A number of difficult design issues were addressed during study protocol development. These are summarized in Table 1 and described in more detail below.

Table 1.

Design Issues and Implementation Barriers Encountered and Suggestions for Future Trial Protocols

Major Design Issues or Implementation Barriers Encountered	Suggestions for Future Trial Protocols
Design Issues
Endpoints	• Pain intensity and interference with activities are suitable endpoints with clinical significance.
Blinding of treatments/assessments	• Blinding of assessments/treatments reduces bias but may increase study complexity and staffing requirements.
Prior Acute Care Treatment	• The intensity of prior opioid therapy may not affect efficacy of subsequent PCA analgesic therapy.
	• Limitations on duration of prior therapy may limit the pool of eligible patients and the enrollment rate.
Oral Opioid Usage Prior to Enrollment	• Restrictions are likely to limit enrollment and reduce clinical applicability.
	• Larger opioid PCA dose ranges may be better approaches than inclusion of only opioid tolerant participants.
Statistical Analysis/Sample Size	• Longitudinal analyses of continuous variables are preferred over time-to–event approaches to improve statistical power and reduce needed sample sizes.
	• Analysis of responses from different instruments simultaneously in a multiple outcomes model may be an additional option. Consistency of treatment effect across instruments is important.
Study Implementation Barriers
Coordination with Emergency Department	• Timely communication with ED personnel about potentially eligible participants will enhance enrollment.
Coordination with Inpatient Care Team	• Overly complex clinical care teams reduce enrollment and study efficiency, and increase training burden.
Electronic medical records (EMR)	• Additional resources may be needed for study startup in EMR systems
	• Sharing of order sets and other materials across sites with common EMR systems may be useful.
PCA Pump Methodology	• Using similar PCA pumps at all sites allows standard drug “libraries”, which may increase participant safety and consistency of data across sites.
Standard opioid dosing	• A similar dosing strategy for pediatric and adult participants may be preferred and facilitate trial startup and conduct
Enrollment	• Adequate staffing around the clock is essential.
	• Alternative recruitment processes may enhance enrollment

Selection of Primary Endpoint as Pain

There is considerable controversy concerning the optimal primary endpoint for evaluating interventions to ameliorate severe pain in SCD patients hospitalized for vaso-occlusive pain. An obvious potential endpoint is the duration of hospitalization (length of stay, LOS); however, there are challenges in using duration of hospitalization as the outcome because patients may have their stay prolonged due to unrelated concomitant medical conditions (e.g. fever/infection, acute chest syndrome) or social issues (transportation, family support, self-care competency, etc). A number of previous clinical trials have attempted to use “crisis resolution” as a study endpoint [13–15], defined as a composite measure of pain reduction, termination of intravenous opioids, and ambulation ability; the success of these trials has been hampered by the lack of standard protocols for weaning opioids and by a wide range of pain intensity deemed acceptable for discharge and subsequent home management. Similarly, given the variation in clinical practice for dosing and administering opioid analgesics, the use of either time to reduction in analgesic usage or change in daily amount of (usually intravenous) opioid consumption may be associated with considerable site-to-site variability. A combined endpoint reflecting both analgesic efficacy and safety was considered for IMPROVE [16], but consensus could not be reached on how to weight each factor. Ultimately, pain reduction was selected for the study endpoint, as an acceptable reduction in pain intensity was likely a clinically significant endpoint to both patients and healthcare providers and logically would result in decreased utilization of hospital-related services.

Selection of Opioid Dosing Strategy

A strong scientific argument can be made for the need to optimize PCA dosing strategies for SCD patients based on the impact of chronic anemia on opioid pharmacokinetics. One of the normal compensatory mechanisms to chronic anemia is increased cardiac output with resultant increase in hepatic and renal blood flow. Many analgesics, such as morphine, that have a component of their metabolic pathway dependent on hepatic and renal clearance, will have accelerated plasma clearance in individuals with SCD. For example, in a study of 20 adult SCD patients in steady state, a mean morphine clearance of 4.1 ± 1.1 L/hr/kg (68 ml/min/kg) was observed, which is over twice that in normal individuals [5]. Thus conventional drug dosing may result in sub-therapeutic drug levels in many SCD individuals; strategies that provide constant infusions could compensate for these pharmacokinetic abnormalities. Alternatively, a steady-state drug level can be approximated and maintained by the frequent delivery of relatively small amounts of analgesics, which is the premise of a frequent demand-dose-only PCA strategy.

Based on clinical experience, the SCDCRN investigators recommended that some degree of constant infusion be included in both arms to allow adequate analgesia at night, and some degree of demand dosing that would allow individual titration of dose to address ethical imperatives to provide adequate pain relief. The investigators thus chose to compare two alternative opioid PCA dosing strategies: higher demand doses with low constant infusion (HDLI) or lower demand doses and higher constant infusion (LDHI). Ratios of opioid demand dose to infusion dose were chosen at 3:1 or 1:3 (depending upon treatment assignment) to provide an a priori reasonable difference in opioid delivery for meaningful evaluation, but there was little literature or clinical experience to support this decision. At investigator request, limited additional clinician-ordered doses were permitted to be consistent with actual clinical practice.

Pain Intensity Measurement

Change in absolute pain intensity is most often used to measure pain improvement. In a research setting, a 100 mm (10 cm) horizontal visual analogue scale (VAS) is used most often to assess such intensity differences, while 0–10 numerical scales often are used in the clinical setting with similar validity [17]. In this study, a VAS was used because there are considerable data supporting its validity in the older child and adult populations examined in this study [18]. Some investigators were concerned that a single daily assessment might not be adequate to reflect rapidly changing pain severity, while others questioned the possibility that frequent measurements might be influenced by recent doses of opioids and not reflect overall daily trends. A compromise position was adopted: 3 separate daily VAS determinations at least 4 hours apart were conducted and averaged. The assessments were collected between 7 AM and 7 PM, as that was the time frame during which healthcare providers typically make assessments and subsequent dosing decisions during routine clinical practice.

Masking of Assessments

After the choice of assessment tool was made, considerable discussion of assessment methodology ensued. The responses provided by the patient to these self-report instruments usually are recorded by the clinical care team. To maintain the integrity of these assessments, separate assessors were used and were kept blinded to randomized treatment assignment. Subsequent dosing decisions were made by the clinical team using their standard clinical assessments. While the intent was to maximize the scientific rigor of these subjective self-reports, the requirement of this single-blind policy for an additional research staff member became a significant study implementation barrier at sites with limited study staff resources, particularly on weekends.

Selection of Minimum Pain Intensity Reduction of Interest

Traditional analgesic clinical trials typically use a reduction in pain intensity of either 30% or 50% from pretreatment values as a measure of successful treatment [17, 19]. However, a decision regarding what constitutes a clinically significant treatment response in this clinical setting was difficult given little data to support a specific value for degree of change. Building on earlier work by Todd et al, in general Emergency Department patients [20], Lopez et al. determined that a 13-mm change from initial baseline score on a 100 mm VAS scale constituted a minimally clinically significant pain intensity difference as reported by patients being treated with opioids for a vaso-occlusive crisis, whereas an average change of 24 mm was reported as a moderate difference [21]. The SCDCRN study investigators choose a 25 mm change in the daily average VAS as the target improvement difference for the study.

Secondary Efficacy Endpoints

Acute pain interferes with a variety of physical activities, and the relief of pain is assumed to be associated with improvement in physical functioning [22]. Recommendations for the use of specific physical functioning measures as endpoints for analgesic clinical trials in adults have been made by an industry-academic consortium, but these measures have not been studied in sickle cell disease, particularly in the inpatient setting [23]. These measures typically assess pain-related changes in activities of daily living and interference with sleep [24, 25]. Unfortunately, few comparable measures are available for children. The SCDCRN investigators selected a number of additional physical activity measures as secondary endpoints including the Brief Pain Inventory (BPI)[26] and the SF-8 Short Form Health Survey [27]. Wrist actigraphy, a motion sensor device that continuously records wrist movements, also was selected as an exploratory outcome measure as it had the potential to be an objective measure of physical activity [28] but had not been well studied in hospitalized patients.

Secondary Safety Endpoints

Opioid-related symptoms rather than SCD complications were chosen as secondary safety endpoints for this study to reflect its analgesic focus. The intensity of opioid adverse symptoms, including sedation, nausea, and pruritus scores, was assessed daily using a validated assessment tool [29]. The magnitude of patient reported opioid-related withdrawal symptoms was assessed by scripted telephone interview at 3 and 14 days post hospital discharge based on a validated questionnaire. These results have been published elsewhere [30].

Immediate Pre-study Analgesic Therapy

The influence of prior therapy on a treatment baseline was a design concern as all participants would have received variable amounts of previous analgesic therapy at home and/or in an acute care setting (usually an Emergency Department [ED]) prior to the institution of study PCA. The investigators recognized that mandating ED treatment would require consenting and enrolling participants prior to institution of analgesic therapy; the logistics of such a process in most busy urban ED settings was judged to be impossible. A pragmatic choice was made to allow 12 hours of prior analgesic therapy which was unlikely to be exceeded in busy EDs; however, this decision limited study participation at those sites with “day hospital” or other infusion centers that typically provided prolonged therapy durations. There was also concern that when study participation began opioid doses recently administered in the ED might affect the baseline VAS measurement against which subsequent comparisons were to be made.

Prior Outpatient Opioid Usage

The final design issue considered by the SCDCRN investigators concerned the participant's history of use of extended-release oral opioids (morphine or oxycodone) or oral opioids with slow clearance (methadone). Investigators' experiences suggested many adult patients were maintained on substantial doses of these medications for chronic pain, often at doses that would be equi-analgesic to some of the lower opioid PCA dosing levels. There was no consistent clinical practice about whether to continue or discontinue these oral medications during the hospitalization. To maintain PCA treatment fidelity and enhance participant safety, it was decided to discontinue such medications at the time of study enrollment whenever not already discontinued by the clinical care team. To avoid enrolling highly opioid-tolerant participants, adult or pediatric patients were excluded who previously received moderate to high dose oral opioids equi-analgesic to the initial opioid PCA dosing (methadone> 40 mg/day, sustained release morphine> 120 mg /day, or oxycodone> 80 mg/day). Unfortunately, this decision resulted in a significant number of exclusions at some sites.

Lessons Learned

Site-Related Study Implementation

The IMPROVE study largely failed at the site level. While 31 clinical sites were able to acquire regulatory and contractual approval within a 3-month period, only 14 sites ultimately enrolled participants during the subsequent 6-month study period. Furthermore, many of these 14 sites started enrollment shortly before the study was terminated. A number of site-specific logistical barriers relating to the increasing complexity of conducting studies in the in-patient setting were encountered and contributed to difficulties with study startup, conduct and enrollment (Table 1).

Site Clinical Care Organization and Communication

The organization of the inpatient care team varied considerably across the 31 clinical sites. Some smaller sites and most of those with pediatric participants often had inpatient services supervised by attending hematologists familiar with SCD, who were also the site principal investigators. However the majority of sites managed inpatient care using a variety of additional individuals or services, including hospitalists, general internal medicine, or oncology services. Many sites also managed PCA therapy using a separate clinical service, either anesthesiology or dedicated pain services. Considerable effort was required after local site protocol approval to address clinical care practices and protocol specifications for opioid dosing and other protocol design issues with these services that had not been represented in protocol design or development. Site-specific procedures had to be developed to train these groups on the rationale for and implementation of the protocol. Most of these clinical sites were teaching hospitals with residents, fellows, and attending physicians who rotated clinical coverage. The need for repetitive teaching to maintain familiarity of rotating personnel with appropriate protocol-related management was unanticipated by most sites.

Close coordination with staff in the ED also was required to identify potentially eligible participants promptly. Frequently clinical care team members who were contacted about SCD patients who needed admission for pain management were not connected with the research team familiar with the trial protocol. Since a considerable amount of clinical experience was necessary to choose the most appropriate initial opioid PCA dosing levels, physician investigators, rather than research coordinators, frequently had to be involved in the study initiation process for each participant, which most site principal investigators had not anticipated. More widespread usage of dedicated research units, such as those provided by Clinical and Translational Science Awards (CTSA) might have facilitated study operations at many study sites with these staffing and organizational communication needs.

Site Infrastructure

The widespread introduction of electronic medical record (EMR) systems and computerized order entry substantially increased the complexity of implementing the protocol. Many research staff did not have the training, experience, or system access to develop the necessary order sets and had to rely on external clinical or information technology (IT) staff to implement the protocol orders. At some sites the entire opioid PCA dosing schedule had to be entered into a pharmacy order system. Several EMR systems have wide market penetration, so in future studies some early-enrolling sites could facilitate study implementation at subsequent sites by sharing their inpatient and pharmacy order sets, or these could be provided as protocol appendices when developed as part of the protocol. Financial resources also may be required to have hospital IT staff implement additions in a timely manner.

The study investigators elected to utilize the PCA pumps available at each hospital as part of routine care to minimize trial expense for pumps and associated medication delivery supplies and to avoid the need for additional competency training and support for clinical care nurses if unfamiliar pumps were used. However, many clinical sites still used older PCA pumps that did not have the accuracy required by the protocol dosing schema for appropriate dose delivery of hydromorphone to young pediatric participants. Newer “smart” PCA pumps have several additional advantages: they provide digital recording and printouts of dose delivery that facilitate dosing documentation, which was often problematic when standard-of-care nursing flow sheets were used. Protocol safety and fidelity also could be enhanced by using electronic dosing guidelines or “libraries”, developed for each pump brand, that reflect all of the protocol-related dosing, and then sharing them among all similar pumps within and between hospitals. Finally, many newer PCA pumps have more complex lockout capabilities that enhance dosing flexibility while maintaining safety for individual participants.

Standard Opioid Dosing Tables

Investigators were required to use study-provided opioid dosing tables for each morphine or hydromorphone dose range to reduce the risk of medication errors or protocol non-compliance. Dosing was weight-based for patients who weighed <50 kg. The dosing range for adults (≥ 18 years) spanned a two-fold range based on investigator clinical experience while a four-fold dosing range, extending the lower dosing levels, was used in pediatric participants (10-17 years). These dosing tables were split into multiple steps to provide a logical methodology for dose escalation and weaning that was particularly critical for the LDHI treatment arm. Subsequent investigator study experience suggested that additional lower dosing levels would also be useful for adult participants. Despite the intent of providing safety for opioid naïve pediatric participants, the decision to use pediatric weight-based doses at weights<50 kg caused significant dosing confusion for some heavier adolescent and lighter adult participants. Providing individual weight-based dosing for multiple dosing levels also resulted in complex and potentially confusing dosing tables. Standard non-weight based dosing schedules over a broad clinically appropriate range may be a more straightforward approach for future studies.

Enrollment at Study Sites

Much of the original enthusiasm for the study was generated by the large pool of potentially eligible patients based on local site patient data, which was consistent with subsequent study screening data. Initial feasibility estimates of enrollment rates of 20–25% were somewhat more optimistic than observed (13% of approached patients). The large number of patients never approached for enrollment (>80%) was unexpected, and limits the representativeness of our enrollment characteristics [31]. The modest available funding for staffing resources and limited staffing redundancy contributed to inadequate coverage to recruit the 60–70% of potential participants who were admitted after 7 PM on weekdays or on weekends. Funding for future studies must permit sites to expand potential study staff with clinically experienced individuals and to incentivize them for after-hours recruitment activities.

Poor communication with the research team about potentially eligible patients in the ED was clearly one component of this enrollment difficulty. Poor communication between clinical care teams and study research teams also contributed to this difficulty, as considerable clinical knowledge was necessary, particularly to initiate appropriate PCA opioid dosing, and was beyond the scope of experience and practice of many study research staff. Communication may be facilitated by enrollment of potential participants in an outpatient clinic setting, where clinical and research staff work together to decide on clinically appropriate individualized order sets for a subsequent study admission.

Initial Study Experience

The study provided valuable preliminary data for several key design issues. Opioid usage in the 22 adult participants confirmed the feasibility of the two PCA dosing strategies, while the demand component of the PCA was not used optimally by the 12 pediatric participants [30]. In adults, the 25 mm drop in VAS scores occurred within a median of 2 days for both the HDLI and LDHI treatment groups, with 100% of adults achieving that threshold during the study (Table 2). Larger drops in VAS scores resulted in discernible treatment differences (Figure 1): a 45 mm drop occurred within a median of 4.5 days (50% attained prior to discharge) for the HDLI treatment group, and 3 days (90% attained) for the LDHI treatment group; this outcome yielded a significant hazard ratio of 0.245 (logrank test: p=0.023). The 12 pediatric participants surprisingly showed slower improvement in pain intensity than adults and no treatment related difference between treatment arms was noted in this small sample.

Table 2.

Time to First Drop in Visual Analog Scale (VAS) Pain Scores

			Median Days (and event rate) to first drop in daily average VAS from baseline VAS				Mean Length of Stay
		N	− 13 mm	− 25 mm	− 35 mm	− 45 mm
Adults (Age ≥ 18)	All	22	1.0 (1.00)	2.0 (1.00)	3.0 (0.86)	3.0 (0.68)	5.6
	HDLI	12	1.0 (1.00)	2.0 (1.00)	3.0 (0.75)	4.5 (0.50)	7.1
	LDHI	10	1.0 (1.00)	2.0 (1.00)	2.5 (1.00)	3.0 (0.90)	3.9
	Hazard ratio		0.77	0.69	0.40	0.25
	p-value *		0.573	0.429	0.067	0.023
Children (Age 10–17)	All	12	1.5 (0.75)	3.0 (0.58)	3.0 (0.50)	3.0 (0.33)	4.3
	HDLI	7	2.0 (0.86)	3.0 (0.57)	3.0 (0.57)	3.0 (0.29)	4.0
	LDHI	5	1.0 (0.60)	3.0 (0.60)	3.0 (0.40)	3.0 (0.40)	4.6
	Hazard ratio		1.67	0.97	1.59	0.65
	p-value *		0.463	0.973	0.594	0.670

^*The hazard ratios are estimated from Cox proportional hazards models comparing the HDLI versus LDHI groups using the Breslow method for handling ties. The p-values correspond to logrank tests on the hazard ratios. In addition to the unadjusted model results presented here, the models also were fit using baseline VAS as a covariate, which resulted in less than 10% effect modification (increase in the hazard ratio). HDLI, higher demand doses with low constant infusion; LDHI, or lower demand doses and higher constant infusion.

Figure 1. Individual Trajectories of Drops from Baseline VAS.

Trajectories of drops in average VAS from baseline for individual participants, ordered by length of stay within treatment and age groups. Within each panel, the first occurrence of a −25-mm drop is represented by a solid black dot, whereas the first occurrence of a −45-mm drop is represented by a larger open circle. The participant-level trend (from a longitudinal model) is represented by a dark black line segment, whereas the group-level (age- and treatment-specific) trend is represented by a light grey reference line. HDLI, higher demand doses with low constant infusion; LDHI, or lower demand doses and higher constant infusion.

Longitudinal Analysis of VAS as Alternative Analysis Approach

We conducted a longitudinal analysis (Table 3) to better understand the rates of change across study day for the two treatment arms. Recent statistical investigations [32] have provided some theoretical justification for the relative efficiency and practical advantage of longitudinal rate of change models versus time to threshold models in similar data settings. Our analysis was based upon 114 observations across 2 to 14 days for 22 adult participants. Two alternative continuous responses were considered: absolute drop from baseline and relative (percent) drop from baseline. Each model included four fixed effects to describe the rate of change for the two treatment groups: intercept (average on day 1 for the HDLI reference group), rate of change across study day for the HDLI group, treatment effect (difference between LDHI versus HDLI on day 1), and interaction term (difference between treatments in rate of change, LDHI versus HDLI). Similar longitudinal models were fit to the secondary outcomes. Despite the small sample size, the interaction term was highly significant (p<0.001) in both models considered. Additionally, the model provided participant-specific random effect intercepts and slopes which yielded estimates of the between-participant variability (expressed as standard deviations [SDs] in Table 3). The intra-class correlation (ICC) was computed as the ratio of the between-participant variability to the total variability and reflects the proportion of variation due to participants versus noise. The estimates in Table 3 also can be used to calculate the expected drop for the less efficacious treatment (HDLI) on day 5 (39 mm), whereas that same drop was achieved in just 1.5 days by the more efficacious treatment (LDHI). Sample size calculations for a future study based on these results are presented in the appendix.

Table 3.

Longitudinal Rate-of-Change Model for Drop from Baseline VAS Scores (22 Participants, Age ≥ 18)

Response: Drop from baseline VAS	Fixed Effects Coefficients				SD Components
	Reference treatment (HDLI)		Diff. between treatments (LDHI vs. HDLI)		Between		Within	ICC
	Average (Day 1)	Rate of change/day	Average (Day 1)	Rate of change/day *	Intercept	Slope	Residual
Absolute (mm)	−22.47	−3.36	−0.45	−8.35	12.13	1.56	14.23	0.42
Relative (%)	−31.24	−4.35	3.03	−10.96	15.54	1.73	19.27	0.39

^*The models included fixed effects for treatment, rate of change across day, and the interaction between treatment and rate as well as random effects for rate across day (intercept and slope) for each participant. The effect size in this model is measured as the difference between rates of change for the two treatment groups. The statistical significance of the difference in rate of change/day parameter was p<0.001 for both models presented here. HDLI, higher demand doses with low constant infusion; LDHI, lower demand doses and higher constant infusion.

Discussion

The IMPROVE trial investigators sought to overcome the limitations of earlier clinical studies of PCA opioid dosing in SCD to provide a model for future inpatient SCD studies. We adopted a feasible minimum VAS pain level as an entry criterion for studies of VOC in SCD. Baseline VAS pain at study entry was high among IMPROVE participants, most commonly 71 to 80 mm in adults, much greater than the entry criterion of VAS pain ≥ 45 mm, even after patients had received up to 12 hours of analgesia in the ED prior to admission. These findings are consistent with previous clinical studies that suggest moderate to severe pain intensity in patients admitted from the ED for vaso-occlusive pain management [33, 34]. There was little correlation within patients between entry VAS and either time to target improvement or length of stay, suggesting our VAS entry criterion also allowed assessment of improvement. Further, there was no correlation between intensity of prior ED treatment for up to 12 hours and entry VAS in adults or in children. Further study of a larger cohort would be useful to confirm these findings.

Our trial design provided rigorous, daily inpatient pain assessment measures and preliminary data on alternative pain intensity, duration, and analgesic usage that may inform the design of future inpatient SCD trials. A VAS decrement of 25 mm on a 100 mm scale was chosen as the target pain intensity improvement threshold because it was twice the minimal important difference (MID) [35] in studies of adults with acute SCD pain [21]. This level of improvement was attained relatively rapidly in adults but somewhat more slowly in pediatric participants. In contrast, time to a much larger drop in pain intensity to about 45–50 mm, which represented a decrease in pain intensity of at least 50% of the baseline level, was associated with treatment effects in this small cohort of adult participants. As the percentage pain drop correlated well with the maximum absolute pain drop, either could be used as a study endpoint for pain relief, but the absolute drop may be a more convenient measure.

About half of the study participants received hydromorphone and half morphine (hydromorphone was used somewhat more commonly in adults), supporting the decision to allow both to be used in the study [9]. Combining these treatments for analyses was supported by this initial experience of nearly equal consumption of both opioids when adjusted for analgesic potency. The two different PCA strategies were feasible and were used by the adult trial participants as planned, with more frequent demand dosing in the HDLI arm compared to the LDHI infusion arm; however, a similar difference in demand dose utilization was not seen in the pediatric participants. The relationship between daily analgesic consumption and daily pain intensity was blunted at high pain intensity levels in the HDLI arm, suggesting that it may be difficult to achieve or maintain high levels of opioid analgesics with demand dosing. Although consistent with clinical experience, this finding should be replicated in larger studies. Future opioid PCA studies would be facilitated by reducing the complexity of dosing guidelines, but further experience will be required to optimize the range of dosing needed to accommodate a range of participant ages, weights, and prior oral opioid exposures. Since these initial data suggest substantial differences between adults and children in amounts of prior oral opioid used, response to initial ED analgesic treatment, opioid usage (particularly for demand-dose usage), time to improvement, and duration of hospitalization, designers of future studies should consider adult versus pediatric differences in design and sample size calculations.

The study was designed with little preliminary data on the expected time to achieve a 25 mm pain intensity improvement. Both treatment groups in this small set of adult participants achieved the 25 mm drop so quickly that the time to threshold models were not efficient in discerning a difference between treatments. Since the interim stopping rules defined efficacy as a difference between treatments in time to achieve the target drop, the trial as originally designed was unlikely to have been terminated early due to efficacy despite the large treatment difference seen in the magnitude of the drop achieved after the median hospital stay. Longitudinal rate-of-change analyses may be more efficient for studies where the magnitude of the treatment effect is not well understood. Longitudinal rate-of-change designs should be used only in studies having four or more measurements per participant, as fewer measurements result in loss of power or biased estimates of variability that could result in misleading assessments of efficacy.

A further statistical consideration concerns assumptions about the missing data mechanism when participants drop out prior to the full intended duration of study. Since all adult participants who were discharged prior to five days in this study had achieved the target reduction in VAS (Figure 1), we assume the missing data mechanism in these data was missing at random [36] in that unobserved responses depend only upon those that have already been observed and not those observed after discharge. Finally, in future studies, it may be necessary to consider that changes on the VAS (and many of the other pain and relief measurement scales) are bounded [37]. While the models presented herein were used to describe the trend in improvement for up to five days into the study, extension of the fitted lines to later time points would “predict” an unattainable decrease.

While the time required for local site regulatory and financial review and approvals often are perceived to be major barriers to study implementation, our experience suggests that for inpatient studies a greater challenge is developing effective communication among the research team and the many members of often large and changing local clinical care teams. While recommendations and best practices for study conduct can be disseminated among sites, the environment and resources of each site require individualized implementation strategies. Selection of some smaller and medium size clinical programs, which may have less complex care systems, also may be an effective strategy to improve study performance.

Although the implications of the IMPROVE trial experience are limited by its low enrollment, it was designed to correct many of the methodological issues of previous in-patient SCD VOC studies. The experience of the SCDCRN investigators in the design and implementation of this trial at academic teaching hospitals provides a template for others wanting to conduct similar inpatient studies of needed novel analgesic or innovative therapies for VOC in SCD. Use of alternative analysis strategies requiring substantially reduced samples sizes could increase the feasibility of such studies in this uncommon disorder.

Appendix

Sample Size Determinations

Sample size for the trial was estimated based on the method described by Lakatos [38] for a two-sided logrank test of the difference in time to achieve the target 25 mm drop between two treatment groups. The median time to achieving the target drop was assumed to be 5 days in the least efficacious group and 3.5 days in the more efficacious group, corresponding to a hazard ratio of 0.7. The calculation also assumed that accrual occurred uniformly across 6 months (183 days) with a median hospital stay of 4.25 days (187 day total study duration), and that no crossover between treatment groups would occur. Based on these assumptions, 258 participants (half in each treatment group) was required to detect the hypothesized hazard ratio with power of 0.80 at alpha=0.05. The sample size was then inflated by 1.02 to account for interim looks at the data for efficacy and safety and 1/0.95 to account for 5% anticipated dropouts, resulting in the final estimated sample size of 278 with 139 participants in each treatment group. These calculations may be reproduced in PASS software [39] [version 8.0] based on the assumptions outlined above.

In designing a future study, the alternative longitudinal rate-of-change model proposed may be more efficient. Only 46 adult participants (23 per treatment arm) would be required to detect a difference between treatments in rate of change of 5 mm per day (25 mm across the five-day observation period) when using 120% of the variability observed in these data as a pilot estimate and allowing for a 20% fraction of missing data from incomplete follow-up (typically caused by early departure due to treatment efficacy). This sample size requirement was estimated based on the method described in Fitzmaurice [40] and Donohue [32, 41]. It also may be desirable to estimate between-site variance, which, in addition to the sample size requirement, typically requires that some minimum number of sites contribute several enrollees each in order to provide stable estimates; simulation studies are recommended for this purpose. A related concern is whether randomization to treatment should occur at patient level or site level, which would affect both the design and analysis [42]. If randomization is at the patient level and global inference [43] is desired, then both site and site by treatment interaction should be included as random effects, thus increasing the variance of the treatment effect and thereby resulting in a larger sample size requirement [44]. Randomization at the site level may address some of the implementation challenges, but special precautions must be taken into consideration during the design and analysis to ensure the results are interpreted at the appropriate level. In either setting, increased power may result from having more sites as opposed to more enrollees per site [45] or from including site-level covariates in the analysis. [42].