Variations in study outcomes relative to intention‐to‐treat and per‐protocol data analysis techniques in the evaluation of efficacy for treatment of venous leg ulcers with dehydrated human amnion/chorion membrane allograft

Abstract

Statistical interpretation of data collected in a randomised controlled trial (RCT) is conducted on the intention‐to‐treat (ITT) and/or the per‐protocol (PP) study populations. ITT analysis is a comparison of treatment groups including all patients as originally allocated after randomisation regardless if treatment was initiated or completed. PP analysis is a comparison of treatment groups including only those patients who completed the treatment as originally allocated, although it is often criticised because of its potential to instil bias. A previous report from an RCT conducted to evaluate the efficacy of dehydrated human amnion/chorion membrane allograft (EpiFix) as an adjunct to standard comprehensive wound therapy consisting of moist dressings and multi‐layer compression in the healing of venous leg ulcers (VLUs) only reported PP study results (n = 109, 52 EpiFix and 57 standard care patients), although there were 128 patients randomised: 64 to the EpiFix group and 64 to the standard care group. Primary study outcome was the incidence of healing at 12 weeks. The purpose of the present study is to report ITT results on all 128 randomised subjects and assess if both ITT and PP data analyses arrive at the same conclusion of the efficacy of EpiFix as a treatment for VLU. Rates of healing for the ITT and PP populations were, respectively, 50% and 60% for those receiving EpiFix and 31% and 35% for those in the standard care cohort. Within both ITT and PP analyses, these differences were statistically significant; P = 0.0473, ITT and P = 0.0128, PP. The Kaplan‐Meier plot of time to heal within 12 weeks for the ITT and PP populations demonstrated a superior wound‐healing trajectory for EpiFix compared with VLUs treated with standard care alone. These data provide clinicians and health policymakers an additional level of assurance regarding the effectiveness of EpiFix.

1. INTRODUCTION

The primary goal when developing a treatment plan for a patient presenting with a venous leg ulcer (VLU) is rapid and complete healing. Customarily, first‐line treatment of a VLU includes debridement, wound dressings, and aggressive compression therapy. Unfortunately, the chronicity of VLUs is afflicted by the high recurrence rates and protracted courses of treatment, with a mean 12‐week healing rate of less than 50% of those with VLUs in receipt of comprehensive therapy.1 During the course of treatment, when VLUs fail to show an adequate healing trajectory, clinicians often consider incorporating advanced therapies such as biological dressings into the treatment plan, yet there is a paucity of evidence‐based information available to help a clinician determine which advanced treatment to select.

Evidence‐based practice (EBP) and randomised controlled trial (RCT) are terms that have become omnipresent to discussions of health care policy development (government and private) and the delivery of health care services.2 EBP is most commonly defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”3 It is generally accepted that the best research evidence is usually found in studies that are clinically relevant and conducted using sound methodology. The RCT is a research methodology developed in an attempt to reduce bias and enhance the accuracy of clinical experimentation while producing generalisable, universal biomedical knowledge.4 To that end, the RCT methodology is considered to generate the highest level of evidence. Yet it remains important to recognise that discernment is still necessary when evaluating the results of an RCT as all study methodologies have weaknesses. RCTs are conducted to quantify the effects of a treatment and provide evidence of the efficacy and safety for either clinical, regulatory, or policy decision‐making. Randomised trials are expected to be free from baseline confounding influences, but as in clinical practice, events will occur that may complicate the description and interpretation of treatment effects.5

The statistical interpretation of data collected in an RCT is often conducted on intention‐to‐treat (ITT) and/or the per‐protocol (PP) study populations. ITT analysis is a comparison of the treatment groups that include all patients as originally allocated after randomisation regardless of if treatment was initiated or completed.6, 7, 8 PP analysis is a comparison of treatment groups that includes only those patients who completed the treatment as originally allocated. When examining data from an RCT, analysis of data with the application of both ITT and PP techniques allows for a more robust interpretation of study results. If both ITT and PP analysis lead to essentially the same conclusions, confidence in the trial results is increased.

A recent multicentre RCT by Bianchi et al9 examined the use of a commercially available dehydrated human amnion/chorion membrane allograft (EpiFix, MiMedx Group Inc., Marietta, Georgia) as a treatment for chronic VLU. That study reported results on only those 109 subjects completing the study PP. The purposes of the present study are to report ITT results on all 128 randomised subjects and assess if both ITT and PP data analyses arrive at the same conclusion of efficacy for EpiFix as a treatment for VLUs, to explain how the results of these two analysis approaches may differ under various study design considerations, and to discuss factors that influence ITT and PP results and how this relates to evaluation of wound care products.

2. METHODS

As previously reported by Bianchi et al,9 an 18‐week multicentre, randomised, controlled, open‐label study was conducted to evaluate the efficacy of the EpiFix allograft as an adjunct to standard comprehensive wound therapy consisting of moist dressings and multi‐layer compression in the healing of VLUs. The study population consisted of patients with VLUs receiving care from physicians and/or podiatrists specialising in wound care at 15 outpatient wound care centres geographically distributed across the United States. Of the 15 study sites, 10 were private practice, and 5 were hospital‐based centres. The study was approved by the Chesapeake Investigational Review Board (IRB) or each site's local IRB, each study site complied with applicable regulatory requirements and adhered to Good Clinical Practice principles, and the study was conducted in accordance with the provisions of the Declaration of Helsinki and preregistered on ClinicalTrials.gov (NCT02011503). The confidentiality of all patient records was maintained. Complete details of the study methods can be found in the previously published paper.9

2.1. Summary of study methods

The 18‐week study consisted of three phases: (a) 2‐week screening phase, (b) 12‐week treatment phase, and (c) 4‐week follow‐up phase. Eligibility for enrolment into the 2‐week study screening phase was assessed as described in Table 1. During the screening phase, VLU treatment for all patients consisted of an alginate dressing and multi‐layer compression bandages (3M Coban 2 Layer Compression System OR Coban 2 Layer Lite Compression System). At the end of the 2‐week screening phase, those subjects whose VLUs had not reduced in size by at least 25%, measured between 1 and 25 cm² post‐debridement, and who continued to meet other eligibility requirements entered the treatment phase and were randomised to receive weekly application of EpiFix or standard care. Subjects were followed for 16 weeks post‐randomisation. During the 12‐week treatment phase, subjects were seen weekly for assessment and treatment group‐appropriate wound care. Multi‐layer compression bandages were continued in both study groups throughout the study. A final follow‐up study visit was conducted at week 16 post‐randomisation.

Table 1.

Study eligibility and exclusion criteria

Eligible for study inclusion	Study exclusion criteria
>18 y of age	VLU penetrating to muscle, tendon, or bone
Full‐thickness VLU of at least 30 d duration	Signs of ulcer infection or cancer
Ankle brachial pressure index of >0.75	VLU located on the dorsum of the foot or more than 50% of ulcer below the malleolus
	Received negative pressure wound therapy or hyperbaric oxygen therapy in the last 7 d prior to evaluation for enrolment or treatment with other advanced wound care products within the past 30 d

Abbreviation: VLU, venous leg ulcer.

The primary study outcome was time to complete wound closure as assessed over a 12‐week period from treatment initiation. Complete healing of the study ulcer was defined as 100% reepithelialisation without drainage. Throughout the study, in order to support accurate and consistent wound assessment and reduce observer bias, the Silhouette camera (Aranz Medical, Christchurch, New Zealand) was used to perform wound imaging, measurement, and documentation at all study sites. Another method used to reduce the risk for bias and ensure standardisation was the adjudication of wound images by a group of three wound care specialists blinded to the treatment group. These blinded independent physicians reviewed all wound images and confirmed the measurements and healing status of the wound. All photographs and measurements of the study ulcer were obtained post‐debridement.

2.2. Summary of statistical methods and study groups

A two‐sided log rank test with an overall sample size of 120 subjects (of which 60 are in group 1 and 60 are in group 2) achieves approximately 87% power at a 5% significance level to detect a difference of 30% between the proportions of subjects whose ulcers are unhealed by 12 weeks in the subject group receiving EpiFix or standard care. Parametric and non‐parametric tests were used as appropriate. Student's t test, analysis of covariance (ancova), or the Kruskal‐Wallis test was used to test for differences in continuous variables. For categorical variables, χ ² or Fisher's exact test were performed to test for statistical differences. Kaplan‐Meier analysis was performed, with two‐sided P‐values <0.05 considered significant.

The ITT study population was comprised of all randomised subjects regardless of protocol deviations, non‐compliance, or early study withdrawal. The PP study population, whose results were reported in the previous paper,9 was a subset of randomised subjects after excluding those patients with absolute protocol deviations and those not completing the study because of early withdrawal.

3. RESULTS

A total of 189 subjects were screened and entered the study for the 2‐week run‐in period between March 19, 2015 and March 3, 2017. At the end of the screening phase, 61 patients were no longer eligible for randomisation. There were 128 patients randomised and included in the ITT population: 64 to the EpiFix group and 64 to the standard care group. The PP population reported in the prior manuscript was comprised of 109 subjects: 52 received EpiFix, and 57 received standard care.9 Data from 19 randomised subjects, 12 from the EpiFix group and 7 controls, were not included in the published report.9 The status of enrolled participants and determination of ITT and PP study groups are summarised in Table 2.

Table 2.

Status of enrolled participants

Status	n
Total enrolment (subjects consented)	189
Screen failures/2‐wk screening phase	61
Total randomised	128
Total treated (ITT study group)	128
Serious adverse eventa	7
Protocol deviationsb	7
Early withdrawal	3
Investigator withdrawal	1
Lost to follow up	1
Subjects with complete primary endpoint data (PP study group)	109

Abbreviations: IIT, intention‐to‐treat; PP, per‐protocol.

^aSerious adverse events include death (cardiac arrest because of coronary artery disease), trauma, alcohol poisoning, and ulcer worsening resulting in additional intervention.

^bProtocol deviations include product not applied weekly according to the protocol or subjects not meeting inclusion criteria.

Descriptive patient demographics and wound characteristics are shown in Table 3. In both ITT and PP populations, the study groups were well matched for clinical factors, including the presence of comorbidities as well as the location, duration, and size of the study ulcer. In both ITT and PP populations, the broad inclusion criteria allowed for the enrolment of subjects with frequently observed severe comorbidities, including: diabetes (regardless of level of blood glucose control), cardiovascular conditions, musculoskeletal abnormalities, smoking history, advanced age, extreme obesity, wound size up to 25 cm², history of recurrent ulceration, and no limit as to how long the study ulcer had remained unhealed.

Table 3.

Descriptive patient demographics and wound characteristics. Data presented as mean ± standard deviation, median (minimum, maximum), or number (percent) as indicated

	Intent‐to‐treat group			Per‐protocol group9
	EpiFix (n = 64)	Standard care (n = 64)	P‐value	EpiFix (n = 52)	Standard care (n = 57)	P‐value
Age, in years	62.2 ± 14.3 63 (29, 93)	60.3 ± 11.4 59 (38, 84)	0.4059	61.5 ± 14.9 63 (29, 93)	60.0 ± 10.6 59 (38, 82)	0.5436
Gender: male	42 (66%)	44 (69%)	0.8508	33 (63%)	39 (68%)	0.6863
Race			0.6897			0.3896
Caucasian	51 (80%)	50 (78%)		41 (79%)	45 (79%)
African‐American	8 (13%)	11 (17%)		6 (12%)	10 (18%)
Other	5 (7%)	3 (5%)		5 (10%)	2 (4%)
Smoker	22 (34%)	31 (48%)	0.1322	16 (31%)	28 (49%)	0.0782
Alcohol use	25 (39%)	28 (44%)	0.7378	17 (33%)	24 (42%)	0.6200
Body mass index	35.4 ± 10.7 33.1 (18.5, 70.0)	36.6 ± 10.8 33.8 (20.1, 80.0)	0.5304	36.0 ± 11.2 33.9 (18.5, 70.0)	37.2 ± 11.0 35.7 (20.1, 80.0)	0.5913
Hx diabetes	15 (23%)	21 (33%)	0.2656	14 (27%)	20 (35%)	0.5122
Hx hypertension	10 (16%)	8 (13%)	0.8000	8 (15%)	7 (12%)	0.7823
Wound characteristics
Ulcer side			0.6015			0.5382
Left limb	36 (56%)	34 (53%)		27 (52%)	31 (54%)
Right limb	28 (44%)	28 (44%)		25 (48%)	24 (42%)
Ulcer position			0.1133			0.1173
Malleolus	24 (38%)	18 (28%)		19 (37%)	14 (25%)
Low gaiter	35 (55%)	32 (50%)		29 (56%)	30 (53%)
Other	5 (8%)	14 (22%)		4 (8%)	13 (22%)
Ulcer location			0.2562			0.0714
Medial	34 (53%)	27 (42%)		30 (58%)	23 (40%)
Anterior	9 (14%)	8 (13%)		8 (15%)	7 (12%)
Lateral	14 (22%)	25 (39%)		10 (19%)	24 (42%)
Other	7 (11%)	4 (6%)		4 (8%)	3 (5%)
Ulcer duration (wk)	40.0 ± 55.6 20.0 (4, 312)	61.5 ± 71.6 39 (4, 384)	0.0683	41.9 ± 60.0 17.5 (4, 312)	58.9 ± 72.6 35 (4, 384)	0.2000
Baseline wound size, cm2	7.4 ± 5.8 5.1 (1.0, 24.3)	8.6 ± 6.8 6.3 (1.2, 24.8)	0.2893	7.6 ± 6.1 5.2 (1.1, 24.3)	8.3 ± 6.7 6.2 (1.2, 24.2)	0.5944

The primary study outcome, rate of complete healing at 12 weeks, is presented in Table 4. In both ITT and PP populations, rates of complete healing at 12 and 16 weeks were significantly greater in subjects receiving EpiFix vs standard care.

Table 4.

Complete healing at 12 and 16 weeks

	Intent‐to‐treat group			Per‐protocol group
	EpiFix (n = 64)	Standard care (n = 64)	P‐value	EpiFix (n = 52)	Standard care (n = 57)	P‐value
Healed at 12 wk	32 (50.0%)	20 (31%)	0.0473	31 (60.0%)	20 (35%)	0.0128
Healed at 16 wk	38 (59%)	25 (39%)	0.0335	37 (71%)	25 (44%)	0.0065

The Kaplan‐Meier plot of time‐to‐heal within 12 weeks for the ITT population demonstrated a superior wound‐healing trajectory for EpiFix compared with VLUs treated with standard care alone. (Figure 1) The Log‐Rank test of equality of the healing function over the two study groups produced a χ ² test statistic of 4.6007, with a P = 0.032. The Kaplan‐Meier plot of time‐to‐heal within 12 weeks for the PP population also demonstrated a superior wound‐healing trajectory for EpiFix compared with VLUs treated with standard care alone. (Figure 2) The Log‐Rank test of equality of the healing function over the two study groups produced a χ ² test statistic of 6.4597, with a P = 0.011.

Figure 1.

A Kaplan‐Meier plot of time‐to‐heal within 12 weeks by study group (intention‐to‐treat population, n = 128)

Figure 2.

A Kaplan‐Meier plot of time‐to‐heal within 12 weeks by study group (per‐protocol population, n = 109)

4. DISCUSSION

The purpose of the present study was to assess if both ITT and PP data analysis demonstrate superiority of EpiFix over standard moist dressings as a treatment for VLU. Our results show that, when study data were analysed using both ITT and PP techniques, VLU treatment with EpiFix in addition to multi‐layer compression therapy conferred more rapid and complete healing than treatment with standard moist dressings and multi‐layer compression therapy alone. These outcomes are impressive and lend credibility to the external validity and generalisation of study results given the broad inclusion criteria that allowed for the enrolment of subjects with frequently observed severe comorbidities, including diabetes (regardless of level of blood glucose control), cardiovascular conditions, musculoskeletal abnormalities, smoking history, advanced age, extreme obesity, wound size up to 25 cm², history of recurrent ulceration, and no limit as to how long the study ulcer had remained unhealed prior to study enrolment. These less restrictive criteria permitted patients often excluded from past RCTs addressing chronic wounds to be enrolled, thus producing results that more closely reflect actual practice.

RCTs are quantitative studies that seek to measure and compare outcomes after the participants receive the interventions. The efficacy of a treatment or intervention is evaluated by whether expected or superior results are produced under ideal circumstances. Effectiveness is measured by the degree of beneficial effect in the clinical setting where ideal conditions may not be present. When analysing data obtained through an RCT in order to evaluate both the efficacy and effectiveness of an intervention or treatment, it is important to look at the data in both PP and ITT fashion. The CONSORT guidelines for reporting of “parallel group RCTs” recommend that both ITT and PP analyses should be reported for all planned outcomes to allow readers to interpret the effect of an intervention.10

PP analysis refers to inclusion in the analysis of only those patients who strictly adhered to the protocol. During the course of a study, participants might withdraw because of unsatisfactory treatment response, overly burdensome treatment protocols, intolerable adverse events, or even death. In addition, subjects may be non‐compliant with the treatment schedule or receive additional interventions outside of the study protocol. Eliminating these subjects from statistical analysis has the potential to influence the power of the study and/or create an imbalance of confounders between the groups, violating the principles of randomisation.11 Yet, the PP analysis provides an estimate of the true efficacy of an intervention among those who completed the treatment as planned. In the current study, rates of healing at 12 weeks for those subjects completing the PP study were 60% for the EpiFix group and 35% for standard care, P = 0.0128.

The principle of ITT analysis is that all study subjects are analysed in the group to which they had been randomised irrespective of if they received the assigned treatment, received additional treatments outside of the study protocol, were non‐compliant, or withdrew from the study altogether.6, 7, 11, 12 The ITT analysis ensures the maintenance of comparability between groups as obtained through randomisation, maintains sample size, and eliminates bias related to attrition. It is recognised, however, that ITT analysis results in a more conservative estimate of treatment effect than PP analysis. Trials may be marred by deviations from protocol, some patients failing to comply with the prescribed treatment, and the same or other patients dropping out before the study endpoint can be observed. While methods to impute outcomes, such as carrying the last observation forward, are commonly utilised, they are not a true outcome. These deviations from the study protocol mean that the ITT analysis no longer validly estimates the true effectiveness of the studied intervention.12 This conservative measure of treatment effect was observed in the present study where, in the ITT analysis, rates of healing at 12 weeks reduced to a more conservative 50% in the EpiFix group vs 31% for standard care, P = 0.0473.

Treatments that promote more rapid healing that are easy to obtain and apply are desired by clinicians caring for patients with chronic wounds. Variations in how study data are reported may make the interpretation of study results and how these results relate to clinical practice difficult. As we have discussed, there are pros and cons to both ITT and PP data analysis. In certain clinical conditions and patient demographics, PP analysis may be a superior indicator over ITT data in providing guidance to choosing improved treatment pathways or solutions and in setting treatment expectations. Patient populations that are inherently non‐compliant because of external forces out of the patient's or clinicians' control (eg, lack of family caregiver support, immobility, financial limitations, chronic comorbid medical conditions), would not be expected to achieve the same treatment results as a population of patients receiving treatment as intended on a consistent basis. Patients with hard‐to‐heal wounds are affected by a multitude of comorbid conditions and are prescribed multiple medications, plus many are frail and malnourished or may need assistance with ambulation and transfer.13 Coordinating follow‐up care is often difficult as many patients are unable to comprehend or retain complex wound care instructions because of cognitive deterioration, education, or language barriers. ITT analysis provides expectations of treatment outcome when these often‐present confounders cause treatment to be altered or prematurely halted, while PP analysis shows us how a treatment performs as it is intended to be used, even in a population where there are confounders present.

Given the knowledge that the process of wound healing involves metabolic, structural, biochemical, and patient factors in a unique way, one acknowledges that wound healing is not a single event; it is a result of complex overlapping processes. The order and combinations of treatments used can be varied and may be guided anywhere along the wound‐healing cascade by these possible confounding factors that cannot be accounted for or controlled, thus biasing or influencing ITT outcomes from one day to the next. This is why focusing on PP results allows clinicians to recognise and select effective evidence‐based therapies that work for patients and caregivers who are able to overcome obstructive circumstances, thus affording greater compliance with such prescribed protocols.

When evaluating the strengths and weaknesses of a clinical trial, it is necessary to examine study methods and data analysis techniques used. The reliability of clinical evidence depends on several aspects, including the internal validity or risk of bias, associated with research design and the final analysis set used with which to draw conclusions. Bias is described as systematic errors in clinical trials that may encourage one outcome over others and is often the explanation as to why investigators may reach different conclusions regarding intervention effects.14 Bias may be introduced at any phase of a research project.15 A good clinical trial minimises the variability of the evaluation and provides unbiased evaluation of the intervention by avoiding confounding from other factors, which are known and unknown.16 Randomisation ensures that each subject has an equal chance of receiving any of the treatments under study and generates comparable intervention groups that are alike in all the important aspects except for the intervention.16 Selection bias is prevented in RCTs, and the multicentre RCT design is often considered to be the gold standard from the clinical research paradigm. For the assessment of treatment effects, the large randomised trial is one of the most reliable sources of evidence.16

In the current study, several methods were used to reduce the potential for biasing study results. The RCT design and stringent concealment allocation reduced the risk of selection bias. Randomisation resulted in equal distribution between the study groups in patient demographics, wound characteristics, and comorbidities. As we were unable to blind the study site investigator because of the nature of the allograft material, efforts were made to reduce observer bias in determining healing status. The Silhouette camera was used to perform wound imaging, measurement, and documentation at all study sites. All wound images were reviewed by a group of three wound care specialists blinded to treatment group and study site for final determination of healing. Attrition bias, a systematic error caused by the loss of study subjects from a RCT, was eliminated in the present study by the examination of data with ITT analysis. Both positive and negative effects on study results related to industry funding and competing interests are frequently debated.14 Disclosure and the peer‐review process are believed to help mitigate the risk of bias related to competing interests.17, 18 To ensure transparency, study funding and competing interests have been disclosed by all authors. Publication bias was addressed through the presentation of both ITT and PP study results.

The evaluation of emerging technologies is often limited to observational studies, which may exaggerate treatment effect. The cornerstone of treatment for VLU is compression therapy. The most advanced interventions used in the management of chronic VLU lack supporting evidence that they add any benefits to compression therapy alone.19 A comparative effectiveness review published in 2013 on advanced treatments for VLU retrieved over 10 000 peer‐reviewed papers related to the treatment of VLUs but identified only 60 adequate studies conducted between 1980 and 2012, the majority of which were deemed to provide insufficient strength of evidence. The review concluded that most interventions used in the management of chronic VLUs lack evidence in the form of high‐quality RCTs to support their use.19 The paucity of quality evidence is clear given that 20 years elapsed between the RCT deemed to provide moderate evidence for use of a bioengineered skin substitute as a treatment for VLUs and the current study.19, 20 The strength of the present study lies in its RCT design, as well as the positive results observed in both ITT and PP analysis, which are both important analyses given a complex patient population. These data provide clinicians and health policymakers an additional level of assurance regarding the effectiveness of EpiFix and the likelihood that these results can be generalised given the large number of clinical study sites and broad patient population included in the present study. In conclusion, the results of this sentinel study show that, in both ITT and PP analyses, VLUs treated with EpiFix as an adjunct to debridement, moist wound dressings, and compression had significantly higher rates of healing than those treated with comprehensive wound care alone.

Bianchi C, Tettelbach W, Istwan N, et al. Variations in study outcomes relative to intention‐to‐treat and per‐protocol data analysis techniques in the evaluation of efficacy for treatment of venous leg ulcers with dehydrated human amnion/chorion membrane allograft. Int Wound J. 2019;16:761–767. 10.1111/iwj.13094