Assessment and classification of protocol deviations

Abstract

Introduction:

Deviations from the approved trial protocol are common during clinical trials. They have been conventionally classified as deviations or violations, depending on their impact on the trial.

Methods:

A new method has been proposed by which deviations are classified in five grades from 1 to 5. A deviation of Grade 1 has no impact on the subjects’ well-being or on the quality of data. At the maximum, a deviation Grade 5 leads to the death of the subject. This method of classification was applied to deviations noted in the center over the last 3 years.

Results:

It was observed that most deviations were of Grades 1 and 2, with fewer falling in Grades 3 and 4. There were no deviations that led to the death of the subject (Grade 5).

Discussion:

This method of classification would help trial managers decide on the action to be taken on the occurrence of deviations, which would be based on their impact.

INTRODUCTION

A protocol is one of the essential documents in clinical research and often standard treatment guidelines (STG) are also known as protocols. In either case, these are developed after due consideration of the evidence-based practice and represent the best method of use of therapeutic regimes. The guideline for Good Clinical Practice of the International Council on Harmonization (ICH GCP) defines the protocol as “A document that describes the objective (s), design, methodology, statistical considerations, and organization of a trial.” The protocol usually also gives the background and rationale for the trial, but these could be provided in other protocol referenced documents. Throughout the ICH GCP guideline the term protocol refers to protocol and protocol amendments.[1] This definition remains unchanged in the draft of the guidelines revision, reiterating ICH's faith in the protocol.[2] Of late interest in protocols is steadily rising as evidenced by increasing publications on the Medline [Figure 1].

Figure 1.

Publications concerning protocols on the Medline

With the increased focus on safety and efficacy, the complexity of the protocol is on the rise, a factor that is hampering recruitment and delaying completion of trials.[3,4] This has also affected the economics of drug development and trial performance.[5] In addition, regulation, governance, and management of biomedical research have also become increasingly complex.[6] Since clinical research is an activity conducted by a team of the personnel, their workload is rising proportionally. Studies show that the workload depends on the complexity of the protocol, as determined by the Ontario Protocol Assessment Level.[7] With the increase in complexity, there is an expected rise in deviations from the protocol[8] and a rise in Type I errors, or an incorrect rejection of the null hypothesis.[9]

The protocol is the document that the research team must follow religiously if the clinical trial is to be compliant with all regulations. Trial protocols are reviewed and approved both by the regulatory authorities and Institutional Ethics Committees (ECs) before they are implemented. Deviations from the approved protocol should only be made with the consent of the regulators and the ECs, as they may reduce the benefit to the subject or increase the risk, and could also compromise the data obtained. Therapeutic procedures in protocols closely follow STG, if there are serious deviations from the same, the trial becomes nonscientific and hence unethical.

Deviations from the approved protocol are common and have been noted both in routine management[10] and in research,[11] at differing frequency. These deviations in a study could be due to the subject, the sponsor, or the investigational team. Compliance by a subject to advised medication regime is also known as medication adherence and is dependent on a large variety of factors including the disease, efficacy of medicine, age, and mental attitude of the patient.[12] Some of the deviations could be avoided by proper counseling of the subjects whereas some are not avoidable. Deviations caused by investigational staff are often due to poor training, and can be prevented.

A lot of work has gone into identifying causes of protocol deviations. Globalization of clinical research has led to differences in the quality of data emanating from different countries, and this has global implications on drug discovery and development.[13] Economics of drug development demands faster recruitment and completion of studies, this could cause protocol deviations,[14] additionally, the need to publish, unreasonable expectations and greed have been identified as other causes.[15]

It is accepted that deviations vary in their incidence and impact and have also been classified accordingly. Minor divergence of a study from the approved protocol is classified as a deviation while one that which affects the quality of data or impacts subjects’ safety is classified as a protocol violation.[16] Deviations are further classified as noncompliance, misconduct, or fraud. A single instance of a deviation could be classified as a noncompliance while repeated and systematic noncompliances (usually despite warnings) are considered as misconduct. Whenever there is a financial motive behind the noncompliance, it may be classified as a fraud.

Misconduct is defined as “fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results.”[17] Polanin-Huk define fraud as “Fraud is an intentional deception made for personal gain or to damage another individual, for instance, intentionally falsifying and/or fabricating research data, and misleading reporting of the results.”[18] Trial managers need to take action against noncompliance depending on which of the above classes they fall in.

Thus, the noncompliance continuum has protocol deviation on one end and fraud on the other. Each of these may occur independently of each other; nonetheless they lie in a logical continuum. Detection of noncompliance, misconduct and fraud may require progressively greater effort and often sophisticated techniques including advanced statistics.[19] Missing deviations or ignoring them, could possibly lead to the occurrence of higher impact incidents such as misconduct and even fraud.

There is a subtle difference between a protocol deviation and a waiver. It may be mentioned here that waivers for informed consent may be granted in some cases, but a failure to take consent in the absence of a waiver is a serious omission.[20] When trial managers are aware that a deviation is likely to take place, they may take a prior approval of the sponsor in the form of a waiver for the deviation, but when the deviation is discovered after it has taken place, and then the incident is a deviation and not a waiver. The medical monitor may grant waivers for deviations that are not likely to impact either the data quality or safety of the subject. Since waivers are preapproved deviations, no action is necessary to be taken.

The role of the EC in the detection and analyses of protocol deviations is of utmost importance. There is fear that ECs, though good at initial review of trials, are notoriously lax in ongoing study reviews.[21] Deviations by their very nature do not occur when a study comes to the ECs for approval but occur after the study has begun and hence can be detected only during ongoing reviews. The detection of deviations therefore rests solely on the shoulders of the ECs, and the trial staff that voluntarily reports the same. If incidences of deviations are detected only during monitoring or audits, it speaks poorly of the performance of the trial staff and the ethical review process.

When the deviations from the protocol are discovered, then there is need to analyze their impact. In general, instances of noncompliance are many, but the impact of each individual instance on the study is minor. Conversely, the incidence of fraud is relatively low[22] but its impact may be very serious [Figure 2]. Misconduct is the gray area between these two extremes. Management of protocol deviation and action to be taken is dependent on the impact of the specific instance.

Figure 2.

Relation between incidence and impact of deviations

Despite the variety of causes and types of misconduct and fraud, they damage the value of clinical trials and must be dealt with severely.[23] Those deviations, which are one off and classified as noncompliances, could have varying degree of impact.[24] The impact ranges from none to most severe leading to the death of a subject or subjects. In addition, the impact of deviations on data quality needs to be analyzed, because data is one of the most important outcomes of trials.[25] Deviations need to be analyzed and their impact assessed when it lies between these two extremes. We have classified deviations in five grades as follows:

• Grade 1: No impact on data quality or patient safety

• Grade 2: Minor impact on data quality

• Grade 3: Minor impact on patient safety

• Grade 4: Major impact on data quality or patient safety

• Grade 5: Leading to patient/(s) death.

Our theoretical gradation was put to test on protocol deviations observed in trials at the Jehangir Hospital, Pune, over the last 3 years.

METHODS

All protocol deviations that have occurred in the last 3 years (from January 2013 to September 2015) in clinical trials (both regulatory and nonregulatory) were examined. The data collected for each deviation were as follows:

• Study code

• Trial and principal investigator

• Assessment of deviation by the ECs.

The entire data were tabulated in excel and analyzed.

RESULTS

In the past, deviations were classified in two groups minor and major. Overall, there were a large number of deviations, and the number of major deviations has shown a reducing trend. A year-wise breakup is shown in Table 1.

Table 1.

Classification of deviations during the observation period

All these deviations were classified by two methods, as follows:

• The Stakeholder responsible for the deviation, i.e., the subject or the investigational team as shown in Table 2

• The grade of the deviation as suggested above.

Table 2.

Protocol deviations by source

On analyzing the deviations on the basis of grades, it was observed that most deviations were of Grades 1 and 2 and as the grade rose further; the number of deviations fell as shown in Table 3.

Table 3.

Grade-wise distribution of deviations

Over the last 3 years, there has been no reduction in the occurrence of protocol deviations; in fact there has been an increase. Since data from 2015 are for only 9 months, the increase in deviations could be due to the improved reporting system. However when the deviations were analyzed on the basis of their impact, it is noted that the relation between impact and incidence of deviations was inverse (deviations with minimum impact had high incidence, whereas those with maximum impact were very few) as shown in Figure 3.

Figure 3.

Grade-wise occurrence of protocol deviations

DISCUSSION

One of the basic principles of research ethics is to offer maximum benefits and minimum risks uniformly to all subjects within a treatment arm in a clinical trial. All trial documents are prepared with the principles of ethics in mind, and a deviation from the laid down procedures may cause a decrease in the benefit or an increase in the risk to the concerned patient. By changing the treatment or by altering observations, it may affect the data quality in the trial. As the incidence and seriousness of deviations increases, the compliance of the trial to regulatory requirements and ethical guidelines goes down.

As noted at our site, protocol deviations do take place. Though a majority are of minor nature and do not have significant impact, each deviation must be identified, reported, and analyzed for its impact. Only then would one know if the deviations may be condoned or not. It is true that the ECs is charged with the role of protecting subjects in the trials, it cannot do so unless it is supported by the entire clinical research unit (CRU) and the investigators. As much as the EC, the CRU needs to be sensitive to protocol deviations and overall compliance of the trial.[26]

The earlier system of classification of deviation does not fully take into consideration the impact of the deviations. We have focused on the impact, both on subject safety and quality of data, since these are two most important aspects of clinical trials. There also needs to be some level of objectivity in classifying deviations, which we hope to have achieved to some degree. Hopefully, regulators will find this classification acceptable and superior to the existing one.

The present study threw up some deficiencies of the original method of classification. Some of the deviations originally classified as major were often procedural and had no impact either on data quality or subject safety. An example of this is the failure of the subject to date the signature on the informed consent form, this is a major procedural lapse but can hardly impact data quality or subject safety. Some deviations classified as minor in fact had an impact on data quality and subject safety. What is satisfying is that no deviation led to fatal consequences.

In our analysis and gradation of deviations, their effect on data was considered of lesser importance than that on the subject, since the rights and well-being of subjects take precedence over everything else in clinical research. Considering the large number of protocol deviations which are in fact noncompliances, there is always the risk that EC members become complacent about them. Such an attitude will be deleterious to the functioning of the EC and expose the subjects to needless harm.

Most deviations in this study were found to be of Grades 1 and 2, deviations of Grades 3 and 4 were progressively lower. There was no deviation of Grade 5, i.e., one which led to the death of a subject. This pattern agrees with the known distribution of deviations. It shows that the largest number of deviations have minor impact and those having major impact are rarer, as suggested by George (2015).[22]

In this study, focus is on protocol deviations and their impact, the impact of waivers either for procedures or for informed consent has not been studied. Waivers involve a different set of stakeholders and since a waiver cannot be granted without the consent of the sponsor, greater responsibility rests with the sponsor. Waivers also should be scientifically justifiable, else they will damage the quality of data obtained and even put the subjects to risk. Hence, oversight of waivers by the ECs is essential for the protection of subjects.

Scientists and physicians are often critical of the objections raised by ECs. It is their contention that ECs must inquire in the ethics of the studies, but should not delve deep into the science of the trials. None of the international guidelines provide any advice on the depth to which the ECs may probe the science of the trials. It is however undeniable that science and ethics are intertwined, and a clear-cut separation between the two is not feasible. The standard operating procedures of the ECs may provide some guidance, but individual members of the ECs will remain free to probe into the science of the trials to their own satisfaction.

Protection of subjects is the primary function of the ECs, all other functions being secondary to this. To fulfill this, the ECs must do a comprehensive preliminary review, ongoing review and also provide oversight for many other activities during a trial. The ECs must review a variety of documents including the clinical trial agreement and insurance certificate since defective agreements or insurance policies lead to the subjects’ reimbursements or compensation being delayed or denied. EC members should also be proactive, viewing subjects as their charges, since they owe their existence to subjects.

Protocol deviations, intentional or otherwise may produce data that are not accurate, reliable, or credible. Care should be taken to identify them and avoid them as seriously as possible. These could lead to research of poor integrity that has serious impact on subjects, physicians, hospitals, and scientific journals. The subject is most vulnerable since protocol deviations are associated with increased risks of treatment failure and overall mortality.[27]

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.