The advance of research governance in psychiatry: one step forward, two steps back

Abstract

Purpose.

To investigate the reasons behind difficulties in recruiting patients to randomized controlled trials (RCTs) in psychiatry and to examine a database of RCTs for differences between studies in mental health and other specialities.

Methods.

A discussion of recent changes in research governance in the UK and Europe followed by an examination of the database of all trials supported by the Health Technology Assessment programme of the National Institute of Health Research in the UK between 1993 and 2007 to determine if three different measures, (i) time between grant approval and study start date, (ii) percentage of additional time given to extend recruitment and (iii) percentage of planned recruitment achieved, changed over the time period studied and differed between mental health, cancer and other medical disciplines.

Findings.

Despite attempts in the UK to accelerate the process of clinical trials in recent years, there was a significant increase in the extension time for trials to be completed (p = 0.038) and the percentage of planned recruitment to mental health studies (71%) was significantly less than for cancer (90.3%) and other studies (86.1%) (p = 0.032).

Summary.

These results suggest that, despite the priority afforded to the advancement of RCTs in healthcare, such studies are encountering increasing difficulty in recruiting to time and target. We suggest that this difficulty can be attributed, at least in part, to the excessively byzantine regulation and governance processes for health research in the UK, and unnecessary bureaucracy in the current National Health Service system. Mental health studies appear particularly vulnerable to delay and better systems to facilitate recruitment are required urgently for the evidence base to be improved and facilitate new cost-effective interventions.

Introduction

About 40 years ago Archie Cochrane, then a public health researcher, wrote a powerful book deploring the wastefulness of research studies in influencing health policy. He argued that randomized controlled trials (RCTs) were not only essential to get good evidence for treatment but their results also needed to be combined. The obsession with cure, a very rare outcome in medicine, had to be replaced with probability of benefit. Otherwise, as he put it, ‘The pursuit of cure at all costs may restrict the supply of care’ (Cochrane, 1972). This book was the catalyst for the development of the Cochrane Collaboration, an international network of more than 28 000 dedicated people from over 100 countries. This prepares, updates and promotes healthcare providers, policy-makers, patients, their advocates and carers, so that they can make well-informed decisions about health care via the Cochrane Database of Systematic Reviews, part of The Cochrane Library. Many things about the clinical research process itself have also changed in the 40 years since Cochrane published his influential book. Much of this evolution has been driven by a desire to improve the quality and safety of clinical research and undoubtedly large advances in the protection of participant's rights, safety and wellbeing have been achieved through this development. There is also greater confidence that the results of these studies are both robust and credible. However, efficiency may actually be reduced due to the complicated web of activities that are required in order to set up and maintain a trial.

Such activities start at the grant application stage, and include the significant amounts of time spent on the appointment and training of research staff, drawing up of contracts between organizations and suppliers and the manufacture of trial medication in the case of trials of medicinal products. Arguably the most complex of these are the research regulation and governance processes where various permissions, approvals and authorizations to carry out the research are sought and given. Many tasks must happen consecutively rather than in parallel, meaning that progress may grind to a complete halt with a single difficulty. Progress can also be burdened by the slow progress that is characteristic of many large organizations or by multiple individuals or organizations being involved in a single process.

One step forward

The growth of large trials of intervention in mental illness has been impressive over the last 25 years. Very large studies such as the CATIE (Lieberman et al. 2005) and STAR-D studies in the US (Insel & Wang, 2009) and CUtLASS in the UK (Jones et al. 2006), all carried out independently of pharmaceutical company support, have changed our thinking about the use of antidepressants and antipsychotic drugs and smaller but high-quality equivalent studies of psychological treatments have led to the adoption of psychological treatments such as cognitive behaviour therapy en masse in the UK in the growth of the improved access to psychological treatments programme (Clark, 2011). The introduction of specific topic networks, including one for mental health, has assisted this greatly, as multicentre studies have been resourced by trained staff, clinical studies officers, who facilitate recruitment, assist with research governance and provide extra input to studies that are recruiting below target. There is clear evidence that this has improved recruitment to trials (McCrae et al. 2012). This initiative has extended to Europe and other countries and the recent Madrid Declaration (Ayuso-Mateos et al. 2011) formalized this cooperation by building on existing European Union-funded projects as well as national initiatives.

In the UK, several relatively recent innovations have resulted in real improvements for RCTs. One is the development of the Integrated Research Application System, a web-based system that allows information required for most regulatory and governance assessments to be entered in one place. Another is the introduction of the option to make an application for ethics review to one ‘main’ Research Ethics Committee (REC) with the opinion of the committee applicable to all UK sites where the research will take place. Another is the introduction of ‘research passports’ for research staff that allows them to cross from one National Health Service (NHS) organization to another without the need for separate accreditation and checks.

In the European Union there has been similar concern over the need to introduce a better regulatory framework for clinical trials. The guidance for trials introduced in 2001 has been widely criticized for being overbureaucratic and constituting a hindrance to advance. The number of applications for clinical trials fell by 25% from 2007 to 2011 and the average delay for launching a clinical trial increased by 90% to 152 days over this period. As a consequence a ‘harmonized authorisation dossier’ (European Commission, 2012) has been introduced, whose main elements are (i) a ‘single portal’ to submit an application for conducting a clinical trial linked to an EU database; (ii) a flexible and swift assessment procedure without establishing a new, central bureaucracy, largely controlled by Member States; (iii) clear timelines with a concept of tacit approval in order to ensure compliance, a coordination and advisory forum to address issues which may arise in the authorization procedure; (iv) the option, in certain well-defined cases, for a Member State to ‘opt-out’ of the conclusions of an assessment of an application for conducting a clinical trial (‘qualified opt-out’) and (v) a swift procedure to ‘extend’ a clinical trial to additional Member States.

This new directive has unfortunately already come under criticism over several of its aspects (Gøtzsche, 2012; Tuffs, 2012; Waligora, 2012) and seems unlikely to solve the fundamental concerns of the research community.

Two steps back

The increasing complexity of research governance has, in our subjective experience, delayed trials by more than the improvements have aided them, and this was the main reason for our research enquiry. To help in understanding some of the difficulties of research governance in the UK, we summarize evidence from a variety of sources, including our own experiences, as an introduction to the topic. We do not know whether these are representative of the difficulties in carrying out trials in other countries and suspect that each country has its own special problems, but nonetheless feel that many of them are common to all.

Fragmented process of research regulation and governance

Proportionate considered regulation and governance of clinical research is vital to maintain public confidence in the research process and ensure the quality of the output. In the UK, research must undergo ethical review, but added to this may be additional and separate review processes where the research involves access to identifiable patient information without prior consent, medicines or healthcare products, gene therapy, radioactive substances or human tissue. Finally, permission must be obtained for it to take place in each NHS organization involved in the research by making an application to those in the Trust responsible for research governance (often named the research and development office or R&D for short). While the pathway that has evolved is understandable, it is also complex, tortuous and felt to be unnecessarily bureaucratic. The consequence can be delays in the time that a study has to recruit participants. Failure to recruit to target is a major concern, resulting in reduced power to answer the important research question itself, increasing the chance of Type II error and incorrect conclusions. It may also result in the loss of credibility to deliver meaningful research trials.

A recent working group chaired by Sir Michael Rawlins (Academy of Medical Sciences, 2011) reviewed the regulation and governance of UK health research. A key finding of this group was that checks that are the responsibility of national regulators such as the National Research Ethics Service (NRES) are being replicated and reinterpreted by NHS Trusts, leading to undue delays. The Academy of Medical Sciences Report recommended simplification of the regulation and governance processes via the introduction of a new health research agency, providing one point of entry and exit for all applications. Their model includes a national research governance service to ‘oversee a streamlined, common process for NHS R&D permission’, a proposal that we fully endorse and which may shortly be introduced.

We also have experience of the rejection by a research ethical committee of urgent safety measures put in place following consultation with the Medicines and Healthcare Regulatory Agency (MHRA). This is in spite of the recommendation that an REC ‘may generally rely on the MHRA to assess the safety of medicinal trials’ and ‘is not required to undertake its own safety assessment’ (Memorandum of Understanding, 2010). The result was a protracted negotiation of seven months to find a solution that satisfied all parties. This is unlikely to be an exceptional case, and there are similar examples dogging non-commercial trials of medication (Sheard et al. 2006). We therefore hope that an integrated system of governance and approval such as that suggested by the Rawlins Report may avoid experiences such as ours, as a decision issued by a single agency would need to be coherent.

Local reinterpretation of regulation can also lead to problems. To give one example, in a study carried out in an acute medical Trust but concerned with a mental health intervention, researchers with many years of experience of carrying out trials and taking consent from patients, were required to pass an assessment of their ability to take consent using role-play. When this exercise was carried out, a relatively junior nurse, with no experience of mental health, who had added this responsibility to her ordinary full-time work, was asked to undertake this assessment of the researchers. She was highly embarrassed about this exercise, as she admitted that she had no experience of obtaining consent for mental health research and had much less knowledge of research than either of the researchers that she was being asked to assess.

Inexperience in local feasibility and governance processes may also result in inefficiency and lack of confidence to move forward. The new Health Research Authority (HRA) may yet provide a comprehensive single source of advice but this is currently work in progress.

An added complication to the approval process in mental health trusts is that care frequently relies on the local acute trust or primary care for certain services that may be an intrinsic part of the research protocol. While services such as pharmacy, phlebotomy and basic physical health check tools such as electrocardiogram machines are easily accessed in acute Trusts, this is not the case in mental health. This considerably complicates the attribution of costs, contracts and even the feasibility of the trial itself.

The NHS as a research partner

The NHS, and indeed publicly funded health services in any country, is an excellent natural laboratory for research as their patients are almost always representative of those with the condition under investigation. Indeed, research is fundamental to the NHS itself, since it aspires to high standards of excellence and professionalism. One of the key principles of the NHS constitution is to ‘innovation and to the promotion and conduct of research to improve the current and future health and care of the population’.

Unfortunately only a few hospitals recognize the time and resources used to support research by directly returning funds to clinical teams. A more standardized funding pathway that positively reinforces research activity at a local service level would, in our opinion, will be an important step in changing attitudes towards research in the NHS.

At an organizational level, yet more barriers to research exist that are related to cost allocation. Patient care costs that would continue if the treatment continued to be provided after the end of the research are expected to be met from existing commissioning arrangements, thus hospital Trusts incur the cost of providing treatment to their patients that may yet show no benefit. The attribution of the various costs can also result in complicated scenarios that hamper set-up. For example, in an RCT of an already licensed medication for a new use (i.e., a phase IV trial), the medication itself is a treatment cost, although the over-encapsulation and production of matching placebo is a research cost. This means that the cost of the medication must be recovered from each Trust that hosts the research; giving rise to often difficult negotiations between various stakeholders. Conversely, if this were simply a new compound the entirety of the cost of medication would be a research cost.

The idiosyncrasies of participant recruitment

RCTs are essential for evidence-based clinical practice to move forward. However, this is not a message that is universally understood by clinical staff. More frustratingly, sometimes it seems that this is understood but is combined with the hope that you will go and do the research with someone else's patients and come back with the results. NHS managers are under considerable pressure to deliver immediate healthcare targets, and understandably therefore afford low priority to research. This message permeates staff at all levels and the result is that the NHS can be a difficult climate in which to carry out research.

Time constraints and competing clinical priorities aside, clinical staff may be reluctant to facilitate research for a variety of reasons including concern about the way in which it will affect relationships with patients. For example, our experience is that non-medical staff often believe that it is wrong for them to identify patients who may be suitable for a study, regardless of the way in which the study is approved to recruit or reassurances that the approval of the consultant would be sought prior to any further action. This leaves only the consultant to make referrals, despite the fact that they may not know the patient as well as other members of the clinical care team. This is particularly pertinent in mental health where patients that are stable may only see their consultant annually.

Anxieties about what is appropriate are inevitable as many clinical staff lack even the most basic understanding of research governance and regulation. We recognize that the system is complex and idiosyncratic, but staff may still not understand that large trials must have clinical equipoise by virtue of having been granted approval to take place. On repeated occasions, researchers have received the response from both hospital staff and, indeed, ethics committees, that a study cannot possibly be ethical as some patients will receive a placebo, which will have no benefit. The answer seems to be to train all clinical staff in the principles of RCTs including communicating their value in developing better treatment. Training may also go some way to reduce the gatekeeping of patients by clinical staff that is often experienced by researchers. Mental health patients are defined as a vulnerable group, and many, such as those with intellectual disability, lack the capacity to consent and this adds to the challenge of recruitment (Tyrer et al. 2009). Although the lack of capacity to consent is a valid reason for not approaching patients about research, there is a halo effect that extends to a far larger group of patients, as clinical staff act as ‘protectors’. RCTs must successfully capture the cross-section of the population under investigation for the results to be valid and reliable otherwise that population will be disenfranchised from advances, which is tending to occur in child psychiatry (Graham, 2000) and is now largely the case in intellectual disability, where evidence of efficacy is in such short supply that the subject is in danger of being deskilled. As Bhaumik et al. (2011) comment ‘developing a research-based evidence base is not only critical to the establishment of new services or interventions, but also necessary to support the value of existing services’. It is vital that patients who are very unwell are included in trials as well as those with fewer symptoms and higher levels of functioning. Indeed many randomized trials are now focused on treatment-resistant patients and are therefore reliant on being able to approach individuals that remain symptomatic.

We recognize there are many examples of clinical staff who clearly understand these issues and go the extra mile in facilitating research, making time in their busy work schedules to do so. However, this means that a patient's opportunity to take part in an RCT may be solely based on whether someone in their clinical team is positive about research. It is incredibly hard to recruit into RCTs, with ineligibility and lack of desire to participate responsible for reducing the number of individuals randomized into a trial by a considerable degree in many cases. It is a shame that the number is further reduced by inconsistent access to eligible patients that arises from reliance on the buy-in of clinical staff.

Constraints on access to patient data

There is a vast amount of data on patients in any public health service. These data are used extensively within the NHS in order to provide consistent care across different settings and are routinely shared in a secure and confidential way. This prevents many of the problems in care that happened in the past because of poor communication. Unfortunately, researchers almost have the status of undesirable aliens when it comes to gaining access to this same information because confidentiality is considered to be breached if an individual that is not involved in the provision of care to a patient views identifiable clinical data.

The handbook to the NHS constitution states that ‘the NHS will do all it can to ensure that patients…are made aware of research that is of particular relevance to them’. However, the current system relies on clinical staff screening their own caseload which, as we have previously commented, often does not happen due to various factors. The message remains that research staff are not equal to clinical staff in their standing in the NHS, even when the NHS is their substantive employer. This seems a clear contradiction of the statement in the NHS constitution that ‘research is a core part of the NHS’.

Analysis of trials funded by the Health Technology Programme

All pragmatic trials funded by the Health Technology Assessment (HTA) programme of the National Institute of Health Research with a contract start date between 1995 and 2007 were included in the survey, provided they were completed at the time of review (January 2013). These were separated into those concerned with cancer, mental health and all other medical disciplines. From HTA records, four time points were recorded; (i) the date of the grant award, (ii) the prime contract start date (funder and lead organization), (iii) the date of expected completion and (iv) the actual date of completion. Date of grant award was only recorded for those funded from 1997 onwards. From the HTA monograph resulting from the trial or an alternative published source where no monograph was identified, the original target number for recruitment and actual recruitment at the end of the study were recorded. Trials that were closed early were noted and excluded from any analyses.

From these data, the following measures were derived:

1. Time between grant award to contract start date (weeks). The HTA as funder sign the contract conditional on REC approval and, where appropriate, other national regulatory authorization such as that from the MHRA, being in place. Delay to governance and regulatory processes will result in a longer time between grant award and contract start date.

2. Percentage time extension granted to the study.

3. Percentage of the originally planned target recruited by the completion date. This was based on the originally intended target recruitment, regardless of any revisions to the target made during the course of the study. Thus, the percentage could be over 100% where the target was revised upwards and achieved.

The hypothesis being tested was that there would be no change to these measures over time. We also hypothesized that there would be evidence of more timely successful completion of trials in other areas of medicine than mental health. Linear regression analyses were used to compare the measures of delay across time (based on contract start year), separately for each of the measures. Kruskal–Wallis non-parametric analysis of variance was used to compare trials in different disease areas on the three measures.

Results

148 RCTs were funded by the HTA programme between 1993 and 2007. Of these, two were continuing at the time of this review and were therefore not included. Of the remaining 146, 21 were in mental health, 19 in cancer and 106 for other disciplines. Table 1 shows the three measures by year of contract start date and Table 2 shows the same measures by disease area.

Table 1.

Comparison of RCTs by contract start date (standard deviations are in parenthesis)

Contract start date year	n	Number closed early	Award to contract start date in weeks	% time extension	Publication identified*	% of intended recruitment achieved
1995	9	1	M	6.5 (9.9)	8	87.7 (16.9)
1996	10	3	M	24.0 (19.0)	7	74.3 (13.7)
1997	6	1	M	9.4 (9.6)	5	86.9 (20.3)
1998	13	0	26.9 (11.7)	33.1 (27.8)	13	105.4 (55.6)
1999	20	0	43.8 (16.9)	37.4 (50.7)	20	82.6 (34.3)
2000	9	0	56.2 (22.6)	41.1 (35.8)	9	78.7 (30.1)
2001	13	1	72.9 (46.8)	40.6 (38.1)	11	89.7 (28.8)
2002	12	2	53.5 (27.3)	36.7 (32.3)	9	67.8 (25.4)
2003	10	0	49.7 (14.1)	49.7 (37.9)	10	73.2 (37.0)
2004	12	0	68.0 (52.3)	35.9 (34.9)	12	62.5 (37.6)
2005	7	0	57.6 (14.8)	20.9 (18.3)	6	87.1 (19.7)
2006	8	0	54.3 (34.7)	42.5 (34.9)	7	89.4 (24.4)
2007	17	0	53.7 (56.2)	41.6 (31.7)	17	100.4 (15.1)
Total	146	8	53.0 (36.2)	34.8 (34.7)	134	84.5 (32.8)

*The number of publications identified relates to the number of trials for which data of the originally planned and actual recruitment could be calculated as there was a key publication reporting the trial. For some later trials, the HTA monograph is still in preparation.

Table 2.

Comparison of RCTs across disease area (standard deviations are in parenthesis)

Disease area	n	Number closed early	Award to start date in weeks (1998 onwards only)	% time extension	Publication identified*	% of intended recruitment achieved
Mental health	21	0	52.7 (19.8)	38.8 (38.9)	20	71.1 (28.7)
Cancer	19	1	56.2 (58.1)	41.5 (40.3)	17	90.3 (29.9)
Other	106	7	52.3 (33.0)	32.7 (32.9)	97	86.1 (33.8)

*The number of publications identified relates to the number of trials for which data of the originally planned and actual recruitment could be calculated as there was a key publication reporting the trial. For some later trials, the HTA monograph is still in preparation.

Linear regression revealed no change in the percentage of the original target recruited (r² = 0.01, F[1,133] = 0.013, p = 0.909) and a non-significant increase in the number of weeks between grant award and contract start date (r² = 0.024, F[1,118] = 2.86, p = 0.093, β = 1.85). However, there was a significant increase in the percentage of time trials extended (r² = 0.031, F[1,137] = 4.41, p = 0.038, β = 1.69).

Comparing disease areas, there was no difference in the number of weeks between grant award and contract start date (χ² = 2.63, p = 0.268), or percentage of time trials extended (χ² = 0.86, p = 0.651). There was a trend level difference between the groups in the percentage of the original target recruited (χ² = 5.54, p = 0.063). When this analysis was repeated with mental health compared with all other trials (cancer, other), the difference was significant (χ² = 4.62, p = 0.032).

Discussion

Although there are some misgivings about the growth of evidence-based psychiatry (Bracken et al. 2012) there is general approval for its development as a way forward in improving services. This is recognized by the gradual introduction of streamlined regulation and governance processes, additional infrastructure to support recruitment to clinical research, and much greater funding for large multicentre studies. Large trials of complex interventions (Campbell et al. 2000) have delivered much more than they have cost, and so need to be fostered and encouraged. However, substantial barriers remain and these appear to be undermining attempts to improve the timely and successful completion of RCTs.

The priority afforded to early identification of important research questions about healthcare treatment followed by funding of adequately powered RCTs has been a major boost to the implementation of evidence-based improvements in care. We sought to determine whether this has been accompanied by a concomitant improvement in efficiency and successful completion of RCTs, and whether this has impacted on different disease areas equally.

The NIHR Health Technology Assessment programme is the largest single national research programme for the UK's NHS and has funded 152 trials at a total cost of £146 031 431 since 1993. It also has an excellent record in being praised for publishing almost all its trials, so that the database is superior to many others (Chalmers et al. 2013). The HTA Programme funds research into the effectiveness, costs and broader impact of healthcare treatment and supports a sizeable number of the non-pharmaceutical industry funded RCTs in the UK. Comparing these trials across contract start date suggested that initiatives to improve the regulation and governance pathway have not succeeded in improving the ability of trials to keep to intended timescales and targets. In fact, the opposite was true; a significantly increasing percentage time extension granted to studies was found yet there was no improvement in the percentage of the originally planned target recruited by the completion date. Although not reaching significance, time between grant award and contract start date also became longer over time, indicating that there may be an increasing burden in obtaining the necessary regulatory and governance approvals to begin a programme of research.

Our findings of a lower percentage recruitment to mental health trials is a worrying statistic. While we acknowledge that each disease area comes with its own challenges, the evidence that trials in mental health have greater difficulty in reaching their planned targets is in keeping with our personal experience of the problems we have encountered in this field. Clearly, a more facilitatory attitude is needed when supporting such studies, apart from assisting recruitment by methods such as cluster randomized trials (Barbui & Cipriani, 2011). It is not easy to overcome the many barriers that these studies face (Rendell et al. 2007; Oliver-Africano et al. 2010) but those responsible for local research governance in NHS organizations have the capacity to reduce them greatly. Local checking processes have evolved quite differently, with one organization asking for specific activities to be completed under the guise of research governance checks, despite others not undertaking the same activities. Local ‘research governance’ processes are often considerable elaborations on the actual requirements, arising from risk-aversion and anxiety over making a mistake in allowing some aspect of the research, particularly when it involves vulnerable individuals.

The necessary protection of potential and actual research participants is addressed by RECs and other national regulatory and governance bodies. Independent trial steering and data-monitoring committees also safeguard the integrity of the programme of research. It is unnecessary for local checks that repeat this process, often inexpertly as they usually lack expertise, and inevitably incurring extra cost and delay.

Good local research governance should instead focus on whether the organization can fulfil the requirements of the trial by checking local staff, resources and procedures, thereby ensuring that the trial can adhere to the ethical and safe working practices to which it has already committed. Local R&D departments should then work with researchers advising them of opportunities instead of raising barriers. These suggestions should not be taken to indicate that research governance should be lax when involved with mental health studies; rather that more sensitivity and support should be given to a population that is obtaining much less benefit from health service research.

Archie Cochrane would have been very concerned about the extra cost of both delaying the accumulation of evidence and the spending of resources on unnecessary activity. In his words, ‘if we are ever going to get the ‘optimum’ results from our national expenditure on the NHS we must finally be able to express the results in the form of the benefit and the cost to the population of a particular type of activity, and the increased benefit that would be obtained if more money were made available' (Cochrane, 1972). We are failing to live up to the Cochrane dictum in our current research system.

Financial Support

This research received no specific grant from any funding agency, commercial or not-for-profit sectors.

Conflict of Interest

None of the authors has any conflicts of interest to declare.