A taxonomy of multinational ethical and methodological standards for clinical trials of therapeutic interventions

Abstract

Background

If trials of therapeutic interventions are to serve society’s interests, they must be of high methodological quality and must satisfy moral commitments to human subjects. The authors set out to develop a clinical-trials compendium in which standards for the ethical treatment of human subjects are integrated with standards for research methods.

Methods

The authors rank-ordered the world’s nations and chose the 31 with >700 active trials as of 24 July 2008. Governmental and other authoritative entities of the 31 countries were searched, and 1004 English-language documents containing ethical and/or methodological standards for clinical trials were identified. The authors extracted standards from 144 of those: 50 designated as ‘core’, 39 addressing trials of invasive procedures and a 5% sample (N=55) of the remainder. As the integrating framework for the standards we developed a coherent taxonomy encompassing all elements of a trial’s stages.

Findings

Review of the 144 documents yielded nearly 15 000 discrete standards. After duplicates were removed, 5903 substantive standards remained, distributed in the taxonomy as follows: initiation, 1401 standards, 8 divisions; design, 1869 standards, 16 divisions; conduct, 1473 standards, 8 divisions; analysing and reporting results, 997 standards, four divisions; and post-trial standards, 168 standards, 5 divisions.

Conclusions

The overwhelming number of source documents and standards uncovered in this study was not anticipated beforehand and confirms the extraordinary complexity of the clinical trials enterprise. This taxonomy of multinational ethical and methodological standards may help trialists and overseers improve the quality of clinical trials, particularly given the globalisation of clinical research.

INTRODUCTION

The ultimate goal of research in human therapeutics is to develop treatments that change the natural history of a disease or condition for the better. When such research progresses to the point which requires trials in human subjects, contemporary society expects that clinical trials will conform to standards that assure the voluntariness and safety of the participants. In fact, many parties in the clinical trials process, including research ethics committees, focus mostly on issues such as informed consent of participants and the risk-benefit ratio of the experimental intervention.

However, the involvement of human subjects in experiments infuses an additional and inseparable requirement, one pertaining to the scientific value and validity of the research. This is set out in the second principle of the Nuremberg Code¹ and has been articulated by others in various ways. ‘…It is a prime duty of the research community to see that this sacred source [of the generosity of the will of research volunteers] is never abused for frivolous ends. For less than adequate cause, not even the freest, unsolicited offer should be accepted.’² ‘…Scientifically unsound research on human subjects is unethical in that it exposes research subjects to risks without possible benefit…’.³ ‘…Without validity the research cannot generate the intended knowledge, cannot produce any benefit and cannot justify exposing subjects to burdens or risks.’⁴

Evaluations of contemporary clinical trials and systematic reviews of trials invariably show that a large proportion have serious deficiencies in their methodological quality^5–8 and, consequently, in their ethical quality. These deficiencies have many causes, but one may relate to the fact that a large number of methodological standards are enunciated in many different documents. In addition, the documents focus either on ethics or on methods but not both. This lack of coherence compounds the difficulties that clinical trialists face in trying to design and conduct trials that adhere to principles yielding least-biased estimates of treatment effects while at the same time protecting the rights and interests of human subjects.

If the ethical and methodological aspects of a clinical trial are inextricably intertwined, then ethical and methodological standards for clinical trials ought to be presented in an integrated fashion, and the full set of integrated standards should be easily accessible to all those involved in the clinical trials enterprise. Moreover, given the globalisation of clinical research,⁹ the integrated standards should reflect substantial cross-national input. Accordingly, we set out to develop a compendium of the standards that govern clinical trials of therapeutic interventions, in which standards for the ethical treatment of human research subjects are integrated with standards for research methods.

METHODS

Definitions

A clinical trial is a prospective research study of human subjects designed to answer specific questions about biomedical or behavioural interventions (drugs, treatments and devices).¹⁰ The ethical quality of a clinical trial can be defined as the extent to which the trial protects moral commitments to respect for persons (treating individuals as autonomous agents and protecting persons who have diminished capacity), beneficence (minimising harms and maximising benefits), and justice (fairness in the distribution of the benefits and burdens of research).¹¹ The methodological or scientific quality of a trial is the extent to which its design, conduct, analysis and reporting minimises or avoids bias in the estimates of the effects of the treatment it is evaluating.¹²

Identification of sources of standards

Our first task was to identify current ethical and methodological standards for clinical trials of therapeutic interventions. We rank-ordered the countries of the world based on the number of active clinical trials listed on http://ClinicalTrials.gov/ as of 24 July 2008, and selected as source countries the 31 with >700 trials (700 was a natural cut-point: five countries had between 700 and 800, and the next-ranking country was Greece with 516). We identified documents from the source countries and their geopolitical alliances that had been issued by seven types of issuers: regulatory agencies, governmental collaboratives, public or private research sponsors, non-profit non-governmental agencies, advisory groups, professional societies and trade organisations.

To identify the standards-issuing bodies and the documents they issued, research assistants, working independently, conducted redundant internet-based searches for English-language documents using a set of common query terms (eg, ‘human’, ‘clinical’, ‘experiments’). Search strategies were specific for nations and for geopolitical alliances and were designed to go very wide in the recovery of standards-containing documents. The search strategy used with each nation is given in table 1. For geopolitical alliances, the initial search-term combinations included words denoting ‘associations’ and known issuers such as the WHO. A priori we excluded documents (or sections of documents) focusing on embryonic/gamete research, stem-cell and other genetics-only research, bio-banks, privacy in clinical medicine but not specifically related to research, and meta-analyses. The search closure date was 14 November 2008. The final document count was 1004: 844 from countries (302 from the US) and 160 from alliances.

Table 1.

Search strategy for issuers and standards-containing documents from each of the 31 nations

Step	Target	Strategy
1	Regulatory and bioethics agencies	Begin by consulting each of four documents listing regulatory and bioethics agencies by nation, collecting all documents found from referenced websites: International Compilation of Human Research Protections, compiled by the Office for Human Research Protections of the US Department of Health and Human Services (located at http://www.hhs.gov/ohrp/international/HSPCompilation.pdf). CRASH2 Regulatory Authorities (located at http://www.crash2.lshtm.ac.uk/Regulatory.htm); a listing of regulatory authorities involved in national approvals and drug import licensing. National Regulatory Agencies (located at http://www.ottosen.com/RegulationsAndGuidelines/Authorities.html); a listing of contact data for national drug regulatory agencies. National Bioethics Advisory Bodies; a listing of national bioethics commissions represented at Global Summits in London in 2000 and Brazil in 2002.
2	Regulatory and bioethics agencies	For each website discovered in Step 1, explore each resource section and all links; search each new website for relevant organisations and documents; continue searching until no new organisations or links are uncovered.
3	All issuers	Explore all national links uncovered during earlier searches of international resources; follow the links section of each new organisation’s website until no new organisations or links are uncovered.
4	Federal agencies	Search federal agencies, with special attention to agencies corresponding to the US agencies that are signatories to the Common Rule (US Agency for International Development; Central Intelligence Agency; Consumer Product Safety Commission; Departments of Agriculture, Commerce, Defence, Education, Energy, Health and Human Services, Homeland Security, Housing and Urban Development, Justice, Transportation, Veterans Affairs; Environmental Protection Agency; National Aeronautics and Space Administration; National Science Foundation; Food & Drug Administration).
5	National health services	Search national health services, in nations where they exist.
6	Professional societies and trade organisations	Search national parent medical organisation corresponding to the American Medical Association; search professional medical organisations in each of the following 33 specialties: Allergy, Anaesthesiology, Cardiovascular Disease, Colon and Rectal Surgery, Dentistry, Dermatology, Emergency Medicine, Family Medicine, Geriatric Medicine, Immunology, Internal Medicine, Medical Genetics, Medical Oncology, Neurological Surgery, Neurology, Nuclear Medicine, Obstetrics and Gynaecology, Ophthalmology, Orthopaedic Surgery, Otolaryngology, Pathology, Paediatrics, Physical Medicine and Rehabilitation, Plastic Surgery, Preventive Medicine, Psychiatry, Radiology, Sports Medicine, Surgery, Thoracic Surgery, Trauma Surgery, Urology, Vascular Surgery.

Extraction of standards from source documents

A standard was defined as a declarative sentence embodying an imperative, an obligation, or a prohibition (eg, sentences or phrases containing the words ‘should’, ‘must’, ‘need’, ‘may…only if/when’, ‘it is important that’). Permission statements were not included as standards, nor were rationales. Statements including ‘generally’, ‘usually’ and ‘only when’ were included as ceteris paribus norms. The language of an individual standard was retained verbatim, but paraphrasing (marked by ellipses or brackets) was permitted so that sentences would not lose meaning outside of context. A ‘one concept per standard’ rule was established so that future users could determine as unequivocally as possible whether a standard was met or not. Accordingly, compound sentences containing two or more discrete standards were split. Compound standards were permitted only if splitting a large or complicated conjunct or disjunct would change its meaning.

To test and refine the standards extraction process and ensure consistency, the research assistants went through independent pilot extraction processes. After all completed the three pilot documents, they met to compare results and develop methods to ensure inter-reviewer reproducibility.

To establish an order of review, we designated the source documents as ‘core’ or ‘non-core’. Documents were designated as ‘core’ based on their international and/or regional influence and the likelihood that they contained an extensive number and type of standards. Additional source documents were incorporated into the core set to ensure balanced coverage of ethical and scientific topics and comprehensive coverage of the stages of clinical trials. When a core document referenced another document that appeared to contain standards, the referenced document (or its relevant sections) was entered into the list of documents for review. After standards were extracted from each of the 50 core documents, we began assessing the non-core documents. Because of our particular interest in trials of surgical procedures,¹³ we conducted thorough searches for documents issued by surgical and anaesthesia organisations. Regardless of issuer, all non-core documents with the words ‘surgery’, ‘surgical’ or ‘device’ or the name of a specific kind of surgery in their title underwent review and extraction (N=39), as did all documents with reference to a specific operative procedure in the title and the extension of the Consolidated Standards of Reporting Trials to nonpharmacological treatment.¹⁴ We then assigned random numbers to the remaining non-core documents and drew a 5% random sample for review and extraction (N=55).

Development of the taxonomy of ethical and methodological standards

We designed a classification system for the standards. Given our goals, we wanted a system that would integrate both ethical and methodological standards. A priori we began with categories corresponding to the manner in which a clinical trial evolves: initiating, designing, conducting and analysing and reporting the results. Later on a fifth category was added: post-trial activities. As standards were extracted and classified, finer taxonomic divisions and subcategories corresponding to specific constructs were developed in an iterative fashion.

Refinement of the set of standards

Once all the standards had been extracted from the source documents and placed into their appropriate taxonomic categories, we examined the standards and refined the set by eliminating duplicates (verbatim standards from different sources), combining logically equivalent standards, and segmenting out standards that were simply administrative or procedural rather than substantive, standards about special populations, and standards not inherent to the ethical and methodological quality of a trial (eg, some pertaining to marketing approval processes). Standards that were platitudinous or so broad as to be meaningless (‘clinical trials should be scientifically sound’) were removed from the set and used as the basis of short narrative preambles that we composed for each taxonomic category.

RESULTS

Table 2 shows the number of recovered source documents by geopolitical origin and their distribution by type of issuing body. The largest number was recovered from the US, UK and global alliances, contributing 302 (30.1%), 208 (20.7%) and 77 (7.7%), respectively. Regulatory agencies issued the largest proportion of source documents overall (432/1004, 43.0%), followed by professional societies (246/1004, 24.5%).

Table 2.

Number (%) of recovered source documents by country and/or alliance and type of issuing body^*. Countries included all those with >700 active clinical trials as of 24 July 2008

	Regulatory agency	Government collaboration	Research sponsor	Non-profit non-governmental organisation	Advisory body	Professional society	Trade organisation	Total by origin
Countries
USA	119 (39.4)	0	2 (0.7)	10 (3.3)	29 (9.6)	140 (46.4)	2 (0.7)	302
Argentina	1 (50.0)	0	0	0	0	1 (50.0)	0	2
Australia	7 (33.3)	0	0	0	7 (33.3)	3 (14.3)	4 (19.0)	21
Austria	2 (66.7)	0	0	0	1 (33.3)	0	0	3
Belgium	6 (100.0)	0	0	0	0	0	0	6
Brazil	2 (40.0)	0	0	0	3 (60.0)	0	0	5
Canada	18 (50.0)	0	9 (25.0)	0	3 (8.3)	6 (16.7)	0	36
China	10 (76.9)	0	0	0	3 (23.1)	0	0	13
Czech Republic	10 (83.3)	0	0	0	0	2 (16.7)	0	12
Denmark	14 (77.8)	0	0	0	1 (5.6)	0	3 (16.7)	18
Finland	4 (50.0)	0	1 (12.5)	0	3 (37.5)	0	0	8
France	2 (14.3)	0	0	0	12 (85.7)	0	0	14
Germany	6 (42.9)	0	2 (14.3)	0	5 (35.7)	1 (7.1)	0	14
Hungary	9 (100.0)	0	0	0	0	0	0	9
India	7 (33.3)	0	0	0	12 (57.1)	0	2 (9.5)	21
Israel	1 (25.0)	0	0	2 (50.0)	0	1 (25.0)	0	4
Italy	2 (66.7)	0	0	0	1(33.3)	0	0	3
Japan	8 (72.2)	0	0	0	0	2 (18.2)	1 (9.1)	11
Mexico†	0	0	0	0	0	0	0	0
The Netherlands	15 (83.3)	0	0	0	1 (5.6)	2 (11.1)	0	18
Norway	14 (73.7)	0	0	0	3 (15.8)	1 (5.3)	1 (5.3)	19
Poland	4 (100.0)	0	0	0	0	0	0	4
Puerto Rico†	0	0	0	0	0	0	0	0
Russia	2 (50.0)	0	0	0	0	2 (50.0)	0	4
Spain	1 (50.0)	0	1 (50.0)	0	0	0	0	2
South Africa	27 (55.1)	0	10 (20.4)	7 (14.3)	1 (2.0)	4 (8.2)	0	49
South Korea	2 (66.7)	0	0	1 (33.3)	0	0	0	3
Sweden	10 (58.8)	0	6 (35.3)	0	1 (5.9)	0	0	17
Switzerland	9 (69.2)	0	0	0	0	4 (30.8)	0	13
Taiwan	5 (100.0)	0	0	0	0	0	0	5
UK	79 (38.0)	0	21 (10.1)	3 (1.4)	45 (21.6)	42 (20.2)	18 (8.7)	208
Subtotal by type	397	0	52	23	131	210	31	844
Alliances
Africa	0	2 (40.0)	1 (20.0)	0	0	2 (40.0)	0	5
Asia	1 (50.0)	0	0	0	0	1 (50.0)	0	2
Australasia	0	0	0	0	0	3 (100.0)	0	3
Eastern Europe	0	1 (100.0)	0	0	0	0	0	1
Europe	33 (52.4)	14 (22.2)	4 (6.3)	2 (3.2)	1 (1.6)	9 (14.3)	0	63
Latin America	0	1 (100.0)	0	0	0	0	0	1
North America	0	0	0	0	4 (100.0)	0	0	4
Middle East	0	3 (100.0)	0	0	0	0	0	3
Nordic Countries	0	0	0	0	0	1 (100.0)	0	1
Global	1 (1.3)	38 (49.4)	1 (1.3)	3 (3.9)	14 (18.2)	20 (26.0)	0	77
Subtotal by type	35	59	6	5	17	36	0	158
Grand total	432	59	58	28	148	246	31	1004

^*Types of issuing bodies included: regulatory agency (a body within a government which sets and enforces standards within specific fields of activity and whose approval is required in order to conduct and/or continue activity within the field. Includes national health services as in UK and Canada.); government collaboration (a group comprised of governments or agencies within governments which have agreed to work together to realise common objectives. The members of the groups work together to achieve their aims, but independent governments retain decision-making powers. Includes multi-national organisations whose members are appointed by the states to represent the states, and organisations that are subgroups of multi-national organisations.); funding agency (a public or private sponsor that funds medical research); non-profit non-governmental organisation (a nonprofit group that operates outside of institutionalised governmental or political structures and that addresses a specific agenda of issues and interests); advisory body (a body comprised of individuals (or groups) who have or exercise the power to give advice, guidance, or counsel pertaining to practices within a specific field of activity. Members of the field often regard this advice as relevant, though the advice may be non-binding or binding in effect only); professional society (a group of individuals engaged in the same professional or in related professions, whose members organise formally to attain shared objectives. These objectives often focus on the role and identity of the professional and on professional development. If a medical specialty was on the list at http://www.abms.org/Who_We_Help/Physicians/specialties.aspx, a corresponding national-level professional group was sought, for each nation); and trade organisation (a group comprised of companies that manufacture similar products or provide similar services. The group organises to promote trade and investment and to ensure fair trade and compliance with trade laws and agreements).

^†We did not seek standards-issuing documents from the following types of entities: research groups based in a single institution (e.g., the University of Aberdeen), individual private corporations, or individual provinces or states (unless the respective national regulatory agency specifically referred to them in place of national regulations).

Standards were extracted from the 144 documents listed in online appendix 1, which shows their titles, issuer and geopolitical origin. Of the 144 documents, 31 (21.5%) came from global or multinational sources. Of the 113 documents from individual nations, 76 (67.2%) came from the US or UK. Regulatory agencies contributed the largest proportion (77/144, 53.5%).

A total of 14 951 standards were extracted from the 144 documents. After platitudes and duplicates were removed and logically equivalent standards were merged, the final set included 5908 substantive standards.

The final taxonomy of standards had five major categories, each having five to 16 divisions, and each division having from 1 to 70 subcategories. The annotated taxonomy with its divisions is given in online appendix 2. Table 3 (see online) shows the number of standards in each category and division of the taxonomy. Numbers are as follows: ‘initiating a clinical trial’ has 1401 standards in eight divisions; ‘design standards for a clinical trial’ has 1869 standards in 16 divisions; ‘conducting a clinical trial’ has 1473 standards in eight divisions; ‘analysing and reporting trial results’ has 997 standards in four divisions; and ‘post-trial standards’ has 168 standards in five divisions.

Box 1 presents a selection of standards showing their taxonomic classification.

Box 1. A sample of standards for clinical trials of therapeutics, organised within their respective taxonomic locations.

Category 1: Initiating a clinical trial

• Division: Standards for research versus innovation (one of eight divisions in this category)

  • Subcategory: Distinguishing research from innovation

    Example standard:

    1.1.250.35.0 When systematic investigation is required to determine an innovative intervention’s safety and efficacy, it should be treated as clinical research needing formal consideration by a research ethics committee.

Category 2: Design standards for a clinical trial

• Division: Standards related to choice of control (one of 16 divisions in this category)

  • Subcategory: Use of placebo controls

    Example standard:

    2.8.300.100.0 The use of a placebo alone or the incorporation of a no-treatment control group is ethically unacceptable in a controlled clinical trial where another available treatment has already been clearly shown to be effective.

Category 3: Conducting a clinical trial

• Division: Standards related to ensuring compliance with protocol and conduct (one of 8 divisions in this category)

  • Subcategory: Ensuring compliance for multi-centre studies

    Example standard:

    3.1.200.100.0 To ensure that the results will be valid, the multi-centre study must be conducted in an identical way at each centre. The manner in which the protocol is implemented should be clear and similar at all centres, and procedures should be standardised as completely as possible.

Category 4: Analysing and reporting trial results

• Division: Standards for reporting results (one of four divisions in this category)

  • Subcategory: Avoiding selective reporting

    Example standard:

    4.3.100.400.0 Research professionals shall ensure the dissemination of only scientifically sound information from clinical trials and other investigations, without regard to study outcome.

Category 5: Post-trial standards

• Division: Standards for after the completion of a trial (one of five divisions in this category)

  • Subcategory: Post-trial follow-up

    Example standard:

    5.1.400.100.0 Following a subject’s participation in a trial, the investigator or institution should ensure that adequate medical care and follow-up is provided to a subject, including for any adverse events and clinically significant laboratory values, related to the trial.

DISCUSSION

The search strategy we used for this study uncovered 1004 source documents from which we extracted 5908 individual substantive ethical and methodological standards. We developed a finely subdivided and annotated taxonomy as the organising framework for the standards. The five major categories of the taxonomy, initiation, design, conduct, analysis and reporting, and post-trial activities, correspond to the sequential stages of a clinical trial. This logical and coherent organisation of clinical-trials standards will make adherence easier for investigators and application easier for oversight officials. We plan on making the compendium of standards widely available. With this many individual standards, it is doubtful that anyone engaged in clinical therapeutics trials can know and ensure they are in compliance with all of them, unless they are aided by use of a compendium like the one developed in this study. In addition, the compendium of standards is essentially a curriculum for education in methods and ethics of investigators engaged in clinical therapeutics research and of people responsible for its oversight.

Our efforts uncovered a quantity of source documents and ethical and methodological standards for clinical trials of therapeutics that we did not imagine beforehand. We believe that these findings reflect the complexity of running clinical trials in therapeutics and of overseeing them. The sheer number of requirements might also explain why different institutional review boards reviewing the same study protocol arrive at different assessments.^15–17 In addition, we found a significant number of instances where standards appear to conflict with each other—another factor that complicates interpretation and application. An example is, ‘The actual granting of consent [to participate in research] should be to someone other than the clinician primarily responsible for the patient’s care’ versus ‘Where the researcher is also the treating health professional, it should be considered whether an independent person should make the initial approach or seek consent…’ (we are currently analysing these conflicts thoroughly and will report them in a future paper).

To make the compendium of ethical and methodological standards a usable and valuable tool, it will be necessary to make it available in a searchable format so that a user can identify the subset of standards applicable to the trial question being posed and the specific modality (drug, device, procedure, etc) under investigation. Even so, the sheer number of standards must be addressed. Does this number legitimately reflect the unavoidable fact that a clinical trial is an extremely complex and multifaceted undertaking from an ethical and methodological viewpoint? Or has the number of standards proliferated as an excessive regulatory response to clinical trials with major shortcomings and at times improper motivations?

If it seems we found too many source documents and too many standards, it is useful to revisit our initial premises. First, we decided to collect standards from a wide variety of countries that had a high density of clinical trials. This seems the most justifiable course of action given the globalisation of clinical research.⁹ But even if we had decided to retrieve only those from US sources, we still would have had over 300 source documents in the pool. Second, we decided to integrate ethical with methodological standards. Since both types of standards must be satisfied in a clinical trial, it seems to us that only a resource containing both kinds of standards would prove helpful to individuals engaged in the planning or conduct of trials or in their oversight. Third, while we could have limited the standards we collected to those issued by government entities, we also included those issued by advisory groups, professional societies and trade organisations. Though not enforceable by law, such norms are exceedingly powerful motivators of behaviour and frequently foreshadow regulatory changes. Finally, we extracted standards from documents issued by regulatory agencies involved in evaluating drugs and devices for approval to be marketed. Trials conducted for marketing approval have specific agency-imposed requirements. We included documents from market-regulating agencies because we believe that a core set of ethical and methodological requirements exists regardless of why a trial is conducted, which specific modality is being evaluated, or whether the trial is initiated from academia or by a commercial entity.¹⁸

Despite the large number of standards we found, it may be an undercount, because we extracted standards only from a 5% sample of the documents that were neither designated as core nor pertaining to trials involving surgical or minimally invasive procedures. Also, because our search for source documents closed on 14 November 2008, we will have missed any new standards issued after that date.

If meeting hundreds of standards in 41 taxonomic divisions does approximate what it takes for a trial to be of high ethical and at the same time methodological quality, what does this signify for the clinical trials enterprise? Judging by the persistent deficiencies that prevail among clinical trials, it seems apparent that additional standards are not what is needed, but rather ways of achieving closer adherence to the ones already in force. The taxonomy of standards suggests some opportunities for division of labour and specialisation in discrete areas among those who oversee clinical trials, for example, specialists in design issues versus specialists in conduct issues. More importantly, the extraordinary complexity of the world of clinical trials argues for a more comprehensive and systematic education than exists today of clinical trialists and those who oversee them. Perhaps it is time to view education and training in the ethics and methods of clinical trials as an equally challenging, time-consuming and worthy a pursuit as education in the biomedical and clinical subject-matter disciplines.