Why (not) go east? Comparison of findings from FDA Investigational New Drug study site inspections performed in Central and Eastern Europe with results from the USA, Western Europe, and other parts of the world

Abstract

Since the mid-1990s, investigational sites in the countries of Central and Eastern Europe (CEE) have been increasingly utilized by pharmaceutical companies because of their high productivity in terms of patient enrolment into clinical trials. Based on the FDA’s publicly accessible Clinical Investigator Inspection List, we present an analysis of findings and outcome classifications from FDA inspections during Investigational New Drug (IND) studies and compare the results for the CEE region to those from Western European countries and the USA. Data from all 5531 FDA clinical trials inspections that occurred between 1994 (when the FDA first performed inspections in CEE) and the end of 2010 were entered into the database for comparative analysis. Of these, 4865 routine data audit (DA) inspections were analyzed: 401 from clinical trials performed in Western Europe, 230 in CEE, 3858 in the USA, and 376 in other countries. The average number of deficiencies per inspection ranged between 0.99 for CEE and 1.97 in Western Europe. No deficiencies were noted during 16.6%, 39.0%, and 21.5% of the inspections in Western Europe, CEE and USA, respectively. The percentages of inspections after which no follow-up action was indicated were 36.9% for Western Europe, 55.7% for CEE, and 44.3% for US sites. CEE was also the region with the lowest percentage of inspections that required official or voluntary action. On the basis of FDA inspection data, the high productivity of CEE sites appears to be accompanied by regulatory compliance as well as by data quality standards that are not inferior to those in Western regions.

Introduction

After the fall of the “Iron Curtain” and the disintegration of the Soviet Union during the late 1980s and the early 1990s, the countries of Central and Eastern Europe (CEE) with their total population of about 340 million people (including the European part of Russia), have become increasingly attractive for international pharmaceutical companies as sites for the conduct of clinical trials. An abundance of well educated, often treatment-naïve patients who are eager to participate in clinical trials that may offer otherwise unavailable treatment opportunities may contribute to this attractiveness. Further, the availability of numerous highly qualified and motivated clinical investigators without competing trials enhances enrollment success. Clinical research associates and monitors are usually physicians in contradistinction to those found in other regions of the world.1

Many CEE countries still adhere to a public healthcare system organization similar to that of the Soviet Union. With respect to the requirements of clinical trials this includes the availability of comprehensive, often lifelong patient records, comparatively few, but large, specialized medical centers and a tight, mainly “vertical” referral system organized according to therapeutic hierarchies, with only minimal competition for patients between the centers. The availability of patients’ medical histories is also associated with lower screening failure and premature withdrawal rates.2 Furthermore, the CEE population in general tends to be less mobile than residents of Western countries, allowing for an easier long-term follow-up.3 Based on data from 50 international phase II and III clinical trials for which enrollment data per center and per month were analyzed, it has been estimated that the average site productivity (measured as patients enrolled per site and per month) in Russia, Ukraine, and the Balkans is more than twice that found in Western Europe and in the USA.2 Consequently the number of internationally sponsored clinical trials initiated in CEE countries has more than tripled between 2002 and 2007.4

In common, CEE and Western countries have traditionally required evidence-based medicine and research. Although not a part of the International Conference on Harmonization (ICH) region, CEE states adopted ICH-GCP (Good Clinical Practice) standards during the 1990s along with Western Europe and the USA.2 In 2004 the Baltic states, Bulgaria, the Czech Republic, Hungary, Poland, Romania, Slovakia and Slovenia joined the European Union (EU) and thus came under the jurisdiction of EU legislation and guidance for clinical trials. The implementation of the EU Clinical Trials Directive5 and GCP Directive6 has informed intensive discussions of GCP principles among the stakeholders of clinical trials and improved their application to clinical trials conduct and subject protection.7

The accelerated recruitment found in CEE countries is advantageous for clinical trial sponsors only if accompanied by commensurate data quality and adherence to GCP. Herein we present an analysis of publicly accessible data compiled by the US Food and Drug Administration (FDA) during trial site inspections carried out in US Investigational New Drug (IND) studies.

Material and methods

The FDA’s Center for Drug Evaluation and Research (CDER) maintains a publicly accessible list of names, addresses, and other pertinent information gathered from GCP compliance inspections of clinical investigators who have performed studies in the context of a United States IND program since July 1977. The list is updated at quarterly intervals. The Clinical Investigator Inspection List (CLIIL) is available for download through the internet under http://www.fda.gov/Drugs/InformationOnDrugs/ucm135198.htm. A searchable version is accessible under http://www.accessdata.fda.gov/scripts/cder/CLIIL/index.cfm?fuseaction=Browse.Home (web link status of 10 May 2011).

Each investigator is identified by a unique investigator ID number. For each inspection, included are the start date and classification code that indicates the focus of the inspection whether “DA” [Data Audit – inspections for verification of study data], “FC” [For Cause – inspections for conduct of the study by the clinical investigator], and “OT” [Other]). Inspection results are classified according to one of the following three main categories:

NAI – No Action Indicated (no objectionable conditions or practices were found during the inspection);

VAI – Voluntary Action Indicated (objectionable conditions were found, but do not justify further regulatory action; any corrective action is left to the investigator to take voluntarily);

OAI – Official Action Indicated (objectionable conditions were found and regulatory and/or administrative sanctions by FDA are indicated).

For the classification of observed findings, 22 deficiency codes are available that can be assigned to an inspection in any applicable combination.

The FDA began inspecting sites in the CEE region initially in 1994. Our analysis is based on completed inspections in the FDA’s database from January 1, 1994 through December 31, 2010. The analysis is restricted to DA inspections because no FC and OT inspections performed in any CEE country are present in the database. DA inspections represent inspections performed by the FDA as a part of the agency’s routine quality assurance measures and account for about 88% of all inspections performed since 1994. Inspections with classification codes CANC (cancelled before start of inspection), MTF (case closed with memo to file), WASH (washout – no meaningful information obtained), or REF (reference), as well as database records without a classification code were excluded from the analyses.

Table 1 shows the regions and countries in which DA inspections were completed between 1994 and 2010.

Table 1.

Regions and countries in which data audit inspections were performed between 1994 and 2010

Region	Countries
Western Europe	Austria, Belgium, Denmark, Ireland, Finland, France, Germany, Greece, Italy, the Netherlands, Norway, Spain, Sweden, Switzerland, United Kingdom
Central and Eastern Europe	Bulgaria, Czech Republic, Croatia, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Russia, Slovenia, Serbia, Ukraine
North America	Canada, USA
Central America	Costa Rica, Guatemala, Mexico
South America	Argentina, Brazil, Chile, Colombia, Ecuador, Paraguay, Peru
Africa	Gabon, Ghana, Kenya (US Army), Malawi, Morocco, Nigeria, South Africa, Tanzania, Tunisia, Zambia
Middle East	Egypt, Israel
Asia	Bangladesh, China, Hong Kong, India, Japan, Korea, Malaysia, Philippines, Singapore, Taiwan, Thailand, Turkey
Australia and New Zealand	Australia, New Zealand

The inspection data were analyzed using methods of descriptive data analysis. Outcome codes are grouped by region of interest: the USA, Western Europe, Central and Eastern Europe, and all other countries from which inspection results were available.

Results

Inspections included into the analysis

The FDA’s database current through December 31, 2010 records a total of 5531 inspections completed between 1994 and 2010 inclusive. The number of inspections per full year ranged between 197 in 1994 and 411 in 2008. Figure 1 shows that between 300 and 400 inspections were completed annually during 12 out of the 15 years from 1995 through 2009, with no clear trend over time towards more or fewer inspections. The database of 31 December 2010 includes records referring to 174 inspections completed in 2010; more inspections started in 2010 and later completed may ultimately be included in subsequent database versions.

Figure 1.

Number of inspections (any type) and data audits per year completed between 1994 and 2010.

Of 5531 inspections performed between 1994 and 2010, 4865 (88.0%) were data audits (DA) and were entered into our analyses.

Table 2 shows the number of DA inspections by region as well as the countries within each region in which at least 10 inspections were conducted. Almost 80% of all DA inspections were performed in the USA. In Western Europe the countries with the highest number of DA inspections were the United Kingdom (97 inspections), Germany (70) and France (53). Together these countries accounted for 54.9% of the DA inspections performed in the Western European region between 1994 and 2010. Russia (75 inspections), Poland (59) and Hungary (21) were the countries where 67.4% of the DA inspections conducted in Central and Eastern Europe were performed. Outside these regions the countries with the largest number of DA inspections were Canada (123 inspections) followed by Argentina (35) and South Africa (33).

Table 2.

Number (%) of data audit inspections by region, as well as countries with at least 10 inspections between 1994 and 2010

Region/country	Inspections*	%*
Western Europe	401	8.2%
Austria	10	0.2%
Belgium	26	0.5%
Denmark	17	0.3%
Finland	13	0.3%
France	53	1.1%
Germany	70	1.4%
Italy	39	0.8%
The Netherlands	22	0.5%
Spain	21	0.4%
Sweden	20	0.4%
United Kingdom	97	2.0%
Central and Eastern Europe	230	4.7%
Czech Republic	15	0.3%
Croatia	14	0.3%
Hungary	21	0.4%
Poland	59	1.2%
Russia	75	1.5%
Ukraine	11	0.2%
North America	3981	81.8%
Canada	123	2.5%
USA	3858	79.3%
Other	253	5.2%
Mexico	21	0.4%
Argentina	35	0.7%
Brazil	24	0.5%
South Africa	33	0.7%
China	13	0.3%
India	18	0.4%
Total inspections	4865	100.0%

Note:

^*Numbers and percentages for countries do not add up to numbers and percentages for regions or total number and percent, because countries with less than 10 inspections are not shown, although they are included in the analyses.

Deficiencies

As shown in Table 3, deficiency codes were reported for 3299 out of the 4865 DA inspections (67.8%) in the FDA’s database. These consist of those inspections during which deficiencies were found (deficiency codes 01 through 21) and not found (deficiency code 00). The percentages of DA inspections during which no deficiencies were found (deficiency code 00) were 16.6% for Western Europe, 39.0% for the CEE region, 21.5% for US sites and 22.6% for other regions of the world. The average numbers of deficiencies (codes 01 through 21) per inspection were 1.99, 0.99, 1.59, and 1.49 for Western Europe, CEE, the USA, and for other regions of the world, respectively.

Table 3.

Number (%) of inspections by deficiencies, based on all data audit inspections for which any deficiency codes were reported (n = 3299)

Deficiency code	Western Europe	Central and Eastern Europe	USA	Other	Total
00 = No deficiencies noted	48	60	557	60	725
	16.6%	39.0%	21.5%	22.6%	22.0%
01 = Records availability	2	2	30	5	39
	0.7%	1.3%	1.2%	1.9%	1.2%
02 = Failure to obtain and/or document subject consent	15	3	124	15	157
	5.2%	1.9%	4.8%	5.7%	4.8%
03 = Inadequate informed consent form	61	10	521	23	615
	21.0%	6.5%	20.1%	8.7%	18.6%
04 = Inadequate drug accountability	56	13	383	36	488
	19.3%	8.4%	14.8%	13.6%	14.8%
05 = Failure to follow investigational plan	193	48	1316	138	1695
	66.6%	31.2%	50.8%	52.1%	51.4%
06 = Inadequate and inaccurate records	148	61	950	123	1282
	51.0%	39.6%	36.7%	46.4%	38.9%
07 = Unapproved concomitant therapy	4	0	27	2	33
	1.4%	0.0%	1.0%	0.8%	1.0%
09 = Unapproved use of drug before IND submission	0	0	1	0	1
	0.0%	0.0%	0.0%	0.0%	0.0%
10 = Inappropriate delegation of authority	2	0	9	0	11
	0.7%	0.0%	0.3%	0.0%	0.3%
11 = Inappropriate use/commercialization of IND	0	0	3	0	3
	0.0%	0.0%	0.1%	0.0%	0.1%
12 = Failure to list additional investigators on 1572	1	0	10	3	14
	0.3%	0.0%	0.4%	1.1%	0.4%
13 = Subjects receiving simultaneous investigational drugs	0	0	3	0	3
	0.0%	0.0%	0.1%	0.0%	0.1%
14 = Failure to obtain or document IRB approval	2	0	56	6	64
	0.7%	0.0%	2.2%	2.3%	1.9%
15 = Failure to notify IRB of changes, failure to submit progress reports	4	3	115	8	130
	1.4%	1.9%	4.4%	3.0%	3.9%
16 = Failure to report adverse drug reactions	47	9	313	32	401
	16.2%	5.8%	12.1%	12.1%	12.2%
17 = Submission of false information	0	0	5	0	5
	0.0%	0.0%	0.2%	0.0%	0.2%
18 = Other	35	4	246	5	290
	12.1%	2.6%	9.5%	1.9%	8.8%
19 = Failure to supervise or personally conduct the clinical investigation*	1	0	4	0	5
	0.3%	0.0%	0.2%	0.0%	0.2%
20 = Failure to protect the rights, safety, and welfare of subjects*	0	0	2	0	2
	0.0%	0.0%	0.1%	0.0%	0.1%
21 = Failure to permit FDA access to records*	0	0	1	0	1
	0.0%	0.0%	0.0%	0.0%	0.0%
Total	290	154	2590	265	3299

Note:

^*Codes 19 through 21 became effective only by October 1, 2005.

The five most frequently reported deficiencies in the total sample and in each region examined individually were failure to follow the investigational plan (code 05, noted in 51.4% of all DA inspections for which deficiency codes were reported), inadequate or inaccurate records (code 06, 38.9%), inadequate informed consent form (code 03, 18.6%), inadequate drug accountability (code 04, 14.8%), and failure to report adverse drug reactions (code 16, 12.2%). Together these five deficiency codes account for 84.6% of all deficiencies uncovered at US sites (3483 of 4119 deficiencies) and for 89.1% of all deficiencies at non-US sites (998 of 1120).

When comparing the percentage of inspections with deficiencies of a certain kind in Western European countries and in the CEE region, 12 of the 20 codes 01 through 21 showed higher deficiency rates in inspections conducted in Western Europe whereas 2 showed higher rates in the CEE region. For 6 codes (03, 04, 05, 06, 16, 18), the differences exceeded 5% in favor of CEE, the larger differences occurring for “ Failure to follow the investigational plan” (05; rate difference: 35.4%), “Inadequate informed consent form” (03; 14.5%) and “Inadequate and inaccurate records” (06; 11.4%).

The overall deficiency rate of DA inspections at US sites was lower than in Western Europe and higher than in the CEE region. Rate differences in favor of CEE were determined for 16 out of the 18 codes indicating the presence of deficiencies whereas 2 codes showed lower rates for US sites. For five codes (03, 04, 05, 16, 18), the deficiency rate difference in favor of CEE exceeded 5% with the larger differences for “Failure to follow the investigational plan” (code 05, rate difference 19.6%), “Inadequate informed consent form” (03; 13.6%), and “Other” (18; 6.9%).

Indicated action

Of the 4865 DA inspections assessed, 44.2% required no action and in 53.5%, voluntary action with or without requested response was indicated (Table 4). In 2.3% of the inspections, inspectors recommended official action with regulatory and/or administrative sanctions by the FDA.

Table 4.

Number (%) of inspections by indicated action

	Western Europe	Central and Eastern Europe	USA	Other	Total
NAI	148	128	1711	164	2151
	36.9%	55.7%	44.3%	43.6%	44.2%
VAI	235	100	2061	206	2602
	58.6%	43.5%	53.4%	54.8%	53.5%
OAI	18	2	86	6	112
	4.5%	0.9%	2.2%	1.6%	2.3%
Total	401	230	3858	376	4865

Abbreviations: NAI, no action indicated; VAI, voluntary action indicated; OAI, official action indicated.

Among the regions presented in Table 4, Western Europe showed the highest percentage of inspections followed by initiation of official action (4.5%) and the lowest percentage of inspections where no action was required (36.9%). In contrast, less than 1% of the cases in the CEE region were classified as OAI inspections and more than 55% of the inspections indicated no further action. The difference between the inspection outcome classifications (NAI/VAI/ OAI) in Western Europe and the CEE region was descriptively significant (two-sided χ²-test, P < 0.001).

Warning letters were issued as a result of the inspection (classification code OAIW) in 4 cases in the USA (0.1% of all inspections at US sites) and after 1 inspection at a CEE site (0.4%).

Discussion

In a paper published in 2004, Platonov and Varshavsky analyzed the FDA’s inspections database from 1994 to 2004, with a total of 3178 inspections performed worldwide.8 Their results showed the percentage of inspections where no action was indicated (NAI) was 32%, 49% and 38% for Western Europe, the CEE region and the USA compared respectively to 37%, 56% and 44% in our analysis through year 2010. Assuming that the FDA’s standards applied during inspections have not become less rigorous during the second half of the first decade of the 21st century, the increase in the percentage of NAI inspections may reflect GCP awareness and the successful implementation of quality control and quality assurance measures by investigational sites, sponsors and contract research organizations during recent years. In the CEE region, more than half of the FDA’s inspections did not require any follow-up action. The sequential findings show a decrease in the percentage of inspections where official action was indicated in Western Europe from 7% in their report8 to 4.5% in our analysis while the percentages in CEE and the USA remained at low levels below 1% and at 2%, respectively.

A comparison of indicated action by region suggests relative superiority for CEE over the USA and even more so over Western Europe. The rate difference for NAI was 17% in Platonov’s and Varshavsky’s report8 and 19% in our analysis. These observations are paralleled by the results of the analysis of deficiency codes in which the percentage of inspections with no deficiencies in the CEE region (39.0%) was more than twice as high as in Western Europe (16.6%) and almost twice that of the USA (21.5%).

It is noteworthy that the top five deficiency codes found in DA inspections (failure to follow the investigational plan, inadequate or inaccurate records, inadequate informed consent form, inadequate drug accountability, and failure to report adverse drug reactions) were the same in all regions in our analysis and that of Platonov and Varshavsky.8 On the other hand, proportionate differences between the regions are noted. The most frequent deficiency in all regions outside CEE was failure to follow the investigational plan; the most frequent finding in CEE sites was inadequate or inaccurate records. The percentage of inspections in CEE countries where inadequate or inaccurate records were an issue (39.6%) was still comparable with US sites (36.7%) and substantially lower than in Western Europe (51.0%) or in other regions of the world (46.4%).

A limitation of this type of numerical analysis is that the impact of these relative differences in deficiency type on overall data quality remains speculative, such as whether inadequate/inaccurate records has less impact than failure to follow investigational plan. Minor protocol deviations versus protocol violations, for example, could be a relevant distinction. Further, the FDA Clinical Investigator Inspection List from which this data is drawn for comparison does not identify the particular trial reviewed for inspection nor how frequently a codified deficiency indentified during an inspection occurred during the trial.

Another limitation inherent in our analysis is that no data is available regarding how investigational sites at which DA inspections were performed were selected and whether the same quality management standards were applied during all inspections. Since regulatory requirements regarding quality standards in clinical trials as well as the FDA’s collective experience in conducting inspections have evolved over time, it is reasonable to consider that the standards applied by the inspectors during their work at a study site may not always have been exactly the same. On the other hand, we could not find any evidence that the FDA’s data may have been biased towards an application of stricter or more liberal standards in any particular region of the world where clinical trials in IND programs are conducted.

Further potentially confounding factors should raise caution in the interpretation of such raw numerical comparisons from the FDA data source. One cannot assure that the sites among the regions were comparable in terms of enrollment numbers for the inspected trials. While countries typically maintain data on relative proportion of clinical trial phases performed within their sovereignties, we have not compared our designated regions on this level nor have we compared ratios of inspections in relation to clinical phase. Such differences could have an effect on the proper interpretation of the numerical comparisons. Higher complexity of a clinical protocol may create further risk for conduction error although, in general, trials performed in CEE that would come under scrutiny by the FDA are part of worldwide or multiregional studies.

Considering that warning letters sent by the FDA reflect serious breaches of GCP rules, the respective percentages of CEE countries (1/230; 0.4%) and the USA (4/3858; 0.1%), suggest a perspective that runs counter to the more general conclusions drawn from the reporting data comparisons. Yet, the absolute numbers of warning letters being very small, a comparison of percentages may exaggerate the apparent difference.

These limitations on our report are in some measure mitigated by the support of its conclusions found in the results of benchmarking analyses that compared the efficiency and productivity of investigator sites in different countries and regions. The number of queries generated per subject was used as an indicator of data quality in two independent studies based on multinational clinical trials.9,10 Both analyses showed that there were substantially fewer queries in CEE countries than in Western Europe or North America and Australia/New Zealand.

Despite these data from the FDA, media reports suggest continued concern by clinicians in the West, perhaps more so in the USA, regarding the acceptance of data from the rest of the world including Eastern Europe. Part of this concern regarding data validity may simply arise from home bias as occurs with financial investing in diverse markets. Nonetheless, fear remains that budgetary challenges constrain the FDA from auditing an adequate portion of clinical trial participation, especially abroad, amid its vast oversight duties in pharmaceutical development, drug and device manufacturing and food safety. Even in 2011, nearly 80% of FDA clinical trial site audits occurred in the USA while the majority of enrollees in NDA submission clinical trials were ex-USA (Table 2).11

Any evidence-based search for validity is threatened when the amount of data is limited. Though moot, our report codifies the data available. Mitigating the residual uncertainties will require more time and data. In the world of multinational pharmaceutical companies attempting to satisfy global markets, clinicians everywhere face the same dilemma: most of the data submitted for registry approval is from somewhere else. Clinical scientists in the USA worry whether a drug that appears efficacious and safe in foreign subjects will perform as well in the American populace.12 It should bear some consolation to clinicians in the USA, a principality with epic diversity, that abundant East European ancestry in the US population as a result of generations of immigration should mitigate the concerns over genetic differences in study populations.

Results of our analysis of FDA inspection findings suggesting superior performance, as with any conclusions drawn from small data sets, may be regarded as more credible when accompanied by plausible explanations. The authors speculate that the recent Soviet history of more central political and economic control that encompasses the national healthcare system may render investigators more accustomed to surveillance and conformity so essential to quality control in the clinical trial setting. East European investigators readily acknowledge that the income earned in clinical trials has a positive impact on staff motivation and the financial health of their divisions. Loss of the economic, professional and social opportunities provided by participation in clinical trials through poor performance may be relatively more palpable in general than the impact on western investigators. The establishment of ICH GCP in the 1990s that opened Western regulatory authorities to East European data has allowed East European clinicians to connect more intimately with western colleagues and technology. The value of this collegiality still appears to play a major role in the motivation for East European sites to sustain quality assurance. Underlying this purpose, highly trained physicians typically continue to fill the roles of site clinical coordinators and contract research associates as well as the roles of investigators and subinvestigators. These strata of professionals may represent the key to the QA differences in Eastern Europe indicated by FDA inspection reporting. Lastly, as the later entrants into the process of new medical product clinical development, we suspect a natural desire by highly qualified Eastern European investigators to demonstrate noninferiority through extra effort expended on assuring clean data.

In conclusion, according to the FDA’s Clinical Investigator Inspection List, investigator site DA inspections in the CEE region led to fewer findings regarding protocol compliance and record keeping, and also reported fewer issues with informed consent documents and procedures, inadequate drug accountability and failure to report adverse drug reactions than inspections performed in Western Europe, the USA or other parts of the world. CEE was also the region with the lowest percentage of inspections that required official or voluntary action. While unresolved confounding factors relevant to regional comparisons of the FDA’s inspection reporting data may exist, the data at large suggests that the high productivity of CEE sites is accompanied by regulatory compliance and data quality standards that are not inferior to those in Western regions.

List of abbreviations

CANC

Cancelled (before start of inspection)

CDER

Center for Drug Evaluation and Research (part of FDA)

CEE

Central and Eastern Europe

CLIIL

Clinical Investigator Inspection List

DA

Data Audit

EU

European Union

FC

For Cause (Inspection)

FDA

Food and Drug Administration

GCP

Good Clinical Practice

ICH

International Conference on Harmonization

ID

Identification

IND

Investigational New Drug

MTF

Memo to File (case closed)

NAI

No Action Indicated

OAI

Official Action Indicated

OAIW

Official Action Indicated – Warning Letter

OT

Other (Inspection)

VAI

Voluntary Action Indicated

WASH

Washout (no meaningful information obtained)