Anti-infective medicine quality: analysis of basic product quality by approval status and country of manufacture

Abstract

Background

Some medicines for sale in developing countries are approved by a stringent regulatory authority (SRA) or the World Health Organization (WHO) prequalification program; many of these are global brands. This study ascertains whether medicines approved by SRAs or the WHO perform better in simple quality tests than those that have not been approved by either.

Methods

Over the past 4 years, 2652 essential drugs (products to treat malaria, tuberculosis, and bacterial infections) were procured by covert shoppers from eleven African cities and eight cities in a variety of mid-income nations. All samples were assessed using the Global Pharma Health Fund eV Minilab^® protocol to identify whether they were substandard, degraded, or counterfeit.

Results

The failure rate among SRA-approved products was 1.01%, among WHO-approved products was 6.80%, and 13.01% among products that were not approved by either. African cities had a greater proportion of SRA- or WHO-approved products (31.50%) than Indian cities (26.57%), but they also experienced a higher failure rate (14.21%) than Indian cities (7.83%). The remainder of cities tested had both the highest proportion of approved products at 34.46% and the lowest failure rate at 2.70%. Products made in Africa had the highest failure rate at 25.77%, followed by Chinese products at 15.74%, Indian products at 3.70%, and European/US products, which failed least often, at 1.70%. Most worrying is that 17.65% of Chinese products approved by the WHO failed.

Conclusion

The results strongly indicate that approval by either an SRA or the WHO is correlated with higher medicine quality at a statistically significant level. The comparatively high failure rates among WHO-approved products suggest there may be some weakness in post-marketing surveillance of these products, especially of Chinese-made WHO-approved products. The discrepancy between the failure rate of WHO-approved products from India (2.39%) and China (17.65%) is cause for concern. It is possible that more of the failures originating in China are counterfeit products, but this cannot be ascertained without greater help from the manufacturers themselves.

Introduction

A stringent regulatory authority (SRA), as defined by the World Health Organization (WHO), is a national drug regulatory authority that is a member, observer, or associate of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use.¹ Members include regulatory bodies and research-based industry in the European Union, Japan, and the USA, and International Conference on Harmonization observers from WHO, Health Canada, and the European Free Trade Association represented by SwissMedic.² The US Food and Drug Administration (FDA) and other major regulatory agencies, such as SwissMedic and the European Medicines Agency, are SRAs and the products they approve are assumed to be of the highest quality. As established in the literature, drug registration by national regulatory agencies of emerging markets is an important determinant of quality.^3,4 However, most regulators cannot approach the sophistication in assessing production processes and product quality SRAs possess.⁵ Hence, companies that wish to sell their products widely, particularly to tender for donor programs that demand high-quality products or in wealthy markets, will want their products registered by an SRA.

Since the registration process by SRAs can be expensive, manufacturers of products that are unlikely to be sold in the USA or other wealthy markets have often balked at the cost of such registration. The WHO’s prequalification program (PQP) is intended to give international procurement agencies the choice of a range of quality products⁶ and it assists manufacturers in tendering a higher level of quality approval without subjecting them to costs that would make their sales unprofitable. Through WHO’s PQP, the WHO approves the production processes and products for these companies at far lower cost.⁶

In 2010, some of the authors of this study analyzed the relationship between registration status and basic quality for drugs procured across the developing world.⁷ Expanding on that analysis, this study compares product approval, both by an SRA and through the WHO’s PQP, with medicine quality. The data are broken down both by the location of purchase and by alleged location of product manufacture.

Materials and methods

Over the past 4 years, 2652 drug samples were procured by covert shoppers from private pharmacies and drug stores in 19 cities across 17 developing and mid-income countries, following previous methodology.^3,7–10 Samplings took place in eleven African cities, three Indian cities, and five mid-income cities: Sao Paolo, Moscow, Bangkok, Istanbul, and Beijing. Although the pharmacies in these cities were considerably different from each other, every effort was made to ensure that the sampling protocol was as similar as possible to provide comparable results. Even following the same protocol, it is quite possible that with different covert shoppers in each of the cities unknown biases will have crept into the data.

All drug samples were from WHO’s essential drug list; samples included antimalarials, antibiotics and anti-mycobacterials. When the collections began (in six of the eleven African cities), the primary aim was to only analyze antimalarial drugs; therefore, the samples from African cities are biased toward antimalarials, with fewer antibiotics and anti-mycobacterials procured. Additionally, no antimalarials were available for purchase from the cities of Istanbul, São Paolo, and Moscow.

Samples were assessed using the Global Pharma Health Fund eV Minilab^® protocol (Frankfurt am Main, Germany) to identify substandard, degraded, or counterfeit products. As this study is largely a re-analysis of data compiled in other studies described elsewhere, the reader is referred to those publications for more detailed methods.^7–10

Approval status was determined after quality testing to eliminate possible bias.

Results

Out of a total of 2652 samples, 395 products had been approved by an SRA, 412 were approved by the WHO (but not an SRA), and the remaining 1845 samples were not approved by either (see Table 1). Overall, less than a third of the products procured had been approved either by an SRA or through the WHO (30.43%, 807/2652). The failure rate among SRA-approved products was 1.01% (4/395), 6.80% (28/412) among WHO-approved products, and 13.01% (240/1845) among products that were not approved by either. Aggregating the samples that had been approved through either process (SRA or WHO), these drugs were over three times less likely to fail (3.97%) than those not approved (13.01%).

Table 1.

Testing results (failures recorded) by location of purchase and approval status

Location of purchase	Approval status
	Total	Failed	% fail
SRA
African cities	239	4	1.67%
Indian cities	14	0	0.00%
Cities of other studied countriesa	142	0	0.00%
Total	395	4	1.01%
WHO
African cities	191	18	9.42%
Indian cities	210	10	4.76%
Cities of other studied countriesa	11	0	0.00%
Total	412	28	6.80%
Neither SRA or WHO
African cities	935	172	18.40%
Indian cities	619	56	9.05%
Cities of other studied countriesa	291	12	4.12%
Total	1845	240	13.01%
Total
African cities	1365	194	14.21%
Indian cities	843	66	7.83%
Cities of other studied countriesa	444	12	2.70%
Total	2652	272	10.26%

Note:

^aCountries include Thailand, China, Turkey, Russia, and Brazil.

Abbreviations: SRA, stringent regulatory authority; WHO, World Health Organization.

The five mid-income cities – Bangkok, Istanbul, Sao Paulo, Moscow, and Beijing – had the highest rates of SRA-or WHO-approved products on sale at 34.46% (153/444) and the lowest failure rate at 2.70% (12/444) (see Table 1). African cities had a greater proportion of approved products (31.50%, 430/1365) than Indian cities (26.57%, 224/843). However, despite higher approval rates, the overall failure rate in African cities was highest at 14.21% (194/1365). While Indian cities had the smallest proportion of approved products (26.57%, 224/843), Indian drugs failed less frequently (7.83%, 66/843) than their African counterparts (14.21%, 194/1365).

To conduct a statistical assessment of the strength of SRA or WHO approval in explaining the difference in failure rates, the data were regressed and their coefficients recorded (see Table 2). The regression revealed that a drug’s approval by an SRA is two and a half times stronger than WHO approval in explaining which drugs were of good quality. WHO approval was weakly correlated with improved quality – lack of approval by either was, as expected, inversely correlated with quality.

Table 2.

Linear regression: registration status on success rate (non-failure rate)

	N	F (1, 2650)	P > F	R2	Coefficient	Standard error	95% CI
SRA	2652	43.76	0.0000	0.0016	0.1086	0.0164	0.0764, 0.1408
WHO	2652	6.36	0.0118	0.0024	0.0410	0.0163	0.0091032, 0.072831
Neither SRA or WHO	2652	50.80	0.0000	0.0019	−0.0904	0.0127	−0.1153076, −0.0655489

Abbreviations: SRA, stringent regulatory authority; WHO, World Health Organization.

The alleged location of manufacture provided interesting results. While every effort was made to establish the real location of manufacture, it was not possible to determine this with 100% accuracy because some products could have been well-packaged counterfeits. Products allegedly manufactured in Africa had the highest failure rate at 25.77% (142/551), followed by Chinese products at 15.74% (79/502), Indian products at 3.70% (44/1188), and European/US products, which failed least often, at 1.70% (7/411) (see Table 3). Within these data, there is a notable discrepancy between the failure rates of WHO-approved drugs of Indian and Chinese origin, the only two manufacturing regions for which we have WHO-approved drug data. It is notable that 17.65% (21/119) of Chinese products approved by the WHO failed quality tests (this was worse than the Chinese products not approved by the WHO). In contrast, only 2.39% of Indian WHO-approved drugs failed testing. The authors did not systematically contact manufacturers to confirm whether the drugs were substandard versus counterfeit, as previous attempts to do this were only partly successful.

Table 3.

Testing results (failures recorded) by alleged location of manufacture and approval status

Alleged location of manufacture	Approval status
	Total	Failed	% fail
SRA
Africa	0	0	N/A
India	12	0	0.00%
China	0	0	N/A
EU/USA	383	4	1.04%
Total	395	4	1.01%
WHO
Africa	0	0	N/A
India	293	7	2.39%
China	119	21	17.65%
EU/USA	0	0	N/A
Total	412	28	6.80%
Neither SRA or WHO
Africa	551	142	25.77%
India	883	37	4.19%
China	383	58	15.14%
EU/USA	28	3	10.71%
Total	1845	240	13.01%
Total
Africa	551	142	25.77%
India	1188	44	3.70%
China	502	79	15.74%
EU/USA	411	7	1.70%
Total	2652	272	10.26%

Abbreviations: EU, European Union; SRA, stringent regulatory authority; WHO, World Health Organization.

Discussion

Previous research^11–13 has established that wealthier cities have better quality drugs and this research confirms that finding. Following the standard pattern, in this study, drugs procured in Africa were of poorer quality than those procured in mid-income nations.

Unique to this research is the strong indication that drugs approved by either SRAs or the WHO outperform drugs that have not been approved by either. However, the average improvement of WHO prequalification over those lacking approval masks a notable difference between WHO-approved Indian- and Chinese-made drugs. Chinese WHO-approved drugs fail seven times more often than Indian WHO-approved drugs.

There are several possible reasons for this, including poor storage in transportation of Chinese-made products, poor manufacturing of at least some of these products, and high-quality counterfeiting of some products. While this latter cause may account for several of the product failures, most failures had considerable active ingredient, which is rare in most counterfeit products. However, regardless of the cause, the dangerously high failure rates indicate that the WHO should improve post-marketing surveillance of the products it approves, especially Chinese products. It is possible that some manufacturers of WHO-approved products are lowering standards in subsequent production, particularly where post-marketing surveillance is weak.

This study lends support to efforts to have products approved by an SRA or the WHO before they are sold in emerging markets. However, it also highlights the need for regulatory authorities in emerging markets to improve their control over the many products in their markets. One way to do this is to encourage manufacturers to target developing markets by providing preferential market access – like the FDA does through its priority review voucher¹⁴ – for companies that gain FDA approval for a drug for a neglected tropical disease. The manufacture is granted a priority review voucher for use at a later date to expedite the certification process for another drug of its choice, such as a profitable new product for the wealthy market, thereby increasing a company’s market share and revenue.

Donors can also help encourage proper manufacturing and oversight among companies and regulators in developing nations. One such donor program is the US Government’s Promoting the Quality of Medicines in Developing Countries project. It aims to help regulators in mid-income and developing countries register products, conduct post-marketing surveillance, form consistent good manufacturing practice requirements, and even works to assist some companies in least developed countries to achieve good manufacturing practice status.¹⁵ Given the data in this study, it is important to recognize the value of this program and others like it.

Conclusion

As expected, products approved by SRAs or the WHO perform better than products not approved in emerging markets; however, the finding that seven times as many Chinese-made WHO-approved products failed quality control as WHO-approved Indian products, is of concern and should be investigated further. Nations importing WHO-approved products, especially those made in China, should step-up post-marketing surveillance to identify the reasons for the failures and strengthen their monitoring process.