Cost savings of paper analytical devices (PADs) to detect substandard and falsified antibiotics: Kenya case study

Abstract

Background:

Over 10% of antibiotics in low- and middle-income countries (LMICs) are substandard or falsified. Detection of poor-quality antibiotics via the gold standard method, high-performance liquid chromatography (HPLC), is slow and costly. Paper analytical devices (PADs) and antibiotic paper analytical devices (aPADs) have been developed as an inexpensive way to estimate antibiotic quality in LMICs.

Aim:

To model the impact of using a rapid screening tools, PADs/aPADs, to improve the quality of amoxicillin used for treatment of childhood pneumonia in Kenya.

Methods:

We developed an agent-based model, ESTEEM (Examining Screening Technologies with Economic Evaluations for Medicines), to estimate the effectiveness and cost savings of incorporating PADs and aPADs in amoxicillin quality surveillance in Kenya. We compared the current testing scenario (batches of entire samples tested by HPLC) with an expedited HPLC scenario (testing smaller batches at a time), as well as a screening scenario using PADs/aPADs to identify poor-quality amoxicillin followed by confirmatory analysis with HPLC.

Results:

Scenarios using PADs/aPADs or expedited HPLC yielded greater incremental benefits than the current testing scenario by annually averting 586 (90% uncertainty range (UR) 364–874) and 221 (90% UR 126–332) child pneumonia deaths, respectively. The PADs/aPADs screening scenario identified and removed poor-quality antibiotics faster than the expedited or regular HPLC scenarios, and reduced costs significantly. The PADs/aPADs scenario resulted in an incremental return of $14.9 million annually compared with the reference scenario of only using HPLC.

Conclusion:

This analysis shows the significant value of PADs/aPADs as a medicine quality screening and testing tool in LMICs with limited resources.

Introduction

Substandard and falsified medicines harm population health and cause economic damage due to increased mortality and morbidity. The World Health Organization (WHO) estimates that 10.5% of medicines in low- and middle-income countries (LMICs) are substandard or falsified. ¹ A recent meta-analysis similarly found this prevalence at 13.6% across essential medicines in LMICs, where antibiotics were identified to be one of the most common medicines tested and found to be substandard or falsified. ² Substandard medicines are defined by the WHO as “authorized medical products that fail to meet either their quality standards or specifications, or both,” whereas falsified medicines are defined as “medical products that deliberately or fraudulently misrepresent their identity, composition, or source.” ³ Poor-quality medicines are detrimental to the health and well-being of patients, increasing the duration of illness and the risk of death and disability. Moreover, substandard and falsified medicines result in economic losses from costs of ineffective treatment as well as lost wages of sick individuals and/or their caregivers.^1,4 Substandard and falsified medicines can also contribute to the development of antimicrobial resistance (AMR). ¹ Finding poor-quality medicines and removing them from the market is a global public health need.

Laboratory assays such as high-performance liquid chromatography (HPLC) can detect substandard and falsified medicines with high sensitivity and specificity. However, the costs and complexity associated with HPLC can be substantial, hindering monitoring and surveillance of poor-quality medicines in LMICs. HPLC requires stocking reagents to test each drug type and can only be performed by highly trained technicians. Use of compendial methods developed by pharmacopeia organizations (US Pharmacopeia (USP), British Pharmacopeia, International Pharmacopeia) in a certified pharmaceutical laboratory is currently required by most regulatory agencies to justify regulatory actions such as recall of substandard or falsified products. Together, HPLC with compendial analysis requires months to complete, during which time the substandard or falsified medicines they aim to identify continue to be utilized by patients.

Recently, paper analytical devices (PADs) have been developed to identify falsified antibiotics. ⁵ To detect falsified products, the user wipes powder from the dry formulation (capsule or tablet) across 12 lanes defined on the paper by wax printing. A small portion (0.5-1 mg) of the sample is deposited into each of the 12 lanes, which contain dried reagents. When the bottom edge of the PAD is placed into water, it moves up the paper lanes through capillary action. The reagents are carried to the sample and perform 12 color reactions that probe the functional groups and materials present in the sample. ⁶ This reveals a color barcode that can be compared with standard patterns to identify active pharmaceutical ingredients (APIs) and excipients such as chalk, talcum powder, or starch, which are often used as adulterants in substandard or falsified medicines. ⁶ The sensitivity and specificity of the PAD to detect falsified amoxicillin are 100% and 100%, respectively.^6,7

In addition, for beta-lactam antibiotics, an antibiotic paper analytical device (aPAD) has been developed to identify substandard antibiotics. Specifically, aPADs measure the API concentration to see if the drug quantity is within pharmacopeial standards (usually 90%–110%). The aPAD is based on USP method <425>, which requires base degradation of a beta-lactam sample, neutralization, addition of excess iodine, and a back-titration.^7,8 By adding drops of the sample to the aPAD card and counting the number of dots that turn blue, the API concentration is measured with an accuracy of ±5%. The sensitivity and specificity of the aPAD to detect substandard amoxicillin (<90% API content) are 97% and 92%, respectively. ⁹ Together, the PAD and aPAD can detect substandard and falsified antibiotics inexpensively ($3 each) with high accuracy, and are easy to use.

This study estimated the costs and benefits of using PADs and aPADs to detect the quality of amoxicillin samples in Kenya. Amoxicillin is a common antibiotic available in most Kenyan pharmacies, and home treatment with this drug is recommended as the national standard of care for pneumonia. ¹⁰ Kenya has a WHO prequalified laboratory with capacity for HPLC at its National Quality Control Laboratory (NQCL) and is developing another certified pharmaceutical analysis laboratory at the Kenyan Pharmacy and Poisons Board. However, the demands on these laboratories are high, and testing hundreds of samples is an added burden. In this study, we evaluated the impact of using an inexpensive screening technology to reduce the demand for HPLC resources and expedite detection of substandard and falsified medical products. Our goal is to inform national medicine regulatory authorities, policy makers, and international organizations of how screening technologies such as the PAD/aPAD could help reduce the burden of substandard and falsified antibiotics.

Methods

We developed an agent-based model, ESTEEM (Examining Screening Technologies with Economic Evaluations for Medicines), to estimate the costs, benefits, and cost savings of using PADs and aPADs to screen for substandard and falsified amoxicillin. This agent-based model used the Python programming language to simulate amoxicillin utilized for treatment of pneumonia in children under five in Kenya. Modeling the drugs as agents, we simulated the process of sampling, testing, and removing failed amoxicillin from the market. The model setup and structure are detailed in the Supplemental Appendix 1.1. Three scenarios were compared: (1) HPLC testing representing the current baseline method of identifying substandard or falsified medicines; (2) expedited HPLC testing where a quarter of samples were tested at a time to accelerate reporting of results; and (3) using PADs and aPADs with HPLC, which involved rapid screening with PADs and aPADs followed by HPLC confirmatory testing. Figure 1 depicts the flow of the three modeled scenarios.

Figure 1.

Framework depicting three medicine quality testing scenarios.

HPLC: high-performance liquid chromatography; PADs/aPADs: paper analytical devices/antibiotic paper analytical devices.

HPLC scenario

In the HPLC scenario, we simulated that it would take 12 months (months 4–15) to test all 520 samples (40 samples each across 13 brands of amoxicillin based on lot-quality assurance sampling, LQAS) using HPLC, with sensitivity and specificity at 100% (Figure 1). Samples identified as substandard or falsified were sent to full compendial analysis to determine why the product failed the assay and to support regulatory actions for removal. Pharmacopeial analysis was modeled with a 3-month time frame (months 16–18). When a modeled sample was identified as substandard or falsified, the entire batch was assumed to be removed from the market in the month after pharmacopeial analysis.

Expedited HPLC scenario

In the expedited HPLC scenario, testing was accelerated compared with regular use of HPLC (Figure 1). The samples were broken up into four random groups, where each group separately underwent HPLC testing, pharmacopeial analysis, and if needed, removal from the market. HPLC testing of a quarter of the sample was assumed to take 3 months (months 4–6, 7–9, 10–12, and 13–15 for each group), followed by the next 3 months of pharmacopeial analysis. Failed batches were removed from the market in months 10, 13, 16, and 19. Instead of releasing the testing results for all 520 samples in month 15 in the HPLC scenario, the expedited HPLC scenario released partial results faster, resulting in earlier removal of substandard and falsified amoxicillin.

Paper analytical devices (PADs) scenario

The PADs and aPADs scenario used PADs to identify falsified products and aPADs to find substandard amoxicillin during screening. PADs and aPADs were applied for all 520 samples in month 4 to screen for both presence and amount of amoxicillin. Samples that failed PADs or aPADs were subsequently sent to HPLC testing for confirmatory analysis. For quality control purposes, 10% of samples that passed both PAD and aPAD screenings were also submitted for HPLC testing. Utilizing PADs and aPADs decreased the number of samples requiring HPLC analysis, where HPLC testing was estimated to take only 1 month in this scenario (month 5). Upon HPLC analysis, samples confirmed as substandard or falsified were submitted to full compendial analysis (months 6–8) before we simulated that regulatory authorities removed failed batches from the market (month 9). This cycle of sampling, testing, and removal was replicated three times over the 3-year modeled time frame, beginning in months 1, 13, and 25.

We simulated that new batches, some of which were poor quality, entered the market each month. Within each scenario we calculated the prevalence of substandard and falsified medicines on a monthly basis as it fluctuated with the removal of medicines, expiration of old medicines, or addition of new batches to the market. Table 1 includes all inputs used to run the model.^6-19

Table 1.

Model input parameters and uncertainty ranges.

Variables	Units	Values	Standard errors or uncertainty ranges	Sources
Epidemiologic & demographic
Population under age 5	Thousand	6997	–	UN DESA 11
Life expectancy at birth	Year	61.10	–	UNICEF 12
Pneumonia incidence for children under age 5	%	3.40	2.64–4.23	O’Brien et al. 13
Pneumonia prescribed with amoxicillin	%	80	70–90	Authors’ assumption
Care seeking from hospital	%	61.98	–	Nair et al. 14
Distribution of patients
National hospital	%	24	3	Ayieko et al. 15
Provincial hospital	%	20	3	Ayieko et al. 15
District hospital	%	23	3	Ayieko et al. 15
Mission hospital	%	31	3	Ayieko et al. 15
Average length of stay
National hospital	Day	8.20	2.05	Ayieko et al. 15
Provincial hospital	Day	6.60	1.65	Ayieko et al. 15
District hospital	Day	5.42	1.35	Ayieko et al. 15
Mission hospital	Day	6.06	1.52	Ayieko et al. 15
Case-fatality ratio
Hospital	%	3.90	0.8	Nair et al. 14
Community	%	9.20	2.3	WHO 1
Medicine quality testing
Screening sensitivity
PADs	%	100	–	Weaver et al. 6
aPADs	%	97	–	Myers et al. 7
HPLC	%	100	–	U.S. Pharmacopeia 16
Screening specificity
PADs	%	100	–	Weaver et al. 6
aPADs	%	92	–	Myers et al. 7
HPLC	%	100	–	U.S. Pharmacopeia 16
Quality control for PADs	%	10	–	Authors’ assumption
Pills needed for each sample tested
PADs	n	1	–	Weaver et al. 6
aPADs	n	1	–	Weaver et al. 6
HPLC	n	100	–	KPPB 17
Substandard & falsified antibiotics
Extra treatment time for SF antibiotics	Day	5	–	Authors’ assumption
Relative risk of mortality if receiving SF antibiotics		2.00	–	WHO 1
Market share and brand specific SF proportion
Brand 1	n (% SF)	13 (0)	–	Myers et al. 17
Brand 2	n (% SF)	37 (0)	–
Brand 3	n (% SF)	9 (11)	10% a
Brand 4	n (% SF)	2 (0)	–
Brand 5	n (% SF)	20 (0)	–
Brand 6	n (% SF)	4 (0)	–
Brand 7	n (% SF)	2 (0)	–
Brand 8	n (% SF)	1 (0)	–
Brand 9	n (% SF)	9 (0)	–
Brand 10	n (% SF)	10 (0)	–
Brand 11	n (% SF)	37 (59)	8% a
Brand 12	n (% SF)	1 (0)	–
Brand 13	n (% SF)	9 (0)	–
Costs
Testing costs
PADs	USD	3.00	0.75 b	Personal Communication
aPADs	USD	3.00	0.75 b	Personal Communication
HPLC	USD	606.00	151.50 b	Quote from MEDS 18
Personnel sampling costs	USD/week	250.00	–	Expert opinion
Amoxicillin (250 mg/tab) price	USD	0.005	0.001 b	Calculated based on UNICEF’s report 8
Diagnostic costs
National hospital	USD	23.73	85.26	Ayieko et al. 15
Provincial hospital	USD	4.48	16.98	Ayieko et al. 15
District hospital	USD	9.34	12.61	Ayieko et al. 15
Mission hospital	USD	31.22	27.75	Ayieko et al. 15
Treatment costs
National hospital	USD	19.25	76.79	Ayieko et al. 15
Provincial hospital	USD	8.34	23.14	Ayieko et al. 15
District hospital	USD	3.74	7.05	Ayieko et al. 15
Mission hospital	USD	26.48	25.03	Ayieko et al. 15
Daily hospital bed costs
National hospital	USD	22.17	28.78	Ayieko et al. 15
Provincial hospital	USD	17.24	12.97	Ayieko et al. 15
District hospital	USD	12.13	9.53	Ayieko et al. 15
Mission hospital	USD	12.18	11.21	Ayieko et al. 15
Discount rate	%	3.00	–	Authors’ assumption
GDP per capita	USD	1455.40	–	World Bank 20

GDP: gross domestic product; HPLC: high-performance liquid chromatography; KPPB: Kenya Pharmacy and Poisons Board; MEDS: Mission for Essential Drugs and Supplies (Kenya); PADs/aPADs: paper analytical devices/antibiotic paper analytical devices; SF: substandard and falsified; UNDESA: United Nations Department of Economic and Social Affairs; UNICEF: United Nations Children’s Fund; USD: United States dollar; WHO: World Health Organization.

^aStandard error was estimated for brand specific substandard and falsified proportions.

^bStandard error was calculated based on 25% of the mean.

Throughout each scenario, we estimated the costs required to perform medicine quality screening and testing. These investment costs included: (1) testing costs incurred by implementing PADs/aPADs and/or HPLC; (2) wastage costs incurred by consuming medications through screening and testing procedures; and (3) costs of personnel to sample medicines and later remove failed batches from the market.^8,17,18 We collected price quotations from the Mission for Essential Drugs and Supplies (MEDS) in Kenya for costs of HPLC testings to estimate the unit price of an identification API assay. ¹⁸ Wastage costs were calculated based on the number of drugs needed for screening and testing and the unit cost of medication. ⁸ We followed the Kenya Pharmacy and Poisons Board guidelines for the number of pills needed for testing samples. ¹⁷ While PADs and aPADs both consumed one pill for every test, HPLC required 100 pills per test. ¹⁸ We included costs per personnel typically needed for sampling and removal, attributing 2 weeks of personnel costs for the sampling of medicines and 4 weeks for the removal of failed batches.

We separately examined the population impact of Kenyan children utilizing substandard or falsified amoxicillin for treatment of pneumonia. Based on the number of pneumonia cases among children under five in Kenya and rates of care-seeking, we estimated the monthly number of pediatric pneumonia treatments with amoxicillin in Kenya. ¹³ We then estimated pneumonia treatments using either legitimate or poor-quality amoxicillin based on the prevalence of substandard or falsified amoxicillin in each scenario. Patients who received poor-quality medicines faced a higher case-fatality rate.^1,14 In addition, we assumed that antibiotic treatment for patients who used medications that were substandard or falsified were prolonged by five extra days. ¹⁵ We used the cost-of-illness approach and took a societal perspective to estimate the economic burden of pediatric pneumonia along with the investment costs of the testing strategy.^20,21

A 3% discount rate was used to estiamate the present value of all future costs.

The economic burden of pneumonia at different levels of prevalence of substandard and falsified amoxicillin was estimated by aggregating: (1) treatment costs; (2) productivity losses of caretakers; and (3) productivity losses due to premature death. Treatment costs in Kenya were extracted from published literature. ¹⁶ This included diagnostic costs, medication costs, and costs per hospital day based on patients’ out-of-pocket expenses. ¹⁵ We defined a treatment as the entire course of amoxicillin (25 pills of 250 mg) needed to treat a case of pediatric pneumonia. Patients who sought care from a community setting did not incur costs of hospital stays. We derived an average treatment cost by weighing the costs of individuals who sought care from national, provincial, district, and mission hospitals. Productivity losses for caretakers were estimated by multiplying the length of treatment by the average daily wage using GDP per capita. ¹⁹ Productivity losses due to premature death were estimated from age 15 to Kenya’s life expectancy, using GDP per capita. ¹² Supplemental Appendices 1.2–1.3 detail the methods used to calculate the costs and benefits.

The analysis was conducted over a 3-year time period. Simulations were run 10,000 times varying inputs probabilistically (Supplemental Appendix 1.4). We present averages across runs with a 90% uncertainty range (UR), demonstrating the spread of simulation outcomes.

Results

The model simulated the prevalence of substandard or falsified amoxicillin per month in Kenya under each scenario (Figure 2). The three testing scenarios showed drops in substandard and falsified amoxicillin prevalence at months when poor-quality amoxicillin were simulated to be removed from the market. In the HPLC scenario, substandard and falsified amoxicillin were removed all at once (month 19). Comparatively, the expedited HPLC scenario was able to lower the prevalence of poor-quality amoxicillin earlier by testing a quarter of the batches at a time. Release of such partial results facilitated removal of batches most frequently in the first two quarters of the samples tested (months 10 and 13). In the PADs/aPADs scenario, screening reduced the number of samples requiring HPLC, resulting in faster removal of substandard and falsified amoxicillin. Moreover, the shortened time frame facilitated more frequent collection and testing of new samples, resulting in removal of substandard and falsified amoxicillin once a year (months 9, 21, 33). After drugs were removed, the prevalence of substandard and falsified amoxicillin slowly built back in each scenario.

Figure 2.

Monthly SF prevalence across medicine quality testing scenarios.

HPLC: high-performance liquid chromatography; PADs/aPADs: paper analytical devices/antibiotic paper analytical devices; SF: substandard or falsified.

The figure shows average monthly prevalence of SF Amoxicillin across 10,000 model runs. Dips demonstrate months where recalls of SF amoxicillin were simulated. After removal, SF prevalence can rebuild as new batches enter the market each month.

Annual costs and outputs of the model averaged over 10,000 model runs are presented in Table 2. By testing all samples with HPLC and removing batches in month 19, the HPLC reference scenario resulted in an average of 25,075 (90% UR 17,602–32,596) substandard and falsified treatments and 12,707 (90% UR 9,157–16,702) child pneumonia deaths, annually. By expediting HPLC and releasing partial results, the simulation averted an average of 3,736 (90% UR 2,325–5,110) substandard and falsified treatments leading to 221 (90% UR 126–332) fewer deaths compared with the reference HPLC scenario. Finally, the PADs/aPADs scenario was the most effective, averting an average of 9,931 (90% UR 6,889–12,903) poor-quality treatments resulting in 586 (90% UR 364–847) fewer child pneumonia deaths annually.

Table 2.

Average annual costs, investments, and returns to test amoxicillin quality in Kenya.

	HPLC (reference)	Expedited HPLC	PADs/aPADs
Costs
Sampling, screening, testing, & removal costs	$9,153	$9,153	$6,120
Incremental investment	$3,033	$3,033	$0
Benefits
Treatment costs	$13,368,329	$13,206,543	$12,937,990
Productivity losses: short-term	$5,364,369	$5,289,872	$5,166,347
Long-term	$308,913,977	$303,549,139	$294,656,953
Incremental return
Excluding productivity		$161,787	$430,339
Including productivity		$5,601,122	$14,855,385
Cost per benefit
Number of child pneumonia deaths	12,707	12,486	12,120
Deaths averted		221	586
Cost per death averted		$41.42	$10.44
Number of SF treatments received	25,075	21,339	15,144
SF treatments averted		3,736	9,931
Cost per SF treatment averted		$2.45	$0.62
SF treatments removed from market	4,685	6,770	13,628
Cost per SF treatment removed	$1.95	$1.35	$0.45

HPLC: high-performance liquid chromatography; PADs/aPADs: paper analytical devices/antibiotic paper analytical devices; SF: substandard or falsified.

The table presents annual average costs over 3 year model runs. Across all scenarios, the numbers of overall treatments (SF treatments + legitimate treatments) are kept constant. The PADs/aPADs with HPLC scenario draws three different samples over 3 years, and is thus is able to detect and remove more SF amoxicillin compared to other scenarios.

We estimated the average annual incremental costs across medicine quality screening and testing scenarios. The baseline annual costs for HPLC were estimated at $9,153 (90% UR $6,074–$12,795). Over the 3-year time frame, $499 (90% UR $314–$718) was used for personnel conducting sampling, $25,963 (90% UR $16,807–$36,885) for the costs of conducting HPLC testing, and $998 (90% UR $628–$1,436) for personnel conducting removal of medicines. The expedited HPLC scenario utilized the same resources as the HPLC scenario, just at different times. This resulted in equal costs across the two scenarios. Overall, PADs/aPADs scenario resulted in cost savings compared with using HPLC, costing $3,033 (90% UR −$178 to $6,769) less annually. Because the PADs/aPADs scenario sampled and removed failed batches three times over the model compared with once in the other scenarios, sampling and removal costs were higher than in the HPLC scenarios. Over 3 years, sampling costs were $998 (90% UR $628–$1,436) higher and removal costs were $1,997 (90% UR $1,257–$2,872) higher. However, the costs of testing samples via PADs/aPADs with HPLC were $12,094 lower (90% UR −$536 to $20,309) than in the HPLC scenario over 3 years.

We simulated that it cost $10.44 to avert a death due to substandard or falsified amoxicillin using PADs/aPADs compared with $41.42 in the expedited HPLC scenario. Similarly, PADs/aPADs cost $0.62 per substandard and falsified treatment averted compared with $2.45 for expedited HPLC. Cost per substandard and falsified treatment removed was $1.95 for HPLC, $1.35 for expedited HPLC, and $0.45 for PADs/aPADs (Table 2). Including the productivity losses averted by ensuring amoxicillin quality for treatment of child pneumonia in Kenya, our model estimated an incremental return for expedited HPLC of $5.6 million and PADs/aPADs of $14.9 million. Productivity losses averted from preventing premature deaths drove the incremental returns in each scenario.

Our probabilistic sensitivity analysis results across 10,000 model runs are shown in Figure 3. We saw that the PADs/aPADs scenario dominated the HPLC and expedited HPLC scenarios (Figure 3), consistently demonstrating that PADs and aPADs resulted in cost savings compared to using HPLC.

Figure 3.

Annual incremental investment and returns (including productivity) across medicine quality testing scenarios.

HPLC: high-performance liquid chromatography; PADs/aPADs: paper analytical devices/antibiotic paper analytical devices; SF: substandard or falsified.

The model was run 10,000 times while varying input parameters based on distributions in Table 1. The HPLC and expedited HPLC scenarios closely overlap, as the testing costs for both scenarios are identical and yielded comparable impact on SF prevalence (Figure 2). The model consistently showed that PADs/aPADs scenario resulted in lower costs and greater benefits than HPLC alone.

Discussion

Our results demonstrate the compelling benefits of using PADs/aPADs for medicine quality screening rather than relying on conventional HPLC techniques alone. Increasing the speed of detection of poor-quality medical products and the speed of regulatory action to remove substandard and falsified medicines had a large impact on the prevalence of substandard and falsified products in our model. This suggests that increasing the speed of detection of these products could enable regulatory agencies to do their job more efficiently. The PADs and aPADs scenario was effective mainly because PADs/aPADs are inexpensive screening devices with high sensitivity and specificity for amoxicillin that can reduce the number of medicines that require expensive and time-consuming confirmatory HPLC testing. As a result, using PADs/aPADs can offer faster detection and cost savings compared with utilizing HPLC alone. While the expedited HPLC scenario was able to speed up the removal of substandard and falsified drugs initially, the benefits were short lived and the need to test each sample with HPLC ultimately slowed down the process. Our model illustrated that the longer stretches of time substandard and falsified amoxicillin were available on the market in the HPLC and expedited HPLC scenarios resulted in many more patients receiving poor-quality amoxicillin, resulting in greater treatment costs and productivity losses.

Time played a critical role in our ESTEEM agent-based model. Specifically, we found that it was important to act quickly to detect and remove substandard and falsified medicines from the market. Using HPLC to analyze every sample extended the time that samples sat in a queue waiting to be analyzed, which meant that substandard and falsified amoxicillin stayed on the market for longer and could be utilized by patients. Quick screening tools, such as PADs/aPADs, enabled time to take regulatory actions before the medicines expired. Although HPLC may still be needed for confirmatory analysis to remove poor-quality products, reducing the amount of samples that require HPLC testing can greatly improve medicine quality testing operations.

The PADs and aPADs scenario dominated over the expedited HPLC scenario with cost savings ($0.45 compared with $1.35 per substandard and falsified treatment removed from the market, respectively) and resulted in higher incremental returns ($14.9 million, compared with $5.6 million, respectively). This makes PADs/aPADs an attractive innovation that can save costs and deliver greater societal benefits at the same time. We showed that the adoption of PADs/aPADs to screen for substandard and falsified amoxicillin in Kenya can save an average of $9,100 over a 3-year period while screening three times as many samples in that time compared with using only HPLC. While our analysis focused on demonstrating the benefits of PADs and aPADs in removing poor-quality amoxicillin for treatment of child pneumonia in Kenya, we expect similar benefits could be shown in other settings to test for antibiotic quality. These results would be of utmost interest to National Medicines Regulatory Authorities (NMRAs) to improve monitoring of medicine quality.

Strengthening the capacity of NMRAs is the most sustainable solution to combating and reducing the prevalence of substandard and falsified antimicrobials in LMICs. To achieve this, NMRAs need to be equipped with low-cost, high-quality drug screening technologies in addition to continued efforts to increase HPLC capacity. ²² Ideal screening technologies are low cost to obtain and maintain, and do not require highly specialized training. Although some countries have the technical capacity at the national level to test drugs via HPLC through WHO Prequalified Medicine Quality Control Laboratories, testing large quantities of medicines can be too costly and may be an inefficient use of funds by national governments. ²³ Using screening technologies such as PADs/aPADs to reduce samples requiring HPLC testing is in line with the guidance for implementing risk-based post-marketing quality surveillance in LMICs using the 3-level approach. ²⁴ , ²⁵ PADs/aPADs offer a promising tool to screen antibiotics in a low-cost way.

In addition to PADs and aPADs, there are other screening technologies that could aid in substantially reducing the cost for routine medicine quality surveillance.^9,26,27 The Global Pharma Health Fund (GPHF)-Minilab is the most widely used screening technology in LMICs. It is similarly based on thin-layer chromatography (TLC) and requires 1–2 weeks of training by the operator.^28–30 TLC is a sensitive tool for detecting falsified products and has some capacity to detect substandard products, although not all users have found it fit for that purpose.^26,31,32 Portable spectrometers (Raman, infrared, and non-infrared) and a multispectral imaging tool called the CDx have become available for pharmaceutical testing.^33–36 These screening tools have the potential to increase the effectiveness of medicine quality testing by better prioritization of which samples undergo full compendial analysis. Some of these technologies have been reviewed and rated for LMIC feasibility. ²⁶ Those that scored the highest and were categorized as the most suitable for use in LMICs (scored 6 out of 8 or greater) included PADs, PharmaCheck, Fourier-transform infrared spectroscopy, near infrared spectroscopy, CDx device, and Raman spectrometry. ²⁶ These technologies have been described briefly elsewhere. ³⁷ In a comparison of six portable screening devices (including PADs) to visual inspection, all six devices were found to be cost-effective for the screening of antimalarials in Lao People’s Democratic Republic. ³⁸ However, few studies to date have estimated the costs and feasibility for NMRAs to employ these technologies for routine medicine quality surveillance, particularly for detection of substandard products.^39,40

There are a number of study limitations to note. First, limited data availability for some variables resulted in the need for us to utilize sub-national or regional data. For instance, we utilized the brand market share and prevalence of substandard and falsified amoxicillin from our recent testing of samples from Western Kenya. ⁷ While there may be other brands of amoxicillin available on the national market, and market share may vary across regions, national-level data were not available. Furthermore, we utilized a regional estimate for the incidence of pneumonia for children under five. ¹³ To compensate for weaknesses in individual data inputs, we conducted rigorous sensitivity analyses to capture robust uncertainty ranges. We also validated our results against other sources, for example, by ensuring our pneumonia deaths are comparable against WHO-Maternal Child Epidemiology Estimation (MCEE) estimates. ⁴¹ Second, while this study focused on the impact of screening and testing of amoxicillin used for treatment of pediatric pneumonia, amoxicillin is also widely used to treat other infectious diseases. Our benefits are therefore conservative by focusing our modeling of the benefits of medicine quality only on one disease. In addition, the study does not include other costs such as transportation to seek health care and opportunity costs relevant to care-seeking due to limited availability of quality data. We believe exclusion of these costs has not significantly altered our conclusions. We did not include costs of newly acquiring HPLC equipment, which would have resulted in greater cost savings for PADs/aPADs. Our analysis also does not account for the benefits of limiting the development of antimicrobial resistance by removing substandard and falsified antibiotics from the market. Including such benefits would demonstrate greater impacts of medicine quality testing. Despite these limitations, we believe this study is valuable to stakeholders in governments, NMRAs, and the medicines regulatory community to ensure that medicines are safe, trustworthy, and of high quality.

Conclusion

PADs and aPADs offer an affordable and efficient alternative to testing medicine quality using HPLC alone. This case study demonstrates the benefits of utilizing PADs/aPADs for screening by reducing the need for HPLC testing, speeding up the identification and removal of substandard and falsified medicines, and reducing the population disease burden. Given the high pneumonia burden for children under five and high prevalence of substandard and falsified antibiotics in some LMICs, screening and testing antibiotic quality is essential in countries such as Kenya. It is critical for governments, NMRAs, and other authorities to consider PADs/aPADs as an option to ensure medicines quality and protect the health of the population.