Abstract.
Global Pharma Health Fund (GPHF) Minilab™, a semi-quantitative thin-layer chromatography (TLC)–based commercially available test kit, is widely used in drug quality surveillance globally, but its diagnostic accuracy is unclear. We investigated the diagnostic accuracy of Minilab system for antimicrobials, using high-performance liquid chromatography (HPLC) as reference standard. Following the Minilab protocols and the Pharmacopoeia of the People’s Republic of China protocols, Minilab-TLC and HPLC were used to test five common antimicrobials (506 batches) for relative concentration of active pharmaceutical ingredients. The prevalence of poor-quality antimicrobials determined, respectively, by Minilab TLC and HPLC was amoxicillin (0% versus 14.9%), azithromycin (0% versus 17.4%), cefuroxime axetil (14.3% versus 0%), levofloxacin (0% versus 3.0%), and metronidazole (0% versus 38.0%). The Minilab TLC had false-positive and false-negative detection rates of 2.6% (13/506) and 15.2% (77/506) accordingly, resulting in the following test characteristics: sensitivity 0%, specificity 97.0%, positive predictive value 0, negative predictive value 0.8, positive likelihood ratio 0, negative likelihood ratio 1.0, diagnostic odds ratio 0, and adjusted diagnostic odds ratio 0.2. This study demonstrates unsatisfying diagnostic accuracy of Minilab system in screening poor-quality antimicrobials of common use. Using Minilab as a stand-alone system for monitoring drug quality should be reconsidered.
INTRODUCTION
Poor-quality drugs are a serious public health problem worldwide. According to the World Health Organization (WHO),1 poor quality drugs include substandard antimicrobials, which are authorized medical products failing to meet either their quality standards and/or specifications, and falsified antimicrobials, which deliberately/fraudulently misrepresent their identity, composition, or source. The prevalence of poor-quality antimicrobials varies from 0% to more than 50% globally, and their consequences as treatment failure, drug resistance, adverse drug events, and even death have been frequently reported.2
Drug quality surveillance involves screening and confirmatory testing. Although screening tests are less accurate than confirmatory tests, they are preferred over more expensive confirmatory tests such as high-performance liquid chromatography (HPLC) for field testing, especially in resource-limited low- and middle-income countries.3–5 There are common screening tools, such as WHO Checklist, paper chromatography cards, PharmaCheck, near infrared spectrometry, Counterfeit Device #3, Raman spectrometry, and Global Pharma Health Fund (GPHF) Minilab™.4,5
GPHF Minilab™ is a semi-quantitative thin-layer chromatography (TLC)–based commercially available test kit made by the GPHF, Merck Darmstadt, Germany. It has been widely used in drug-quality monitoring in as many as 95 countries, most of them low- or middle-income.6 More than 800 GPHF-Minilab units have been supplied globally until 2017.6,7 Its use has been supported by the WHO and the United States Pharmacopeia Drug Quality Information program.6,7 In light of its widespread use, even as the reference test to evaluate the diagnostic accuracy of a handheld Raman spectrometer,8 several studies have investigated the performance reliability of the Minilab.7,9–15 Most of these studies focused mainly on anti-malarial drugs9,10,12,14,15 and few studies on other antimicrobials.13,15 Based on the limited sample sizes, some studies reported the Minilab as a tool that can identify falsified, but not substandard, medicines.9–13,15
Being the second country only after India in producing poor-quality antimicrobials,2,16 China is in need of reinforcing drug-quality control by using cost-effective screening tests for its heavy surveillance workload. In this study, we examined the Minilab’s applicability in screening/monitoring (specifically, its diagnostic accuracy) for five common antimicrobials (amoxicillin, azithromycin, cefuroxime axetil, levofloxacin, and metronidazole) which are usually prescribed for infectious diseases caused by gram-positive and gram-negative bacteria, mycoplasma, Chlamydia, and anaerobes. Anti-malarial drugs, the most commonly investigated antimicrobials by the GPHF Minilab™ system in literature, were not included in this study, because malaria is not endemic in the study region. HPLC was used as the reference standard confirmatory test because it is recommended by different pharmacopeias including the Pharmacopoeia of the People’s Republic of China (PPRC) and widely used by the China Food and Drug Administration.
MATERIALS AND METHODS
The STARD flowchart of the study is shown in Figure 1. Five kinds of antimicrobials in 506 batches were purchased from community pharmacies in Shantou, Guangdong, China, in this cross-sectional study (May to July 2013) as described previously.3 The selection criteria3 for these antimicrobials were as follows: 1) frequently used in hospitals and pharmacies17; 2) easily destroyed under suboptimal storage conditions2,17; 3) commonly reported to have no active ingredient2,17; 4) listed in the National Essential Drug List of China.3,17
Figure 1.
Flowchart outlining the procedure of sample collection, testing, and reporting the quality of common antimicrobials and the diagnostic accuracy of the Minilab. HPLC = high-performance liquid chromatography; TLC = thin-layer chromatography.
With the estimated prevalence (15%) of poor-quality drugs in developing countries, the sample size of antimicrobials was calculated using the formula previously described18,19 as follows: n = confidence level2 * estimated prevalence * (1 − estimated prevalence) * design effect * (1 + non-response error)/margin of error2 = 1.962 × 0.15 × (1 − 0.15) × 2 × (1 + 0.05)/0.052 = 411.
From 2,180 pharmacies registered in Shantou, 124 pharmacies were randomly selected and visited based on their geographic distributions, nine were found to be closed during the sampling period, and 115 were finally included in the study. Following the consumer purchase behavior in China, our study staff and student volunteers approached the target pharmacies like patient’s relatives (mystery shoppers) and purchased the target antimicrobials without any prescription and following the pharmacy staff’s recommendation. Totally, 506 batches of antimicrobials (out of the estimated sample size as 411) were collected and transported to our laboratory in the same day and stored at room temperature (15–25°C) and relative humidity level of 60% or lower before testing. GPHF Minilab™ kits and the reference standard antimicrobials (all in tablet formulation) used for Minilab tests were purchased from the GPHF, Germany. Following the Minilab protocols, TLC was used to test drug samples in triplicate for the relative concentration of active pharmaceutical ingredients (APIs). Two HPLC-experienced operators received a 2-week Minilab TLC training, twice the standard training period recommended by the device’s manufacturer.20 In our proficiency testing, the operators achieved 100% agreement with the preset standard and substandard samples, nearly twice the rate achieved in a standard 1-week training.21 Reference standard antimicrobials for HPLC tests were purchased from Huipeng Science and Technology, China (http://www.szhuipeng.cn/), one of the authorized suppliers for the National Institutes for Food and Drug Control, China. Testing samples with HPLC was described previously.3 Minilab TLC and HPLC tests were performed simultaneously, while the operators were blinded to the results. All tests were done in two independent laboratories (the Shantou-Oxford Clinical Research Unit and the Bio-analytical Laboratory) of Shantou University Medical College.
For HPLC, “out of range” (i.e., tested positive) was defined as an API outside the acceptable range 93–107% for metronidazole or 90–110% for all other drugs (Table 1). For the Minilab, “out of range” was defined as an API of less than 80% of the pharmacopoeial standards for all drugs. The diagnostic test characteristics of Minilab TLC test were measured (i.e., sensitivity [Sn], specificity [Sp], positive predictive value [PPV], negative predictive value [NPV], positive likelihood ratio [PLR], negative likelihood ratio [NLR], diagnostic odds ratio [DOR],22 and adjusted diagnostic odds ratio [ADOR] with 95% confidence intervals [95% CI]). ADOR was calculated by adding 0.5 to all original counts as some of them were zero.23 All the analyses were done through SPSS statistics V17.0 and SAS 9.4. Our results were reported according to the MEDQUARG guidelines,24 the STARD checklist,25 and the QUADAS tool26 where possible.
Table 1.
The active pharmaceutical ingredients (API) of antimicrobials (N = 506)
| Antimicrobial (N = 506) | Actual content of API* mean (SD) | Range* (minimum–maximum) | Lower and upper quartiles | PPRC standard |
|---|---|---|---|---|
| Amoxicillin (N = 114) | 94.1% (5.1) | 74.3–122.0% | 91.8–96.6% | 90.0–110.0% |
| Azithromycin (N = 92) | 101.3% (7.9) | 84.5–122.9% | 96.9–102.6% | 90.0–110.0% |
| Cefuroxime axetil (N = 91) | 97.9% (2.9) | 90.4–103.9% | 95.8–100.3% | 90.0–110.0% |
| Levofloxacin (N = 101) | 98.7% (4.1) | 83.5–110.2% | 96.1–101.5% | 90.0–110.0% |
| Metronidazole (N = 108) | 93.4% (1.8) | 84.1–98.1% | 92.1–94.5% | 93.0–107.0% |
PPRC = Pharmacopoeia of the People’s Republic of China (2010); SD = standard deviation.
Compared with reference standard antimicrobials by high-performance liquid chromatography.
RESULTS
Our results are summarized in Tables 1 and 2. Of 506 batches of antimicrobial samples, 2.6% (13/506) did not meet the standard limits of the Minilab TLC system, but all of them (13/13) were within the expected quality standards of the PPRC in the HPLC system. This false positive detection occurred exclusively with cefuroxime axetil, accounting for 14.3% of cefuroxime axetil samples (13/91). On the other hand, false negativity was seen in 15.2% of samples (77/506) that met the quality standard limits in the Minilab TLC but failed in the HPLC system. Twenty-two percent (17/77) of the poor-quality samples were higher than the upper standard limit of the PPRC. Accordingly, the characteristics of Minilab TLC test were recorded as follows: Sn 0% (95% CI: 0– 4.7%); Sp 97.0% (95% CI: 94.9–98.4%); PPV 0; NPV 0.8 (95% CI: 0.8–0.9); PLR 0; NLR 1.0 (95% CI: 1.0–1.0); DOR 0; and ADOR 0.2 (95% CI: 0.01–3.4).
Table 2.
Diagnostic accuracy of GPHF Minilab™ tests in the surveillance of poor-quality antimicrobials (N = 506)
| Antimicrobial | Minilab TLC | HPLC | Sn (95% CI) (%) | Sp (95% CI) (%) | PPV (95% CI) | NPV (95% CI) | PLR (95% CI) | NLR (95% CI) | DOR (95% CI) | ADOR* (95% CI) | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Positive (n) | Negative (n) | ||||||||||
| Amoxicillin (N = 114) | Positive | 0 (TP) | 0 (FP) | 0 (0–19.5) | 100.0 (96.3–100.0) | NaN† | 0.9 (0.8–0.9) | NaN† | 1.0 (1.0–1.0) | NaN† | 5.6 (0.1–290.2) |
| Negative | 17 (FN) | 97 (TN) | |||||||||
| Azithromycin (N = 92) | Positive | 0 (TP) | 0 (FP) | 0 (0–20.6) | 100.0 (95.3–100.0) | NaN† | 0.8 (0.7–0.9) | NaN† | 1.0 (1.0–1.0) | NaN† | 4.6 (0.1–242.2) |
| Negative | 16 (FN) | 76 (TN) | |||||||||
| Cefuroxime axetil (N = 91) | Positive | 0 (TP) | 13 (FP) | NaN† | 85.7 (76.8–92.2) | 0 | 1.0 (1.0–1.0) | 0 | 1.2 (NaN†) | NaN† | 5.8 (0.1–305.7) |
| Negative | 0 (FN) | 78 (TN) | |||||||||
| Levofloxacin (N = 101) | Positive | 0 (TP) | 0 (FP) | 0 (0–70.8) | 100.0 (96.3–100.0) | NaN† | 1.0 (0.9–1.0) | NaN† | 1.0 (1.0–1.0) | NaN† | 28.1 (0.5–1635.3) |
| Negative | 3 (FN) | 98 (TN) | |||||||||
| Metronidazole (N = 108) | Positive | 0 (TP) | 0 (FP) | 0 (0–8.6) | 100.0 (94.6–100.0) | NaN† | 0.6 (0.5–0.7) | NaN† | 1.0 (1.0–1.0) | NaN† | 1.6 (0.03–83.5) |
| Negative | 41 (FN) | 67 (TN) | |||||||||
| Total (N = 506) | Positive | 0 (TP) | 13 (FP) | 0 (0–4.7) | 97.0 (94.9–98.4) | 0 | 0.8 (0.8–0.9) | 0 | 1.0 (1.0–1.0) | 0 (NaN†) | 0.2 (0.01–3.4) |
| Negative | 77 (FN) | 416 (TN) | |||||||||
ADOR = adjusted DOR; DOR = diagnostic odds ratio; FN = false negative; FP = false positive; HPLC = high-performance liquid chromatography; NLR = negative likelihood ratio; NPV = negative predictive value; PLR = positive likelihood ratio; Positive = failing TLC or HPLC; Negative = passing TLC or HPLC; PPV = positive predictive value; Sn = sensitivity; Sp = specificity; TLC = thin-layer chromatography; TN = true negative; TP = true positive.
ADOR was calculated by adding 0.5 to all counts in table (see Materials and Methods for details).
NaN = not a number.
The prevalence of poor-quality samples determined, respectively, by Minilab TLC and HPLC was amoxicillin (0% versus 14.9%), azithromycin (0% versus 17.4%), cefuroxime axetil (14.3% versus 0%), levofloxacin (0% versus 3.0%), and metronidazole (0% versus 38.0%).
DISCUSSION
The prevalence of poor-quality antimicrobials as determined by the Minilab system varies by country and by drug from 0% to 26%.9–13,27,28 For example, the prevalence of poor-quality amoxicillin was between 0% and 10% in Tanzania, Ghana, Nigeria, and the United Kingdom.13,27
This wide range of reported prevalence most likely originates from the subjectivity and skills required in operating the Minilab system and its diagnostic accuracy that varies with drugs. Highly variable Minilab test outcomes have been reportedly due to the operator’s visual estimation of TLC and operational skills for manually spotting on the chromatographic plate.15,21,29 In a Minilab proficiency test conducted by the Tanzania Food and Drugs Authority, only five of nine experienced inspectors were able to correctly identify the API levels of antimicrobials.21 Drug-dependent variation in the diagnostic accuracy of the Minilab has been exemplified by its Sn as 0% for amoxicillin and 14% for co-trimoxazole, compared with HPLC-photodiode array detection.13 The WHO QAMSA study performed in six countries found that the Minilab tests could only identify 32% of the poor-quality drugs.7 The performance of the Minilab in literature is summarized in Supplemental Table 1.
The diagnostic accuracy of the Minilab seems to be independent from the prevalence of poor-quality antimicrobials in this study, because the Sn (0% for nearly all antimicrobials) and Sp (range: 85.7–100%) of the Minilab were not correlated with the prevalence between 0% and 38% by HPLC. The rationale behind this could be that the APIs of most samples were higher than 80% API threshold that the Minilab is designed to detect. The Minilab did not, in fact, fail to detect most poor-quality drugs in this study by the standards that the manufacturer set for the device, but it regarded the drugs having > 80% APIs as good quality, except the 13 cefuroxime axetil samples. Only one sample, that is, amoxicillin (API: 74.3% by HPLC) fell below the 80% API cutoff.
This highlights the importance of considering and evaluating the performance characteristics and indications of the Minilab TLC properly before use. Whereas HPLC could be less affordable or even unavailable in developing countries or regions with limited resources, the Minilab system is much cheaper (e.g., the cost of consumables for Minilab TLC versus HPLC: approximately 70 versus 300 RMB per sample in this study; 1 USD = 6.6 RMB), portable, and rather efficient for drug quality surveillance.4,5,15 As a semi-quantitative test, the Minilab performs well in detecting falsified drugs which contain no API or wrong API.10–15,27 For this reason, it has been broadly used as a screening test globally, especially in Vietnam, Nigeria, Tanzania, Congo-Kinshasa, Ghana, Laos, Myanmar, Ethiopia, Indonesia, and Colombia.30 However, this study confirms that the Minilab performs very poorly in detecting drug levels that are close to, but fall short of, pharmacopoeial standards. In addition, there is no upper-detection limit in the Minilab system; therefore, it failed to identify drugs with higher APIs, further reducing our estimates of its Sn. Along with others’ findings,10–15,27 a relatively high false-negative rate (15.2%) of detecting poor-quality antimicrobials with the Minilab TLC in this study suggests that had other studies9,28 used more sensitive and specific methods such as HPLC, near-infrared spectrometry, or Raman spectrometry, they would have yielded higher estimates of poor-quality drugs.
The three-level approach involving the Minilab system has been used by the WHO QAMSA and USP-PQM studies7,31,32 to monitor drug quality in limited-resource countries: 1) visual and physical inspections of the drug and its packaging; 2) rapid screening with GPHF Minilab™; 3) confirmatory testing. All samples failing in the screening test and a random selection of the samples passing the screening test are further tested by pharmacopeial methods for confirmation.7,31,32 This approach is helpful for detecting most falsified drugs and a certain proportion of substandard drugs in a large amount of samples15; however, it has limitations,31 such as leading to inconclusive results and failing to identify all substandard medicines. As proven by this study, this approach might not be applicable in countries such as China, where strict law enforcement requires conclusive identification of both falsified and substandard medicines.
Low Sn of the Minilab in detecting substandard medicines with either low or high APIs has negative influences at clinical level. To the extent that pharmacopoeial standards and dose formulations correctly reflect the therapeutic efficacy of a medicine, the medicines that have lower APIs threaten patient welfare and may also promote antimicrobial resistance.33 On the other hand, the medicines with higher APIs (comprising 22% of poor-quality samples in this study) which the Minilab falsely regard as good quality have a potential to cause adverse drug events. On the basis of these notions, we would recommend reconsidering the Minilab manuals for medicines with higher APIs and caution against the use of the Minilab as a stand-alone system for monitoring drug quality.
Sn and Sp used in previous studies7,13 can only partially indicate the performance of the Minilab. Its other test characteristics, such as PPV, NPV, PLR, NLR, DOR, or ADOR, are not available in literature, and therefore, the test accuracy of the Minilab system has been unclear. This study has demonstrated unsatisfying test characteristics of the Minilab—PPV (0), PLR (0), NLR (1.0), DOR (0), and ADOR (0.2)—in screening poor-quality antimicrobials. To better understand the test characteristics of the Minilab system on different medicines, further studies utilizing comprehensive sampling strategy and reliable confirmatory tests are needed.
In summary, this study demonstrates unsatisfying diagnostic accuracy of the Minilab system in screening poor-quality antimicrobials of common use. Using the Minilab as a stand-alone system for monitoring drug quality should, therefore, be reconsidered.
Supplementary Material
Acknowledgments:
We are grateful to Dr. Elizabeth Pisani (London) and Prof. Paul N. Newton (Lao-Oxford-Mahosot Hospital-Wellcome Trust Research Unit, Vientiane, Lao People’s Democratic Republic) for helpful suggestions.
Note: Supplemental table appears at www.ajtmh.org.
REFERENCES
- 1.WHO , 2017. Definitions of Substandard and Falsified (SF) Mdical Products Available at: http://www.who.int/medicines/regulation/ssffc/definitions/en/. Accessed September 23, 2017.
- 2.Kelesidis T, Falagas ME, 2015. Substandard/counterfeit antimicrobial drugs. Clin Microbiol Rev 28: 443–464. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Pan H, Luo H, Chen S, Ba-Thein W, 2016. Pharmacopoeial quality of antimicrobial drugs in southern China. Lancet Glob Health 4: e300–e302. [DOI] [PubMed] [Google Scholar]
- 4.Martino R, Malet-Martino M, Gilard V, Balayssac S, 2010. Counterfeit drugs: analytical techniques for their identification. Anal Bioanal Chem 398: 77–92. [DOI] [PubMed] [Google Scholar]
- 5.Kovacs S, Hawes SE, Maley SN, Mosites E, Wong L, Stergachis A, 2014. Technologies for detecting falsified and substandard drugs in low and middle-income countries. PLoS One 9: e90601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.GPHF , 2017. The GPHF-Minilab™–Protection against Counterfeit Medicines Available at: https://www.gphf.org/en/minilab/index.htm. Accessed August 30, 2017.
- 7.World Health Organization , 2011. Survey of the Quality of Selected Antimalarial Medicines Circulating in Six Countries of Sub-Saharan Africa Available at: http://www.who.int/medicines/publications/WHO_QAMSA_report.pdf. Accessed June 16, 2016.
- 8.Visser BJ, de Vries SG, Bache EB, Meerveld-Gerrits J, Kroon D, Boersma J, Agnandji ST, van Vugt M, Grobusch MP, 2016. The diagnostic accuracy of the hand-held Raman spectrometer for the identification of anti-malarial drugs. Malar J 15: 160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Visser BJ, et al. 2015. Assessing the quality of anti-malarial drugs from Gabonese pharmacies using the MiniLab®: a field study. Malar J 14: 273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Vijaykadga S, Cholpol S, Sitthimongkol S, Pawaphutanan A, Pinyoratanachot A, Rojanawatsirivet C, Kovithvattanapong R, Thimasarn K, 2006. Strengthening of national capacity in implementation of antimalarial drug quality assurance in Thailand. Southeast Asian J Trop Med Public Health 37: 5–10. [PubMed] [Google Scholar]
- 11.Roger B, Richard T, Kimberly H, Lorraine M, Karen P, 2009. Pilot study comparing technologies to test for substandard drugs in field settings. Afr J Pharm Pharmacol 3: 165–170. [Google Scholar]
- 12.Lalani M, Kaur H, Mohammed N, Mailk N, van Wyk A, Jan S, Kakar RM, Mojadidi MK, Leslie T, 2015. Substandard antimalarials available in Afghanistan: a case for assessing the quality of drugs in resource poor settings. Am J Trop Med Hyg 92: 51–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fadeyi I, Lalani M, Mailk N, Van Wyk A, Kaur H, 2015. Quality of the antibiotics–amoxicillin and co-trimoxazole from Ghana, Nigeria, and the United Kingdom. Am J Trop Med Hyg 92: 87–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bate R, Hess K, 2010. Anti-malarial drug quality in Lagos and Accra–a comparison of various quality assessments. Malar J 9: 1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Khuluza F, Kigera S, Heide L, 2017. Low prevalence of substandard and falsified antimalarial and antibiotic medicines in public and faith-based health facilities of southern Malawi. Am J Trop Med Hyg 96: 1124–1135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Delepierre A, Gayot A, Carpentier A, 2012. Update on counterfeit antibiotics worldwide; public health risks. Med Mal Infect 42: 247–255. [DOI] [PubMed] [Google Scholar]
- 17.Syhakhang L, Lundborg CS, Lindgren B, Tomson G, 2004. The quality of drugs in private pharmacies in Lao PDR: a repeat study in 1997 and 1999. Pharm World Sci 26: 333–338. [DOI] [PubMed] [Google Scholar]
- 18.Sapkota AR, et al. 2010. Self-medication with antibiotics for the treatment of menstrual symptoms in southwest Nigeria: a cross-sectional study. BMC Public Health 10: 610. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.IFAD , 2017. Calculating the Sample Size Available at: https://www.ifad.org/topic/resource/tags/food_and_nutrition_security/2141517. Accessed September 23, 2017.
- 20.GPHF , 2016. The GPHF-Minilab™–Training. Available at: https://www.gphf.org/en/minilab/schulungen.htm. Accessed September 5, 2016.
- 21.Risha P, Msuya Z, Ndomondo-Sigonda M, Layloff T, 2006. Proficiency testing as a tool to assess the performance of visual TLC quantitation estimates. J AOAC Int 89: 1300–1304. [PubMed] [Google Scholar]
- 22.Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM, 2003. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 56: 1129–1135. [DOI] [PubMed] [Google Scholar]
- 23.Littenberg B, Moses LE, 1993. Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method. Med Decis Making 13: 313–321. [DOI] [PubMed] [Google Scholar]
- 24.Newton PN, et al. 2009. Guidelines for field surveys of the quality of medicines: a proposal. PLoS Med 6: e52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bossuyt PM, et al. 2015. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 351: h5527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J, 2003. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 3: 25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Risha PG, Msuya Z, Clark M, Johnson K, Ndomondo-Sigonda M, Layloff T, 2008. The use of Minilabs to improve the testing capacity of regulatory authorities in resource limited settings: Tanzanian experience. Health Policy 87: 217–222. [DOI] [PubMed] [Google Scholar]
- 28.Bate R, Tren R, Mooney L, Hess K, Mitra B, Debroy B, Attaran A, 2009. Pilot study of essential drug quality in two major cities in India. PLoS One 4: e6003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kaale E, Risha P, Layloff T, 2011. TLC for pharmaceutical analysis in resource limited countries. J Chromatogr A 1218: 2732–2736. [DOI] [PubMed] [Google Scholar]
- 30.GPHF , 2017. Global Use of the GPHF-Minilab™ Available at: https://www.gphf.org/en/minilab/einsatzgebiete.htm. Accessed September 2, 2017.
- 31.Pribluda V, Barojas A, Coignez V, Bradby S, Dijiba Y, El-Hadri L, Hajjou M, Krech L, Phanouvong S, Smine K, 2014. The three-level approach: a framework for ensuring medicines quality in limited-resource countries. Pharm Regul Aff 3: 2. [Google Scholar]
- 32.Hajjou M, et al. 2015. Monitoring the quality of medicines: results from Africa, Asia, and south America. Am J Trop Med Hyg 92: 68–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Pisani E, 2015. Antimicrobial Resistance: What Does Medicine Quality have to do With it? Available at: http://amr-review.org/sites/default/files/ElizabethPisaniMedicinesQualitypaper.pdf. Accessed September 5, 2016.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.