Evaluating Diagnostic Point-of-Care Tests in Resource-Limited Settings

Abstract

Diagnostic point-of-care (POC) testing is intended to minimize the time to obtain a test result, thereby allowing clinicians and patients to make an expeditious clinical decision. As POC tests expand into resource-limited settings (RLS), the benefits must outweigh the costs. To optimize POC testing in RLS, diagnostic POC tests need rigorous evaluations focused on relevant clinical outcomes and operational costs, which differ from evaluations of conventional diagnostic tests. Here, we reviewed published studies on POC testing in RLS, and found no clearly defined metric for the clinical utility of POC testing. Therefore, we propose a framework for evaluating POC tests, and suggest and define the term “test efficacy” to describe a diagnostic test’s capacity to support a clinical decision within its operational context. We also proposed revised criteria for an ideal diagnostic POC test in resource-limited settings. Through systematic evaluations, comparisons between centralized diagnostic testing and novel POC technologies can be more formalized, and health officials can better determine which POC technologies represent valuable additions to their clinical programs.

Introduction

Diagnostic technologies have improved and expanded substantially over the last several decades.¹ In developed countries, laboratory testing has become increasingly automated, which improves reliability and reduces operator time. Diagnostic testing is now a fundamental part of medical practice, particularly in this era of drug-resistant infectious diseases. When rapid laboratory testing is integrated with electronic medical records, clinicians can receive test results even faster, which should, in principle, improve patient care and outcomes.²

There are tradeoffs, however. Foremost, most advanced diagnostic laboratory technologies are centralized, and require highly trained staff and specialized facilities. The equipment is generally expensive and requires regular maintenance from skilled technicians. Consequently, many current laboratory-based tests are cost-prohibitive and inaccessible to most patients and clinicians around the world.^3,4 In recognition of this disparity, the World Health Organization (WHO) and others have called for new clinical diagnostic tools that can function in settings with limited access to a central laboratory.^5–9 By some estimates, deploying rapid, laboratory-independent diagnostic tests for just four infections (bacterial pneumonia, syphilis, malaria, and tuberculosis) could prevent more than 1.2 million deaths each year in developing countries.^10–12

Diagnostic testing conducted at or near the site of patient care, called point-of-care (POC) testing, can provide results to a clinician without having to wait days or even hours for sample transport and laboratory processing.¹³ The POC testing era began in 1962, when a new, rapid method to measure blood glucose levels¹⁴ was developed and was bolstered in 1977 with the introduction of a rapid pregnancy test.¹⁵ Clinic- or hospital-based POC testing gained significant traction in the early 1990s with small, portable devices capable of measuring multiple electrolytes of patients in emergency departments.^16,17 In the two decades since, many diagnostic POC tests have been developed. POC tests now exist for many diseases and medical specialties, and are used in most medical contexts–from general outpatient clinics to intensive care units (Table 1).^18,19 As of 2012, nearly 100 companies worldwide were marketing, manufacturing, or developing test instruments or reagents capable of use at the clinical POC,²⁰ suggesting that POC testing, including novel nucleic acid-based POC tests, will become even more available and prominent in coming years.^21,22

Table 1.

Currently available diagnostic point-of-care tests.

Disease or Medical Specialty	Diagnostic Point-of-care Test
Cardiology	Brain Natriuretic Peptide;128 Creatine Kinase-MB;129 Human-type Fatty Acid Biding Protein;129 Myosin Light Chain-1;129 Myoglobin;129 N-Terminal Prohormone of Brain Natriuretic Peptide;130 Troponin I;129 Troponin T129
Endocrinology	Cholesterol;131 C-reactive Protein;66,132,133 Glucose;68 Hemoglobin A1c;134,135 Lactate; Urine Microalbumin
Gastroenterology	Fecal Occult Blood; Liver Function Tests136
Genetics	CYP2C 19*2 allele for anti-platelet therapy137
Hematology	D-dimer;130 Hemoglobin; Prothrombin time138
HIV/AIDS	CD4 T cell count;50,51 HIV Antigen; HIV Antibody38
Infectious Diseases (non-HIV)	African Trypanosomiasis;70 Chlamydia;139 Cryptococcus;60 Cryptosporidium;140 Falciparum-Malaria;56 Giardia;140 Group A Streptococcus;141 Hepatitis C;142 Influenza A & B; Parainfluenza; Respiratory Syncytial Virus; Schistosomiasis;143 Syphilis;62–64 Tetanus;69 Trypanosomiasis;144 Tuberculosis;52,53 Visceral Leishmaniasis71,72
Nephrology	Urinalysis; Urine Microalbumin; Serum Creatinine
Neurology	Nerve Conduction Device145
Obstetrics	Pregnancy and Ovulation Prediction Tests146
Pulmonology	Airflow Meters147
Substance Abuse	Blood Alcohol Level; Drugs of Abuse
Emergency Room	Serum Electrolytes; Medication Levels; Drugs of Abuse; Blood Alcohol Level; Troponin-I; Troponin-T; Lactate; Arterial Blood Gas
Intensive Care Unit	Serum Electrolytes; Ionized Calcium; Magnesium; Arterial Blood Gas; Blood pH; Glucose; Lactate; Hemoglobin; Prothrombin Time
Primary Care Clinic	Urinalysis; Pregnancy Test; Group A Strep; HIV Antibody; Fecal Occult Blood

The emergence of POC tests has the potential to improve health care services and patient-centered outcomes in diverse settings, particularly those with limited health service or laboratory infrastructure.^4,23 Access to improved diagnostic technologies in resource-limited settings (RLS) will bring unique challenges.²⁴ The development and design of user-friendly devices, regulatory approval and quality assurance programs, and product service and support all need to be addressed in novel ways, as reviews of POC testing in RLS have noted.^4,22,25–31 Moreover, in order to make prudent decisions about adopting new diagnostic technologies in RLS, decision-makers need well-executed studies on diagnostic accuracy, clinical impact, and costs.

We searched for and examined manuscripts containing definitions of POC testing and studies of POC tests in RLS. Based on this review, we provide a novel framework for evaluating diagnostic POC tests in RLS to encourage standardized reporting of performance and impact, which will enable more direct and accurate comparisons between POC technologies and their central laboratory-based counterparts.

Definition of Point-of-Care Diagnostic Tests

Several definitions of a POC test exist, based on a geographical, functional, technological, or operational context.³² An early definition was “a medical test that is conducted at or near the site of patient care.”¹³ Years later, another definition was “any test that is performed at the time at which the test result enables a clinical decision to be made and an action taken that leads to an improved health outcome.”³³ More recently, experts on HIV and tuberculosis diagnostic testing defined a POC test as “a diagnostic test that is performed near the patient or treatment facility, has a fast turnaround time, and may lead to a change in patient management.”²² Notably, none of these definitions privilege any particular technology or method of use, and POC tests do not require reagent-free operation, battery-powered operation, or a specific degree of operator training. All definitions emphasize the idea that POC testing is performed near the patient and leads to an expedited clinical decision.

In our view, a unified definition of POC would depend not only on the function of a test or device, but also on how and where the test or device is used. For example, a “rapid” HIV antibody test based on lateral flow and conducted by a technician in a central laboratory should not be considered POC testing, but the same test should be considered POC testing if performed in a timely manner during a clinical encounter resulting in a management decision. Some tests may be usable at the clinical point-of-care in certain settings, but may be limited to non-clinical sites in settings lacking physical infrastructure and trained users. Therefore, a POC test is defined as much by the process and workflow of diagnostic testing, as by the test itself. In addition, POC tests can also be used in small peripheral laboratories, as might be done with sputum smear microscopy, within primary health clinics.³⁴

There are stark differences in the usability of currently available diagnostic POC tests in different RLS, such as limited infrastructure, information systems, and human resources. Based on these disparities, WHO’s Sexually Transmitted Disease Diagnostics Initiative developed the ‘ASSURED’ criteria for an “ideal” POC test in RLS (Table 2).^3,35 Others have made additional suggestions, such as operational independence from training of laboratory personnel, specimen preparation (including a centrifuge or pipette), and/or significant physical infrastructure (including electricity, refrigeration, and/or running water).^4,22,28,36 Regardless of the usability of an individual POC test, the goal of POC testing is to improve patient outcomes. Therefore, meeting the “ideal” criteria may be rather arbitrary if a POC test is performed near the patient and leads to an expedited clinical decision, which improves patient outcomes.¹⁸

Table 2.

World Health Organization’s ASSURED criteria of ideal characteristics for a point-of-care test in resource-limited settings.

Affordable by those at risk of infection Sensitive (few false-negatives) Specific (few false-positives) User-friendly (simple to perform and requiring minimal training) Rapid (to enable treatment at first visit) and Robust (does not require refrigerated storage) Equipment-free Delivered to those who need it

Studies of Point-of-Care Tests in Resource-Limited Settings

Many POC tests have been designed for use in developed countries, and their application may not be readily transferrable to RLS.⁴ When POC tests are used in resource-replete emergency departments or intensive care units, the goal of POC testing is to obtain immediate test results to help guide an emergent intervention. In RLS, where limited access to care is one of the major reasons for the failure of health services,³⁷ POC testing is more applicable in outpatient clinics or mobile testing units. In these settings, the goal of POC testing is to expedite diagnostic testing without requiring the services of a remote clinical laboratory in order to can accelerate treatment initiation, increase access to next steps in care, and improve health outcomes. POC tests often have properties that could be generalizable across different epidemiological settings and clinical scenarios. We searched and examined studies of POC tests in RLS; here, we highlight studies that evaluated the relevant patient-centered measures.

The development of POC tests for use in RLS has primarily focused on infectious diseases that require prompt diagnosis and treatment, such as HIV, tuberculosis, and malaria. For example, rapid HIV antibody tests are accurate and reliable,^38–41 and increase the proportion of pregnant women who receive their test result.^42,43 Studies of POC CD4 count tests have demonstrated good agreement with lab-based testing,^44–49 and two studies have demonstrated accelerated time to antiretroviral therapy initiation and a higher rate of people initiating antiretroviral therapy.^50,51 Novel POC tests for pulmonary tuberculosis have shown promising sensitivity for detecting disease,^52–54 and Xpert MTB/RIF testing accelerated time to treatment for smear-negative tuberculosis.⁵⁵ A POC test for Plasmodium falciparum malaria demonstrated good agreement with thick smear microscopy,^56,57 while reducing overdiagnosis of malaria among children presenting with fevers.^58,59 A rapid test for cryptococcal antigen has shown strong agreement with laboratory-based titers,^60,61 but currently lacks field-based evaluations of patient-centered outcomes. Several rapid syphilis tests have shown high specificity in comparison to a fluorescent treponemal antibody absorption test, but have also not been evaluated for patient-centered outcomes.^62–65 Additional validation studies of POC tests in RLS include a C-reactive protein test among neonates suspected of meningitis in South Africa,⁶⁶ a glucose test to diagnose diabetes in rural India⁶⁷ and hypoglycemia in Nigerian children,⁶⁸ a tetanus immunity test in Iran,⁶⁹ a test for trypanosomiasis in Angola,⁷⁰ and a test for visceral leishmaniasis in Brazil, Kenya, Sudan, and India.^71,72 However, while a phased approach to evaluating diagnostic POC tests is appropriate, few of these studies went beyond the first step of accuracy assessment to measure the clinical impact of utilizing the POC test in the appropriate clinical setting.

Most published studies of POC tests in RLS have focused on analytical performance, while few studies have evaluated a test’s impact on patient-centered outcomes. In addition, very few studies of analytical performance evaluated the POC test when operated at the clinical point-of-care, and we identified only one study that assessed the detailed costs of implementing a POC test in a RLS.⁷³ In our view, a direct comparison of the performance characteristics between a POC test and its laboratory-based counterpart in a controlled laboratory environment is not sufficient. Instead, evaluations of POC tests should be comprehensive and include accuracy at the clinical point-of-care, impact on relevant patient-centered outcomes (such as time to initiation of therapy,⁷⁴ retention in care, and mortality), as well as differences in cost and cost-effectiveness. Since there is no widely accepted framework for measuring and reporting these parameters, we present a standardized methodology for evaluating POC tests in RLS in the following section.

Evaluating Point-of-Care Tests in Resource-Limited Settings

Accuracy

As POC testing expands into RLS, conducting appropriate evaluations of accuracy is paramount. A fundamental criterion for success of any diagnostic POC test is its accuracy and reliability.⁷⁵ Studies of test performance are usually the first and most frequent to appear in the medical literature, and they estimate the accuracy of any diagnostic POC test by comparison to an accepted gold standard test or the next best proxy measure. However, test results from a controlled laboratory environment may not be adequate or appropriate, since the test performance could significantly differ when a POC test is operated at the clinical point-of-care.⁷⁶

The accuracy of any POC test may be reduced when operated in a clinical setting than in a laboratory environment.⁷⁷ Analyses of diagnostic POC tests that rely on laboratory-based evaluations may give an inaccurate representation of a test’s performance characteristics in a real world setting. Evaluation of a POC test by technicians in a controlled laboratory environment may be as erroneous as attempting to evaluate a laboratory-based test by nurses in a clinical setting. The most appropriate measure of accuracy will be generated when a POC test is evaluated in the setting and location in which it will be used (i.e., at the clinical point-of-care).

For binomial test results, the most appropriate statistical measures of diagnostic accuracy are sensitivity and specificity, which by conventional teaching have been considered inherent values to a test and therefore amenable to comparisons across various populations and disease prevalence. However, a recent analysis suggested that both sensitivity and specificity can be influenced by prevalence of disease, which may be partially accounted for by recruiting patients at various stages of disease progression.⁷⁸ Since sensitivity and specificity are not always suitable for clinical decision-making, additional analyses should include likelihood ratios, receiver operating curves (ROC), positive predictive value (PPV), and negative predictive value (NPV) (Table 3).

Table 3.

Evaluating diagnostic point-of-care tests for use in resource-limited settings.

	Measure	Ideal Reporting Requirements
Diagnostic Accuracy	Binomial Test Result	Sensitivity and Specificity
		Likelihood Ratio Positive and Negative
		Receiver Operating Curve
	Continuous Test Measure	Bland-Altman Plot
		Limits of Agreement
Clinical Impact	Patient-centered Outcomes	Number of people initiating treatment
		Time to treatment initiation
		Number of visits required
		Percentage retained in care
		Improved adherence
		Reduced morbidity/mortality
Costs	Near-Universal Costs	Total material costs per test
		Price and lifespan of device/equipment
		Time to conduct each test
	Location-Specific Costs	Required training for personnel
		Salary of health worker performing test
		Transportation of specimens
		Costs for quality control
	Common Hidden Costs	Local taxes
		Cost of shipping and storing materials
		Discount rate

The likelihood ratio positive (LR+) is a useful value to assess the overall clinical value of a diagnostic test.^79–81 A likelihood ratio can be applied to the clinical pre-test probability (or odds) of a patient having a disease to estimate the post-test probability (or odds) of the disease. Similarly, the likelihood ratio negative (LR−) allows clinicians to interpret the results of the diagnostic test in order to predict the absence of disease.⁸²

The PPV and NPV are directly proportional to the prevalence of the disease or condition in the study population, and are therefore not considered intrinsic to a diagnostic test’s characteristics. Both PPV and NPV will vary by geographic regions, due to varying patient characteristics, which limits their suitability for comparisons between different study populations.⁸³ While these values are generally most interpretable by clinicians, they should be extrapolated to other populations with a great deal of caution.

Most POC tests have binary outcomes, but some have continuous data measurement, such as CD4 cell counts. Evaluating and reporting diagnostic accuracy for tests with continuous data measurements requires different statistical techniques. For measurements in which both the POC test and the gold standard test have some intrinsic error, a Bland-Altman plot may provide the best assessment of diagnostic accuracy and allow for an accurate comparison of agreement.^47,84,85 In addition, the limits of agreement should be calculated and reported. Another feasible comparative method is using the percentage similarity between data pairs, and not the absolute differences, represented as a histogram.⁸⁶ The coefficient of variation, which is used to estimate either the within-laboratory or between-laboratory reproducibility, should also be reported.⁸⁷

Correlation coefficients may be appropriate for continuous test measurements,⁶⁰ but they often do not account for variation among the samples or differences at the ends of a testing spectrum.^88,89 In addition, correlation tends to be high for studies that measure a broad range of values, but will miss systematic differences or bias.⁹⁰ In most instances, reporting a Bland-Altman plot along with the limits of agreement will be a more accurate representation of the data.^87,89

In addition to evaluations of test accuracy, additional operational characteristics of POC tests should be assessed in a standardized manner in order to facilitate comparisons with other care strategies. Method of quality control, need for disposable materials, and specifics of operability are examples of additional metrics for POC tests that should be evaluated as part of comprehensive assessments of POC tests in RLS. As Lehe et al. suggest, a “scorecard” for direct comparisons between POC technologies could help to standardize such comparisons.⁹¹

Clinical Impact

After assessing diagnostic accuracy of a POC test when used at the clinical point-of-care, the next step would be evaluating the clinical impact of POC testing on patient-centered outcomes. A POC test with seemingly inferior diagnostic accuracy when compared to a reference laboratory test may still be a valuable test that warrants clinical evaluation. Since the goal of POC testing is to expedite a clinical decision to improve patient outcomes, a POC test that is neither as sensitive nor as specific as a reference standard test may still have important clinical and public health benefits, if clinicians and patients can act on the results more frequently or quickly. In certain settings, POC tests can triage patients for an effective intervention, more resource-intense care, or more comprehensive diagnostic testing. However, these studies and implementation programs must be done within settings that can accommodate POC testing programs, including the ability to rapidly report and act on a test result, and these studies can be conducted as pre-/post- study designs or even as retrospective studies.

Clinical impact is not the same as diagnostic accuracy. In RLS, POC testing increases access to test results, which extends the clinical impact of the test.²² Furthermore, direct comparisons of the diagnostic accuracy between a POC test and its laboratory-based standard can be misleading. A more comprehensive assessment of the trade off between accuracy and usefulness of the test involves an evaluation of the impact on relevant patient-centered outcomes, such as time to treatment initiation, retention in care, or mortality (Table 3). As with clinical drug trials, a longitudinal study design or randomized controlled trial to compare POC testing versus laboratory-based testing may provide the most accurate depiction.^92,93 However, these studies are expensive and time consuming, and may not reflect ‘real-world’ outcomes if a study is overly monitored or highly controlled.

Since a fundamental advantage of POC testing in RLS is to provide clinicians and patients with an immediate test result, the primary outcomes should be based on the most relevant clinical decisions. The goal of all diagnostic POC tests is to improve access to care, but the mechanism will depend on the nature of the test and clinical question. For example, if a CD4 cell count is used to decide whether to initiate antiretroviral therapy, then an evaluation of POC testing for CD4 count should measure both the time to and proportion of people initiating treatment, as compared to a laboratory-based testing strategy.^50,51 For P. falciparum malaria and cryptococcal infections, both diseases with high mortality rates in the absence of immediate treatment, POC testing is used to accelerate treatment initiation in order to reduce mortality. Therefore, the most relevant clinical outcomes for studies of these infections would be time to treatment initiation and mortality rate. Other POC tests may be used to improve patient adherence to medications or linkage to medical care.

Since the goal of POC testing is to expedite a clinical decision, a POC test that performs as accurately as a laboratory-based test may be far superior for improving patient-centered outcomes. In fact, a POC test with somewhat reduced diagnostic accuracy might still outperform a laboratory-based test for achieving the primary clinical outcomes of interest. This would especially be true in settings with high loss to follow-up rates and poor retention in care. For example, when considering other clinical benefits, such as immediate treatment initiation to reduce transmission or the ability to pre-screen patients prior to more specific testing, a POC test could be more useful than the more accurate laboratory-based test. Therefore, decisions about whether to investigate the clinical impact of a POC test should be based on the relative importance and potential impact of the test, not on the comparison of diagnostic accuracy with the laboratory-based test. Balanced with these potential advantages are the challenges associated with maintaining the quality of POC testing; laboratory quality assurance (QA) programs face significant challenges when they need to be implemented in remote settings. Without appropriate QA practices, POC tests can undermine their very purpose of improved access to health care.

Measurement of the relevant clinical outcomes can then be used with the diagnostic accuracy of the POC test to calculate the “test efficacy.” If “vaccine efficacy” is the percentage of persons who are protected by a vaccine, and “drug efficacy” is the ability of a drug to produce a desired effect or specific response,⁹⁴ then diagnostic “test efficacy” would be the ability to produce an effect or response by administering a diagnostic test.^95,96 The “test efficacy” then combines the diagnostic accuracy of the POC test along with clinical effectiveness of POC testing. We can further define diagnostic “test efficacy” using the following formula:

$$\mathit{Test}\mathit{Efficacy}=\mathit{Likelihood}\mathit{Ratio}\mathit{Positive}\times \mathit{Patient}\mathit{Notification}\mathit{Rate}of\mathit{Result}$$

where “Likelihood Ratio Positive”, calculated as LR+ = sensitivity/(1 − specificity), refers to the estimated test accuracy of the POC test when operated in an appropriate clinical setting. “Patient Notification Rate of Result” reflects the percentage of patients receiving their test result over a defined time period. The defined period of time should be consistent and determined a priori when comparing a POC test against its lab-based counterpart. However, the defined time period may vary depending on the severity of the disease. For example, serious life-threatening infections, such as P. falciparum malaria or cryptococcal meningitis, might measure patient notification rate within 24 hours. Less acute infections, such as HIV or tuberculosis, might define a time to patient notification at 72 hours. More chronic disease, such as certain rheumatological conditions, may define patient notification rate within 1 week of testing to reflect fewer consequences of delayed diagnosis using certain tests, such as rheumatoid factor or antinuclear antibody. A definitive time point must be chosen, and the metric of ‘patient notification rate’ was chosen over other possible patient-centered outcomes, such as ‘treatment initiation’, since negative test results may not require treatment, medications may be unavailable, and some patients may refuse treatment. However, another valuable metric to measure and report is ‘provider receipt of test results’, which also captures the benefit of POC testing.

By using this formula, we can then evaluate and compare the efficacy of implementing POC testing in RLS. For example, if similar POC and laboratory-based tests have a LR+ = 10, but the POC test has a 95% patient notification rate within 48 hours while the laboratory-based test has a 70% patient notification rate within 48 hours,⁹⁷ then the test efficacy measures would be 9.5 (10 × 0.95) and 7.0 (10 × 0.70), respectively. In this scenario, the POC test had a higher test efficacy, which was due to more people receiving the test result within 48 hours. Even if the LR+ = 8, then the calculated test efficacy (7.6) suggests that POC testing would still be superior to laboratory-based testing, when assuming the same patient notification rates (95% vs. 70%). In addition, “rule out” tests, which rely on the LR− value, may define ‘negative test efficacy’ as the LR− value multiplied by the inverse of the ‘patient notification rate’. The inverse of the ‘patient notification rate’ would be needed, since smaller LR− values represent a better test. Thus, a measure like “test efficacy” can summarize the primary outcome of interest of POC testing–a combination of performance and patient outcome–while accounting for delays (time), inaccuracies (test result), and the clinical consequences of missed or delayed diagnosis.

Cost Analysis

A comprehensive evaluation of accuracy and clinical impact for a novel diagnostic POC test does not provide health officials with enough information to determine whether implementing POC testing represents good value. Cost analyses should be utilized to determine the incremental cost per diagnostic POC test, which can then be analyzed alongside accuracy and clinical impact–or test efficacy–in cost-effectiveness models. The usefulness of cost analyses will depend on applying methods that are appropriate for specific technologies and applied consistently among study settings and locations.⁹⁸ Caution has to be exercised when extrapolating cost data from high-income areas, and transparency in reporting all cost data will facilitate application to resource-limited settings.

The basic goal of a cost analysis is to quantify the cost per test, which includes materials, equipment, health worker time, and logistical expenses, and commonly divided into near-universal and location-specific costs (Table 3). Some costs, such as the manufacturer’s price per test or time to operate the test, may be similar among study locations. Other costs, such as staff salary, transportation costs, and quality assurance programs, will vary greatly between countries and locations. Common hidden costs also need to be taken into account. In order to have an accurate comparison between a POC test and its laboratory-based counterpart test, all costs must be identified for both testing modalities.

The estimated cost of an individual test typically includes more than the single test device cost. Although seemingly trivial, estimated cost per test should include materials used to obtain the biological specimen (i.e. alcohol swab, lancet, gauze, specimen container, medical gloves), test consumables, and materials for reporting or printing the test result. An estimated failure rate for the test, which depends on the technology being used, must also be factored into the analyses, since it necessitates either repeating or a confirmatory test. Studies of Xpert MTB/RIF use in a reference laboratory setting had an indeterminate rate of 2.4–3.7% for diagnosis of pulmonary tuberculosis,^52,55 while a recent POC CD4 count tests have had failure rates (or indeterminant test results) between 3–23%.^47,99–103 The standard equipment costs must include the base cost, maintenance of the equipment, and optional service plans to cover repairs or failures after a warranty period, which can then be amortized over the working life of the equipment. Risks of equipment being stolen or broken are greater in certain locations, which makes estimates of the average working life even more uncertain.

The cost per test also depends on the time required to operate the test and the salary of the health worker(s) conducting the test, which can be summarized as staff costs per test. If workers are able to process multiple patient samples simultaneously, then estimates should reflect the average time for one test. POC testing may also need to include the time required for counseling following the test result. The time per test will be calculated along with the health workers’ salary to obtain the cost per test. A health worker salary per working hour can be deduced from a worker’s annual salary, but should include benefits and intangibles. The estimates of time to conduct a test and estimated staff costs per tests, which vary both within and between countries, may lend themselves to further scrutiny by sensitivity analyses, but transparency in reporting these estimates is essential.

Some costs may be hidden. First, some POC tests and most laboratory equipment require regular quality control measures. For example, Alere’s PIMA CD4 test requires a daily quality control test by inserting a standardized cartridge, which needs to be replaced every six months.⁷³ For the PIMA, the additional costs per test may be calculated by dividing the cost of the standard cartridge over the number of tests conducted during a six-month period. Second, cost analyses need to include both transportation and local taxes for the supply chain management, as well as purchasing supplies and equipment. Third, applying an appropriate discount rate when evaluating new equipment purchases or investments is more complicated, and beyond the scope of this manuscript.¹⁰⁴ Finally, and perhaps most importantly, cost analyses should consider the opportunity costs in terms of reducing reliance on laboratory-based infrastructure in certain settings.

Discussion

The popularity and potential applications of diagnostic POC testing in RLS have been growing and are likely to continue expanding. The goal of introducing a POC test is to provide a faster test result to clinicians and patients so that they may make an expedited clinical management decision, in order to improve patient outcomes and overall public health.¹⁸ Better patient outcomes can still be achieved at the expense of decreased test performance compared to a laboratory-based test, if, for example, the POC test achieves better patient outcomes through improved linkage to care and retention in treatment.

Improved access to care is needed in RLS, since patients often travel for hours to reach clinics with medical staff and supplies, but without capabilities for laboratory testing.⁴ Furthermore, existing laboratory structures that are reachable by patients may not be staffed or operating, or broken equipment may lack regular servicing or repair.^105,106 When tests are processed in a laboratory, the test results do not always reach patients or support management decisions, such as an initiation or change of therapy. For example, in South Africa only 69% of newly diagnosed HIV-infected adults received a laboratory-based CD4 count within 90 days, and less than half of those whose results made them eligible to start antiretroviral therapy did so within the following 12 months.⁹⁷ Other studies in sub-Saharan Africa found that roughly 40% of patients diagnosed with HIV do not obtain a CD4 count within 90 days, which can lead to life-threatening delays in antiretroviral therapy initiation.^107–110

Limited availability of laboratory-based diagnostic tests will continue to be harmful in RLS. Syndromic management – i.e., management based on clinical history, unsupported by laboratory testing – can be effective in limited circumstances; but may also result in underdiagnosis and/or overtreatment, which further exacerbates problems of antimicrobial resistance and drug toxicity.^111,112 Conversely, undertreatment can be a problem if treatable diagnoses are missed. Delays in diagnostic testing can lead to higher loss-to-follow-up rates and transmission of infectious pathogens, as well as higher incidence and mortality rates.^2,113 For these reasons, POC testing is needed in RLS, particularly those with underdeveloped laboratory infrastructures and high rates of patient attrition.²²

A new POC test must be rigorously evaluated on the basis of test accuracy, clinical impact, and costs with appropriate and comprehensive reporting of study results.¹¹⁴ Using an approach like the one we have outlined, the routine reporting of “test efficacy” would enable comprehensive evaluations of new POC technologies and comparisons with standard of care testing. In the absence of empiric data on these parameters, comparison studies will be required to make certain assumptions in cost-effectiveness analyses. Cost-effectiveness analysis to compare such strategies can help to determine whether the value of introducing a novel POC test or technology would be worth investing scarce resources for local, regional, or national programs.

Cost-effectiveness analyses account for test accuracy, clinical impact, and costs to determine how a new, often more expensive, diagnostic or treatment modality compares in value to other uses of similar resources. For diagnostic POC testing, cost-effectiveness analyses can estimate whether the cost of a POC testing strategy offers increased value for the expected clinical outcomes. These analyses can also be adapted to various epidemiological and economic settings. Recent cost-effectiveness analyses have shown the following POC tests to be of good value in RLS: routine HIV screening,¹¹⁵ novel diagnostic tools for TB diagnosis,¹¹⁶ the Xpert MTB/RIF test,^117,118 and routine HIV plasma viral load for monitoring antiretroviral therapy.¹¹⁹ While cost-effectiveness analysis may be best conducted with simulation models, some indirect benefits may still be difficult to measure.^120,121

Efforts remain underway to develop standards for diagnostic POC tests.^122–124 The WHO’s Grading of Recommendations Assessment, Development, and Evaluation (GRADE) process has traditionally relied solely on diagnostic accuracy of tests, and does not account for a POC test’s important gains on clinical impact. Cobelens et al. have suggested that the system be revised for POC testing, by placing more weight on clinical impact than diagnostic accuracy, and to develop a pathway for both technical and programmatic policy recommendations.³⁰ Others have suggested similar measures of “patient outcome efficacy”, as well as assessing the overall public health impact.^125–127 Additional measures of value may be analytic performance, clinical utility, and field applicability assessments of POC tests. We support these recommendations and also suggest a revised set of “ideal,” but not absolute, criteria for a diagnostic POC test for RLS (Table 4). Most notably, these criteria include impact on relevant clinical outcomes (test efficacy) and cost-effectiveness, both of which would provide a more comprehensive assessment of the value of new diagnostic POC technologies for RLS. Furthermore, cost-effectiveness analysis can also project the outcomes of failing to implement certain POC tests and programs.

Table 4.

Suggested revised criteria for an ideal diagnostic point-of-care test in resource-limited settings.

Allows an expeditious clinical decision Capable of use at the clinical point-of-care by health workers Affordable (low average cost per test) Rapid (provides result during a clinic visit or within a reasonable waiting time) Acceptable Test Efficacy Cost-Effective

In conclusion, diagnostic POC testing can accelerate clinical management and improve patient-centered outcomes in situations with limited availability of a laboratory or highly trained staff. As POC tests are being increasingly designed for use in RLS, the approach to evaluating POC tests will be different from the experience of POC testing in developed countries. For this reason, POC tests should be rigorously evaluated for the patient-centered outcomes of interest and in the setting for which the test was designed. Therefore, accurate and complete data regarding a test’s clinical utility, quality, and potential impact on patient-centered clinical outcomes should be available before widespread implementation. Rigorous evaluations of the many novel POC tests, along with test efficacy and cost-effectiveness analyses, will help determine their utility in various RLS. Although there will remain challenges to adoption and scale-up of POC testing,¹⁹ they should not be seen as a deterrent to improving patient care, but instead an emerging field that can greatly improve access to care and patient-centered outcomes for people in RLS.

Search strategy and selection criteria.

We searched PubMed with the terms “point-of-care”, “diagnostic tests”, and “resource-limited settings” for all available articles without time period restrictions through March 2013. We selected case reports, case series, epidemiological studies, and reviews of human diseases published in English. We also reviewed references from selected publications.