Evaluating Point-of-Care Nucleic Acid Tests in Adult Human Immunodeficiency Virus Diagnostic Strategies: A Côte d’Ivoire Modeling Analysis

Abstract

Background

The World Health Organization (WHO) human immunodeficiency virus (HIV) diagnostic strategy requires 6 rapid diagnostic tests (RDTs). Point-of-care nucleic acid tests (POC NATs) are costlier, less sensitive, but more specific than RDTs.

Methods

We simulated a 1-time screening process in Côte d’Ivoire (CI; undiagnosed prevalence: 1.8%), comparing WHO- and CI-recommended RDT-based strategies (RDT-WHO, RDT-CI) and an alternative: POC NAT to resolve RDT discordancy (NAT-Resolve). Costs included assays (RDT: $1.47; POC NAT: $27.92), antiretroviral therapy ($6–$22/month), and HIV care ($27–$38/month). We modeled 2 sensitivity/specificity scenarios: high-performing (RDT: 99.9%/99.1%; POC NAT: 95.0%/100.0%) and low-performing (RDT: 91.1%/82.9%; POC NAT: 93.3%/99.5%). Outcomes included true-positive (TP), false-positive (FP), true-negative (TN), or false-negative (FN) results; life expectancy; costs; and incremental cost-effectiveness ratios (ICERs: $/year of life saved [YLS]; threshold ≤$1720/YLS [per-capita gross domestic product]).

Results

Model-projected impacts of misdiagnoses were 4.4 years lost (FN vs TP; range, 3.0–13.0 years) and a $5800 lifetime cost increase (FP vs TN; range, $590–$14 680). In the high-performing scenario, misdiagnoses/10 000 000 tested were lowest for NAT-Resolve vs RDT-based strategies (FN: 409 vs 413–429; FP: 14 vs 21–28). Strategies had similar life expectancy (228 months) and lifetime costs ($220/person) among all tested; ICERs were $3450/YLS (RDT-CI vs RDT-WHO) and $120 910/YLS (NAT-Resolve vs RDT-CI). In the low-performing scenario, misdiagnoses were higher (FN: 22 845–30 357; FP: 83 724–112 702) and NAT-Resolve was cost-saving.

Conclusions

We projected substantial clinical and economic impacts of misdiagnoses. Using POC NAT to resolve RDT discordancy generated the fewest misdiagnoses and was not cost-effective in high-performing scenarios, but may be an important adjunct to existing RDT-based strategies in low-performing scenarios.

Key Points.

We used a spreadsheet tool and the Cost-Effectiveness of Preventing AIDS Complications (CEPAC)–International model to simulate alternative HIV diagnostic testing strategies in Côte d’Ivoire. Point-of-care nucleic acid tests could be an important adjunct to rapid diagnostic test–based strategies.

With improved diagnostics and expanded human immunodeficiency virus (HIV) testing services over the past decade, 79% of people with HIV (PWH) globally were aware of their status in 2018 [1]. The World Health Organization (WHO) recommends a sequence of rapid diagnostic tests (RDTs) for HIV diagnosis [2]. Before 2019, this sequence depended on a country’s HIV prevalence. In response to declines in HIV testing positivity, the WHO now recommends all countries use 3 consecutive RDTs in order to minimize misdiagnoses [3]. This strategy requires a total of 6 RDTs for diagnosis and confirmation testing before antiretroviral therapy (ART) initiation.

RDTs are used widely for HIV diagnosis due to their high sensitivity (>99%) and specificity (>99%) and low cost [4]. However, RDT sensitivity and specificity may differ between laboratory and field settings [5, 6]. Program audits evaluating RDT-based testing strategies report substantial false-negative (FN, up to 9%) and false-positive (FP, up to 10%) results [5, 6]. Additionally, RDT sensitivity is diminished in acute HIV infection, when anti-HIV antibodies are still developing. With multiple RDTs required for diagnosis and limited capacity at HIV clinics, RDT-based testing strategies are prone to attrition and result delays [7].

Emerging point-of-care nucleic acid tests (POC NATs) may improve the accuracy of current testing strategies when implemented in combination with RDTs. NATs detect HIV RNA or DNA rather than HIV antibodies, which could improve diagnosis for individuals with acute HIV infection in whom antibodies have not yet developed. Compared to RDTs, POC NATs have lower sensitivity (95%), except during acute HIV infection, and greater specificity (100%) [4]. POC NATs have also been demonstrated to reduce time to diagnosis in adult populations receiving discordant RDT results [8]. Although POC NATs may require electric supply and may have longer run times compared to RDTs, they utilize single-use cartridges, minimize cross-reactivity between samples, and are automated (reducing human intervention and/or error) and digital (eliminating the user interpretation that is needed with RDT) (Table 1). They require only a few hours of training and may be run by nurses and/or nonlaboratory technicians [9]. However, concerns have also been raised about FN diagnoses if POC NATs are used for PWH on ART who present for testing without disclosing a known HIV diagnosis (ie, individuals who participate in community screening programs without disclosing previously identified HIV status) [10].

Table 1.

Technical and Implementation Comparisons Between Rapid Diagnostic Tests and Point-of-Care Nucleic Acid Tests for Human Immunodeficiency Virus Diagnosis

Characteristics	Rapid Diagnostic Testa	Point-of-Care Nucleic Acid Testb
Technical characteristics
Temperature constraints	Ambient (2°C–30°C) [4]	Ambient (2°C–40°C) [4]
Reagents	Test cassette, assay diluent, specimen transfer device [4]	Single-use cartridge [4]
Reagent shelf-life	18 mo [4]	9–12 mo [4]
Capacity for additional testing	None	Tuberculosis, HCV, early infant HIV diagnosis, HIV viral load monitoring [11–13]
Electric supply	None [4]	Required; some use battery power [4, 14]
Implementation characteristics
Assay mechanism	User prepares specimen and places on to colloid containing immobilized antigen (immunochromatography) [4]	User adds specimen to cartridge and instrument automates HIV RNA extraction, amplification, and detection via RT-PCR [4]
Training required	Minimal	Minimal, can be task shifted to nurses and/or nonlaboratory technicians [9, 12, 14]
Cross-reactivity	Immunochromatography poses risk for cross-reactivity	Negligible
Interpretation	User interprets the presence or absence of a reactive sample relative to control [4]	Digital [4]
Instrument run time	10–20 min [4]	52–90 min [4]
Time to result	Must be read <20 min [4]	56–115 min [4]
Cost (2018 USD)c	Low cost ($1.47/assay) [15]	High cost ($27.92/assay) [15]

Abbreviations: HCV, hepatitis C virus; HIV, human immunodeficiency virus; RT-PCR, reverse-transcription polymerase chain reaction; USD, United States dollars.

^aData shown for Alere Determine HIV-1/2, Chembio STAT-PAK HIV-1/2, and Abbott Bioline HIV-1/2 3.0 (most commonly used rapid diagnostic test assays in Côte d’Ivoire).

^bData shown for Cepheid Xpert HIV-1 Qual-1 and Abbott m-PIMA HIV-1/2.

^cApproximate the full cost of ownership, including device cost, service level agreement, personnel time, and training.

We previously found that despite greater testing costs, POC NAT–based testing strategies may require fewer assays than RDT-based strategies and improve diagnostic accuracy [16]. However, the long-term clinical and economic outcomes and cost-effectiveness of such strategies remain undetermined. In the present analysis, our objective was to determine the clinical and economic impact of misdiagnoses and the potential cost-effectiveness of a universal 1-time adult HIV screening process (ie, a 1-time universal screen) in Côte d’Ivoire (CI), a setting with a low prevalence of undiagnosed HIV, using existing RDT-based testing strategies and an alternative strategy incorporating POC NAT in combination with RDTs.

METHODS

Analytic Overview

Using a previously published spreadsheet-based tool and the Cost-Effectiveness of Preventing AIDS Complications (CEPAC)–International model [16–18], we simulated alternative HIV testing strategies for a universal 1-time screening process of all adults without a reported HIV diagnosis in CI (ie, excluding PWH already diagnosed), performed in addition to current Ivoirian HIV testing practices (ie, status quo). We chose to model this population rather than a subset of testers in order to evaluate alternative testing strategies on a national level. We projected the clinical and economic outcomes for true-positive (TP), FP, true-negative (TN), and FN results from the 1-time screen. We considered the negative impact of false results received from the HIV screen: clinical for an FN result (life expectancy lost compared to a TP diagnosis) and economic for an FP result (increase in HIV-related care costs compared to a TN diagnosis). We estimated misdiagnoses, CD4 cell count at diagnosis, primary transmissions averted, life expectancy, and costs for each strategy. We reported incremental cost-effectiveness ratios (ICERs: the difference in lifetime costs divided by the difference in life expectancy, discounted 3%/year) for each strategy compared to the next least costly alternative from the health care payer perspective. We defined a strategy as “cost-effective” if its ICER was <$1720/year of life saved (YLS) (2018 CI per-capita gross domestic product [GDP]) [19, 20].

Modeled Strategies

We modeled 3 strategies: (1) the WHO-recommended RDT-based testing strategy (RDT-WHO); (2) the nationally recommended RDT-based testing strategy in CI (RDT-CI); and (3) an alternative testing strategy using POC NAT to resolve RDT discordancy (NAT-Resolve) [16]. RDT-WHO, RDT-CI, and NAT-Resolve all begin with 2 consecutive RDTs but differ if the initial pair produces discordant results: RDT-WHO implements a third RDT, RDT-CI implements another pair of RDTs, and NAT-Resolve implements POC NAT (Supplementary Figure A). Per WHO recommendations, 3 additional RDTs are performed to confirm an HIV diagnosis (pre-ART retesting); we included the cost of these tests but as a simplifying assumption, assumed that pre-ART retesting would not resolve FP results (Supplementary Methods). Due to its reported near-perfect specificity and based on previous POC NAT implementation studies in adult populations, we assumed that pre-ART retesting would not follow a positive POC NAT result [8]. Based on in-country experience, we assumed that individuals with inconclusive results would have a sample sent to a reference laboratory instead of being asked to return in 14 days. This permitted comparison of strategies, limiting the difference to the management of initially discordant results. We modeled 2 scenarios: “high-performing” (test sensitivity/specificity from WHO prequalification reports) and “low-performing” (test sensitivity/specificity, attrition, and result delays from field reports).

Model Structure

The spreadsheet-based tool applies Bayes’ theorem, using HIV prevalence and diagnostic test sensitivity and specificity to estimate TP, FN, FP, and TN diagnoses, and weight clinical and economic outcomes projected by CEPAC [16]. The CEPAC model is a validated simulation model of HIV disease progression, testing, and treatment [17, 18].

HIV Disease and Testing.

Individuals “enter” the CEPAC model with a TP, FN, FP, or TN diagnosis and are simulated until death. PWH (TP or FN diagnosis) are assigned user-specified characteristics including CD4 cell count and HIV RNA. In the absence of effective ART, PWH experience a monthly CD4 decline and age- and CD4-dependent risks of opportunistic infection and mortality. For those carrying an FN diagnosis or who acquire HIV after an initial TN result, subsequent TP diagnosis can occur via routine HIV testing or upon seeking care for a severe opportunistic infection. PWH linking to care are prescribed ART and experience an initial probability of virologic suppression and subsequent increase in CD4. Those with virologic suppression face risks of later failure. PWH who become lost to follow-up experience a probability of returning to care or may present with an opportunistic infection. People with an FP diagnosis experience no risk of HIV infection or HIV-related events but accrue costs of ART and HIV-related care until death or loss to follow-up. Individuals with an FP diagnosis are assumed not to return to HIV care if lost to follow-up.

HIV Transmission.

We define primary HIV transmission as 1 generation of new infections transmitted from PWH. Differences in transmissions between strategies result from differences in FN diagnoses generated. We assume that primary HIV transmissions are diagnosed at current Ivoirian testing rates (Supplementary Methods).

Model Inputs

High-Performing and Low-Performing Scenario Inputs.

In the high-performing scenario, modeled RDT (third generation) and POC NAT sensitivity/specificity were 99.9%/99.1% and 95.0%/100.0%, respectively (acute HIV RDT sensitivity: 88.5%) (Table 2) [4, 21]. In the low-performing scenario, we assumed that RDT is more reliant on human performance and interpretation and thereby has lower sensitivity/specificity compared to automated POC NAT (91.1%/82.9% and 93.3%/99.5%, respectively). We modeled attrition (29% after repeat RDT testing and 32% after laboratory RDT testing) and result delays (1-month delay after laboratory RDT testing) [5, 14, 22, 23].

Table 2.

Model Input Parameters for a Simulated 1-Time Human Immunodeficiency Virus Screen of Adults in Côte d’Ivoire

Variable	Base Case Value	Range Examined	Source
Scenario-specific testing characteristics
High-performing scenario
RDT sensitivity, specificitya, %
Acute HIV	88.5, 99.1	66.0–88.5, 82.9–100.0	[4, 21]
Chronic HIV	99.9, 99.1	91.1–99.9, 82.9–100.0	[4]
POC NAT sensitivity, specificitya, %	95.0, 100.0	93.3–95.0, 99.5–100.0	[4]
Low-performing scenario
RDT sensitivity, specificitya, %
Acute HIV	66.0, 82.9	66.0–88.5, 82.9–100.0	[22]
Chronic HIV	91.1, 82.9	91.1–99.9, 82.9–100.0	[5, 23]
POC NAT sensitivity, specificitya, %	93.3, 99.5	93.3–95.0, 99.5–100.0	[14]
Attrition after repeat RDTs, %	29	…	[14]
Attrition after laboratory RDTs, %	32	…	[14]
Result delay after laboratory RDT	1 mo	…	[14]
Cohort characteristics
Initial age, y, mean (SD)
Undiagnosed, acute HIV	30 (9)	30–40	[18]
Undiagnosed, chronic HIV	32 (9)	32–42	[24]
Without HIV at model start	36 (15)	…	[25]
Male sex, PWH, %	29	20–50	[26]
Undiagnosed HIV prevalence, %	1.8	0.1–5.0	[26]
Acute HIV	0.9	0–100	[1, 26]
Chronic HIV	99.1	0–100	[1, 26]
HIV incidence, infections/1000 PY	1.1	…	[1]
Initial CD4 count, mean (SD), cells/µL
Acute HIV	667 (134)	…	[24]
Chronic HIV	550 (205)	100–550	[27]
Current Ivoirian HIV detection characteristics
Via routine testing, monthly probabilityb	0.003	0.0015–0.0060	[24]
Upon presentation with severe OIb, %	90	50–100	[24]
Linkage to care, probability	0.88	50–100	[26]
HIV treatment
ART efficacy: 48-wk virologic suppression, %
First-line: TDF + 3TC + DTG	98	…	[28]
Second-line: TDF + FTC + EFV	86	…	[28]
Third-line: ZDV + 3TC + LPV/r	75	…	[28]
LTFU, range by adherence to ART, monthly probability	0.005–0.030	0.5–2.0×	[29]
Costs (2018 USD)
Assay, per testc
RDT	1.47	0.74–2.94	[15]
POC NAT	27.92	13.96–55.84	[15]
Laboratory monitoring, per test
CD4 cell count test	11.88	5.94–23.76	[17]
HIV RNA test	37.11	18.56–74.22	[17]
ART, monthly, range by ART regimen	6–22	0.5–2.0×	[30]
HIV care costs, monthly, range by CD4 cell count	27–38	0.5–2.0×	[31]

Abbreviations: 3TC, lamivudine; ART, antiretroviral therapy; DTG, dolutegravir; EFV, efavirenz; FTC, emtricitabine; HIV, human immunodeficiency virus; LPV/r, lopinavir/ritonavir; LTFU, loss to follow-up; OI, opportunistic infection; POC NAT, point-of-care nucleic acid test; PWH, people with human immunodeficiency virus; PY, person-years; RDT, rapid diagnostic test; SD, standard deviation; TDF, tenofovir disoproxil; USD, United States dollars; ZDV, zidovudine.

^aSensitivity and specificity estimates are averages of multiple testing platforms that had overlapping 95% confidence intervals.

^bInputs estimated from published data regarding age and CD4 cell count at HIV acquisition and detection (Supplementary Methods). Severe OIs include World Health Organization stage 3 and 4 clinical events and tuberculosis.

^cCost estimates are averages of multiple testing platforms.

Base-Case Definition.

Base-case cohort characteristics and HIV testing, natural history, treatment, costs, and transmission did not vary between modeled testing strategies or scenarios (Table 2).

Cohort Characteristics.

We modeled adults with unknown HIV status undergoing HIV screening in CI. At model start, mean age was 30 years (standard deviation [SD], 9 years) for acute HIV and 32 years (SD, 9 years) for chronic HIV (Table 2, Supplementary Methods) [18, 24]. The mean age for people without HIV was greater (36 years; SD, 15 years) because it represented the mean age among all adults in CI [25]. Among PWH, 29% were male [26]. Undiagnosed HIV prevalence was 1.8% and HIV incidence was 1.1/1000 person-years based on the CI Population-Based HIV Impact Assessment and Joint United Nations Programme on HIV/AIDS reports [1, 26]. CD4 counts at model start were 667 cells/µL (SD, 134) and 550 cells/µL (SD, 205) for acute and chronic HIV, respectively [24, 27]. We applied results to the population eligible for testing in CI (24.5 million) [1, 19], and reported in units per 10 million people to demonstrate small differences between strategies.

HIV Testing, Natural History, and Treatment.

We derived current Ivoirian practices of HIV detection (0.003 monthly probability via routine testing and 90% HIV detection after developing a severe opportunistic infection) by calibrating to published age (38 years) and CD4 count at detection (254 cells/µL) (Supplementary Methods) [24]. From published data, we derived CD4 decline for PWH not on effective ART and AIDS-related and non-AIDS-related mortality (Supplementary Table A). We modeled HIV care based on 2019 CI guidelines, including 3 ART lines, opportunistic infection prophylaxis, and annual CD4 and HIV RNA monitoring (Table 2, Supplementary Table A).

HIV Transmission.

Transmission rates (0–65.5/100 person-years) varied by HIV RNA level, with the highest rates for those with acute infection and not on ART (Supplementary Table A).

Costs.

RDT ($1.47) and POC NAT ($27.92) assay costs were derived from the United Nations Children’s Fund, including device cost, service-level agreements, personnel time, and training [15]. For all PWH (including primary transmissions), HIV-related costs included laboratory monitoring, opportunistic infection prophylaxis and treatment, ART, and HIV care (Table 2, Supplementary Table A).

Clinical and Economic Impact of HIV Misdiagnoses.

The clinical impact of FN diagnoses was based on current Ivoirian HIV testing parameters: receiving an FN vs a TP diagnosis at the time of the screen led to 4.4 mean undiscounted life-years lost. The clinical impact of FP diagnoses was based on loss to follow-up input parameters: mean time spent with an FP diagnosis was 10.3 years (SD, 0.01 years); the median was 7.2 years (interquartile range, 2.3–18.5 years). Compared to receiving a TN diagnosis, an FP diagnosis increased discounted lifetime costs by $5800.

Sensitivity Analyses

We varied RDT and POC NAT sensitivity and specificity to determine their impact on projected misdiagnoses generated by each strategy. We next varied individual parameters (Table 2, range examined) in the high-performing scenario to examine when NAT-Resolve was preferred. Independent of testing strategy or scenario, we varied Ivorian HIV detection parameters (affecting life-years lost from an FN vs TP diagnosis), loss to follow-up parameters (affecting the time spent with an FP diagnosis), and both cost and loss to follow-up parameters (affecting the costs of an FP vs TN diagnosis). In multiway sensitivity analyses, we varied the clinical and economic harms of HIV misdiagnoses to examine their impact on cost-effectiveness outcomes.

Additional Scenario Analyses

We modeled “intermediate” scenario analyses with assay parameters lying between high-performing and low-performing scenarios. We also varied the proportion of FP diagnoses resolved by pre-ART retesting in the low-performing scenario. In a “test-again” scenario, we varied the proportion of PWH in the high-performing scenario who presented for testing with a previous, unreported HIV diagnosis (0% [high-performing scenario] to 39% of all PWH who receive the 1-time screen) [10]. Of PWH previously diagnosed, we assumed that 53% were on ART, and among these, 76% were virologically suppressed (Supplementary Table A). In this test-again scenario, we modeled lowered assay sensitivity with ART and/or virologic suppression (RDT sensitivity on/off ART: 95.2%/99.9%; POC NAT sensitivity on ART and virologically suppressed/not virologically suppressed: 0%/95.0%) (Supplementary Table A).

RESULTS

Clinical Outcomes: High-Performing and Low-Performing Scenarios

In the high-performing scenario, NAT-Resolve generated the fewest misdiagnoses per 10 000 000 adults tested (409 FN and 14 FP), followed by RDT-CI (413 FN and 28 FP) and RDT-WHO (429 FN and 21 FP) (Table 3). Any of the 1-time screening strategies compared with current Ivoirian testing practices increased mean CD4 count at diagnosis: 500 vs 225 cells/µL. Discounted life expectancy for the tested population (including PWH and those without HIV) was similar, within rounding, across strategies: 228.60 months (Table 3, Supplementary Table B).

Table 3.

Clinical and Cost-Effectiveness Outcomes of a Simulated 1-Time Screen in Côte d’Ivoire: High-Performing and Low-Performing Scenarios

	Misdiagnosesb		Life Expectancyc			Per-Person Lifetime Costsc		ICERd
	FN	FP	PWH	Tested Population	Tested Population	Tested Population	Tested Population	Tested Population
Strategya	No.	No.	Months, Undiscounted	Months, Undiscounted	Months, Discountede	USD 2018, Undiscounted	USD 2018, Discountede	$/YLS
High-performing
RDT-WHO	429	21	358.16	388.29	228.60	458.16	222.89	Comparator
RDT-CI	413	28	358.16	388.29	228.60	458.19	222.90	3450
NAT-Resolve	409	14	358.16	388.29	228.60	458.40	223.11	120 910
Low-performing
NAT-Resolve	22 845	83 724	351.59	388.17	228.52	517.53	271.56	Cost-saving
RDT-WHO	28 479	104 439	349.88	388.14	228.49	527.11	278.62	Dominated
RDT-CI	30 357	112 702	349.35	388.13	228.49	532.69	283.03	Dominated

Abbreviations: CI, Côte d’Ivoire; FN, false-negative; FP, false-positive; ICER, incremental cost-effectiveness ratio; NAT, nucleic acid test; PWH, people with human immunodeficiency virus; RDT, rapid diagnostic test; USD, United States dollars; WHO, World Health Organization; YLS, year of life saved.

^aStrategies are listed in order of increasing lifetime costs, per cost-effectiveness convention.

^bOutcomes are reported for a tested population of 10 000 000.

^cLife expectancy is rounded to 2 decimals. Costs are rounded to the nearest cent.

^dICERs (rounded to the nearest $10) are used to summarize the cost-effectiveness of an intervention: the degree to which the intervention provides benefit relative to its cost. An ICER is the difference in cost divided by the difference in life expectancy for each strategy compared with the next least costly strategy. If a strategy is less effective and more expensive compared to another strategy, then it is said to be “strongly dominated.” A strategy is “cost-effective” if it is not dominated by any other strategy and it has the largest ICER not exceeding the willingness-to-pay threshold. The willingness-to-pay-threshold is a normative value that varies widely by setting and decision-maker; for interpretability, we have chosen the per-capita GDP in CI ($1720/YLS).

^eResults are discounted 3%/year.

Compared to the high-performing scenario, in the low-performing scenario, FN diagnoses increased by 2 orders of magnitude for all strategies (22 845–30 357) and FP diagnoses increased even more (83 724–112 702) (Table 3). Among PWH, life expectancy was highest for NAT-Resolve (351.59 months), followed by RDT-WHO (349.88 months) and RDT-CI (349.35 months). Among all tested, discounted life expectancy was similar across strategies (228.49–228.52 months) due to low HIV prevalence.

In the high-performing scenario, compared to current Ivoirian testing practices, all testing strategies averted similar numbers of transmissions (53% reduction); in the low-performing scenario, NAT-Resolve averted the most (46% reduction compared to 45% with RDT-WHO and 44% with RDT-CI, Supplementary Table C).

Cost and Cost-Effectiveness Outcomes: High-Performing and Low-Performing Scenarios.

In the high-performing scenario, discounted lifetime costs were similar across all strategies ($222.89–$223.11) (Table 3, Supplementary Table B). Compared to RDT-WHO, RDT-CI had slightly higher life expectancy and costs, leading to an ICER of $3450/YLS. NAT-Resolve had slightly higher life expectancy and costs than RDT-CI, with an ICER of $120 910/YLS.

Compared to the high-performing scenario, in the low-performing scenario, discounted lifetime costs increased due to more FP diagnoses (ie, more people accruing HIV care costs): $271.56 (NAT-Resolve), $278.62 (RDT-WHO), and $283.03 (RDT-CI). NAT-Resolve led to higher life expectancy and lower lifetime costs compared to RDT-WHO and RDT-CI, and thus was cost-saving.

Sensitivity Analyses: Misdiagnoses.

Numbers of FN diagnoses were most impacted by RDT sensitivity (Figure 1A). POC NAT sensitivity was minimally influential in NAT-Resolve due to the small number of PWH who receive discordant RDTs and subsequently receive POC NAT. Numbers of FP diagnoses were most sensitive to RDT specificity (Figure 1B). With the lowest average reported specificity for POC NAT (99.5%), FP diagnoses generated in NAT-Resolve increased from 14 to 452; with the lowest average reported specificity for RDT (82.9%), FP diagnoses generated increased by nearly 4 orders of magnitude (14–28 to 90 000–160 000).

Figure 1.

A, Impact of varying rapid diagnostic test (RDT) and point-of-care nucleic acid test (POC NAT) sensitivity on projected false-negative diagnoses generated by each testing strategy. The thick orange bars represent projected false-negative diagnoses generated by each testing strategy in the high-performing scenario. ^aBest (acute 88.5%; chronic: 100.0%) and worst (acute: 66.0%; chronic: 91.1%) combinations of RDT sensitivity. B, Impact of varying RDT and POC NAT specificity on projected false-positive diagnoses generated by each testing strategy. The thick red bars represent projected false-positive diagnoses in the high-performing scenario. In both panels, the vertical axis lists testing strategies in order of increasing impact of varying RDT and POC NAT sensitivity and specificity, and the horizontal axis shows the number of misdiagnoses per 10 000 000 tested. Longer thin lines indicate the parameters to which the numbers of misdiagnoses were more sensitive. Parameters are varied through plausible ranges, shown in parentheses, to illustrate their individual impact on the number of misdiagnoses generated. Ranges are followed by a semicolon and the high-performing scenario input parameter. Abbreviations: CI, Côte d’Ivoire; POC NAT, point-of-care nucleic acid test; RDT, rapid diagnostic test; WHO, World Health Organization.

Sensitivity Analyses: High-Performing Scenario.

In 1-way sensitivity analyses for the high-performing scenario (Supplementary Table D), the ICER of NAT-Resolve was always above the CI per-capita GDP ($1720/YLS; ie, not cost-effective) except when RDT sensitivity was poor (91.1%; ICER: $1690/YLS), POC NAT cost was ≤0.1 times the base-case values (≤$3; ICER: $1230/YLS), or >35% of PWH were acutely infected (ICER: $1680/YLS). When RDT specificity was ≤93.3%, NAT-Resolve was cost-saving.

Sensitivity Analyses: Clinical and Economic Impact of Misdiagnoses.

With base-case inputs, the life-years lost due to an FN vs TP diagnosis was 4.4 years. Projected life-years lost never dropped below 3.0 life-years but ranged up to 13.0 life-years when varying current Ivoirian HIV detection practices (Figure 2A). The increase in lifetime costs due to an FP vs TN diagnosis was $5800 and was most sensitive to the time spent with an FP diagnosis. One year spent misdiagnosed with HIV corresponded to an increase of HIV-related costs of $590; a lifetime spent misdiagnosed with HIV corresponded to a $14 680 increase (Figure 2B, Supplementary Table E). The time spent with an FP diagnosis was sensitive to loss to follow-up rates (mean range, 6.0–15.4 years; median range, 3.6–14.3 years) (Supplementary Table F). Using combinations of the lowest and greatest impact of FN and FP diagnoses, NAT-Resolve never became cost-effective in the high-performing scenario (Supplementary Table G).

Figure 2.

A, One-way sensitivity analysis of the projected undiscounted life-years lost due to receiving a false-negative human immunodeficiency virus (HIV) diagnosis compared to a true-positive HIV diagnosis. The horizontal axis shows the undiscounted life-years lost and the vertical axis lists key input parameters. The vertical black line represents the mean life-years lost projected in the base case analysis (4.4 years). B, One-way sensitivity analysis on the projected increase in discounted lifetime costs due to receiving a false-positive HIV diagnosis compared to a true-negative HIV diagnosis. The horizontal axis shows the increase in lifetime costs and the vertical axis lists key input parameters. The vertical black line represents the increase in lifetime costs projected in the base-case analysis ($5800). Parameters are varied through wide ranges, shown in parentheses, to illustrate their individual impact on the life-years lost due to a false-negative diagnosis or the increase in costs due to false-positive diagnosis (shown in the blue horizontal bars). Longer blue horizontal bars indicate parameters to which the model results were more sensitive. Ranges are followed by a semicolon and the base-case input parameter. Abbreviations: ART, antiretroviral therapy; FP, false-positive; HIV, human immunodeficiency virus; OI, opportunistic infection; USD, United States dollars.

Additional Scenario Analyses.

In intermediate scenario analyses, with combinations of test characteristics falling between the high-performing and low-performing scenarios, RDT-WHO was the preferred strategy when test characteristics were all <50% worse than the high-performing scenario (Supplementary Table H). NAT-Resolve was cost-saving when test characteristics were all >60% worse than the high-performing scenario. NAT-Resolve was the preferred strategy when pre-ART retesting resolved <70% of FP diagnoses (Supplementary Table I).

In the test-again scenario, which evaluated how ART use would affect high-performing scenario outcomes (reflecting lower RDT sensitivity [95.2%] for PWH taking ART and the inability of POC NAT to detect HIV in the setting of virologic suppression on ART [sensitivity, 0%]), NAT-Resolve generated more FN diagnoses than RDT-WHO or RDT-CI when >0.1% of PWH retested, and always generated the fewest FP diagnoses; RDT-WHO was then the preferred strategy (Supplementary Table J).

DISCUSSION

We simulated a 1-time HIV screening process for adults in Côte d’Ivoire, a setting with a low undiagnosed HIV prevalence, to estimate the clinical and economic impact of HIV misdiagnoses and evaluate the potential cost-effectiveness of employing POC NAT in combination with RDTs compared to existing RDT-based testing strategies. We had 3 key findings.

First, NAT-Resolve was only preferred (cost-saving or cost-effective) in scenarios with poor RDT performance (reflecting programmatic realities in some settings) or low POC NAT cost. We interpret these findings to suggest that in places where RDT performs suboptimally or POC NAT costs are low (eg, POC NAT is already implemented for viral load monitoring or early infant diagnosis), the relatively high individual POC NAT test cost compared to RDT ($27.92 vs $1.47) should not necessarily preclude its use to resolve discordant RDTs [32]. Moreover, setting-specific RDT test characteristics are greatly needed to better understand the need and utility of more robust testing platforms like POC NAT. Importantly, programs have reported that a substantial proportion (up to 39%) of PWH presenting for HIV screening may already be on ART with suppressed HIV RNA, which would lower the sensitivity of POC NAT [10, 33]. This reduction in POC NAT sensitivity is dependent on the rate of retesting among already diagnosed PWH and the proportion virologically suppressed among this population (Supplementary Table J). Further studies are needed to understand the characteristics and motivations of retesting among already diagnosed PWH. Providers that use POC NAT to resolve discordant RDT results should be aware of the risk of misdiagnosis among PWH that do not disclose their HIV status; improved counseling and privacy at testing sites could mitigate this risk.

Second, using POC NAT to resolve discordant RDTs generally led to the fewest misdiagnoses. Small variations in the sensitivity and specificity of diagnostic assays translated to large changes in estimated misdiagnoses when applied to large populations. In the high-performing scenario, the projected misdiagnoses were consistent with the positive and negative predictive values estimated by WHO testing guidelines [2, 3, 34]. RDT sensitivity and specificity were most influential on misdiagnoses generated because RDT is the first test administered in each strategy. In the low-performing scenario, FN and FP diagnoses increased >50-fold and >4000-fold, respectively. Automated POC NAT platforms—which are less reliant on humans than RDTs for performance and interpretation—are anticipated to have reproducible results in field settings [35], which could substantially mitigate poor RDT field performance, as observed in NAT-Resolve [5, 6]. These findings highlight the importance of expanded access to HIV diagnostic testing with reproducible test characteristics that can minimize misdiagnoses [36].

Third, this analysis underscores that early HIV testing markedly improves clinical outcomes. The projected life-years lost due to an FN diagnosis depended on the frequency of current HIV testing. To our knowledge, there are no published estimates of life-years lost due to an FN diagnosis in similar settings. FP diagnoses are costly, increasing HIV-related lifetime costs by $5800. This modeled cost depended on assumptions about the duration of an FP diagnosis (based on Ivoirian loss to follow-up rates) [29], and was consistent with other published estimates in low-income settings ($3770–$4655) [34, 37]. Notably, these estimates do not capture additional individual and societal costs, such as the psychological burden and the erosion of trust in health care systems resulting from misdiagnoses [38].

This model-based analysis had several limitations that may have over- or underestimated the benefits of NAT-Resolve. First, we assume that the results of sequential testing are conditionally independent. In practice, potential cross-reactivity between assays may lead to poorer RDT performance, thereby increasing the value of POC NAT [6]. Second, we assume that WHO-recommended pre-ART retesting is concordant with the final result of the testing strategy. Although pre-ART retesting may serve as a barrier to rapid ART initiation, it could avert FP diagnoses in the low-performing scenario for RDT-based testing strategies [39], thereby decreasing the value of NAT-Resolve. Third, the duration and cost of an FP diagnosis were based on estimated Ivoirian loss to follow-up parameters for PWH. People without HIV who receive an FP diagnosis may engage differently in HIV care than PWH, so these results may vary between settings. However, our findings show that plausible variation in the cost of an FP diagnosis ($590–$14 680) does not affect cost-effectiveness results in the high-performing scenario (Supplementary Table G).

With a simulated 1-time HIV screening process in Côte d’Ivoire, we found that using POC NAT in combination with RDTs led to the fewest misdiagnoses for people with previously undiagnosed HIV. We projected substantial life-years lost and increases in lifetime costs due to misdiagnoses. In settings where RDTs perform poorly and testing strategies are prone to attrition and result delays, the number and impact of misdiagnoses are large. In these settings, POC NAT can be an important adjunct to existing RDT-based testing strategies.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

This study was approved by the Partners Human Research Committee. No patient-level data were included in this modeling study; only published data were included, so no consent was required.

Author contributions. All authors contributed significantly to this work and reviewed and approved this manuscript for publication. A. M. N. and A. L. C. developed the initial conceptualization, design, and analysis plan for this study. A. M. N. and A. R. G. led the execution of this model-based analysis, interpretation of model results, and manuscript writing efforts. J. C., E. S., P. F., and M. N. K. contributed to the initial conceptualization of this analysis, procured data for model inputs, and contributed to manuscript preparation. R. P. W., K. A. F., and A. L. C. provided significant input on the analysis plan, interpretation of model results, and manuscript preparation.

Disclaimer. The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the views of the Elizabeth Glaser Pediatric AIDS Foundation, Unitaid, the National Institutes of Health, the Massachusetts General Hospital (MGH) Executive Committee on Research, the US Centers for Disease Control and Prevention, or the US government.

Financial support. This work was supported by the Elizabeth Glaser Pediatric AIDS Foundation and Unitaid (to A. M. N., A. R. G., and A. L. C.). Additional support for this research included the Eunice Kennedy Shriver National Institute for Child Health and Human Development (grant numbers K08 HD094638 to A. M. N. and R01 HD079214 to A. L. C.); the National Institute of Allergy and Infectious Diseases (grant numbers R37 AI058736 to K. A. F. and R37 AI093269 to R. P. W.); and the Steve and Deborah Gorlin MGH Research Scholars Award (to R. P. W.).

Potential conflicts of interest. All authors: No reported conflicts of interest.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.