CD4 Count Outperforms World Health Organization Clinical Algorithm for Point-of Care HIV Diagnosis among Hospitalized HIV-exposed Malawian Infants

Abstract

Objective

To determine, for the WHO algorithm for point-of-care diagnosis of HIV infection, the agreement levels between pediatricians and non-physician clinicians, and to compare sensitivity and specificity profiles of the WHO algorithm and different CD4 thresholds against HIV PCR testing in hospitalized Malawian infants.

Methods

In 2011, hospitalized HIV-exposed infants <12 months in Lilongwe, Malawi were evaluated independently with the WHO algorithm by both a pediatrician and clinical officer. Blood was collected for CD4 and molecular HIV testing (DNA or RNA PCR). Using molecular testing as the reference, sensitivity, specificity, and positive predictive value (PPV) were determined for the WHO algorithm and CD4 count thresholds of 1500 and 2000 cells/mm³ by pediatricians and clinical officers.

Results

We enrolled 166 infants (50% female, 34% <2 months, 37% HIV-infected). Sensitivity was higher using CD4 thresholds (<1500, 80%; <2000, 95%) than with the algorithm (physicians, 57%; clinical officers, 71%). Specificity was comparable for CD4 thresholds (<1500, 68%, <2000, 50%) and the algorithm (pediatricians, 55%, clinical officers, 50%). The positive predictive values were slightly better using CD4 thresholds (<1500, 59%, <2000, 52%) than the algorithm (pediatricians, 43%, clinical officers 45%) at this prevalence.

Conclusion

Performance by the WHO algorithm and CD4 thresholds resulted in many misclassifications. Point-of-care CD4 thresholds of <1500 cells/mm³ or <2000 cells/mm³ could identify more HIV-infected infants with fewer false positives than the algorithm. However, a point-of-care option with better performance characteristics is needed for accurate, timely HIV diagnosis.

Introduction

Despite great strides in prevention of mother-to-child transmission (PMTCT) of HIV infection in developing countries, approximately 330,000 children acquire HIV each year, more than 90% of these in sub-Saharan Africa¹. More than half of these HIV-infected children will die within two years without antiretroviral therapy (ART).^2,3 Early ART initiation within the first few months of life is even more beneficial, reducing infant mortality by 76% and HIV progression by 75%.⁴

Although early ART initiation hinges on timely infant diagnosis, diagnosing HIV in this age group poses major challenges. First, point-of-care antibody tests cannot definitively diagnose HIV since maternal antibodies circulate in some infants until 18 months of age.⁵ Second, HIV PCR testing using DNA or RNA is definitive but results are frequently delayed due to slow turnaround times at specialized central referral laboratories and loss to follow-up.⁶ Ultimately, up to 70% of HIV-infected children may not receive HIV PCR results.⁷ As a consequence, HIV-related mortality continues among undiagnosed infants in HIV-endemic African countries.

Two potential alternatives to infant HIV PCR testing are the WHO clinical algorithm for symptomatic HIV infection and point-of-care CD4 testing.⁸ The WHO clinical algorithm enables clinicians to immediately initiate potentially life-saving ART for infants meeting its criteria,⁹ circumventing the long delays plaguing current PCR testing systems. Malawi, a southern African country with epidemic HIV, recommended use of this algorithm during routine care, with subsequent HIV DNA PCR confirmation starting in 2008.¹⁰ To date little is known about the sensitivity and specificity profiles of this algorithm when used by non-physician clinicians, as previous work focused on algorithm performance solely by pediatricians with sensitivities of 23–77% and specificities between 53–93%.^11–14 This knowledge gap is critical since non-physician clinicians, not pediatricians, deliver the majority of clinical care in Malawi and other high HIV prevalence African countries.

Point-of-care CD4 cell count testing also offers a potential diagnostic alternative to HIV PCR. CD4 count testing can be made available at the point of care with existing technology. Performance characteristics at different CD4 count thresholds have not been explored. Furthermore, direct comparisons between the performance of different CD4 count thresholds, CD4 percentages, the WHO algorithm, and gold standard PCR testing have not been thoroughly studied. If point-of-care CD4 testing were to outperform the WHO clinical algorithm, it would provide an immediate laboratory-based point-of-care option for infant HIV diagnosis and ART eligibility.

Our primary objectives for this study were 1) to determine WHO algorithm agreement levels between pediatricians and non-physician clinicians, and 2) to compare sensitivity and specificity profiles of the WHO algorithm and different CD4 thresholds against HIV PCR testing in hospitalized Malawian infants. We hypothesized that pediatricians would outperform non-physician clinicians in using the WHO algorithm and that a CD4 threshold which outperforms clinical diagnosis could be identified.

Methods

Study Setting

This study was performed among hospitalized children in the pediatric wards of Kamuzu Central Hospital (KCH) in Lilongwe, Malawi. With pediatrics, KCH serves as both a referral and district hospital, has 215 beds, admits more than 13,000 children annually, and has an inpatient pediatric HIV prevalence of 8.5%.¹⁵ Since 2008, hospitalized children and their caregivers have been routinely offered HIV antibody testing as part of an inpatient pediatric HIV testing program in accordance with Malawi guidelines. ¹⁵ Children are subsequently eligible for DNA PCR if HIV antibody-positive and younger than 12 months. A WHO algorithm evaluation by either a pediatrician or clinical officer (CO) also occurs, but less routinely, since assessment depends upon the practitioner’s training and preference.

Study Procedures

This sub-study took place between February and November 2011 and was nested within a prospective randomized controlled trial. ¹⁶ The parent study’s primary objective was to compare outcomes of standard of care that included a pediatrician examination with the WHO algorithm plus laboratory-based HIV DNA PCR (turnaround time of several weeks) with a rapidly processed HIV RNA PCR test (turnaround time of 48 hours). Study subjects of the parent study were hospitalized at KCH, were consented by a guardian, were less than 12 months of age, and were HIV-exposed without a definitive HIV status. At enrollment blood was collected from all participants for PCR HIV testing (either DNA or RNA PCR per randomization outcome), CD4 absolute cell count and percentages, complete blood count and differential, malaria smear, and blood culture. Chest radiographs and tuberculin skin tests were also done, with induced sputum tests performed only for patients with clinical suspicion of Mycobacterium tuberculosis or Pneumocystis jirovecii infection.

Study infants testing HIV antibody-positive using the standard Malawi HIV testing algorithm were eligible for this sub-study. The algorithm consisted of serial testing with Determine HIV-1/2 (Alere) first, followed by Unigold Recombigen HIV-1/2 (Trinity Biotech) for those testing antibody positive. Both a study pediatrician and one non-physician CO evaluated each sub-study infant, also at the time of enrollment into the parent study, filled out an algorithm checklist for each criterion, and assigned either a positive or negative HIV status per WHO algorithm criteria. For an infant to be considered algorithm-positive they needed either two HIV-related conditions (oral thrush, severe or very severe pneumonia, or severe sepsis) or one AIDS-specific condition (P.jirovecii pneumonia, esophageal candidiasis, treatment-unresponsive severe acute malnutrition, extra-pulmonary tuberculosis disease, Kaposi sarcoma, cerebral toxoplasmosis with onset after one month of age, or cryptococcal meningitis). The pediatrician and COs were blinded to one another’s clinical evaluations and to PCR results. However, they were not blinded to the PMTCT and breast-feeding history of the mother-infant pair.

All COs working in the KCH pediatric wards were invited to participate in the sub-study, provided written informed consent, underwent a half-day training in the WHO algorithm and study procedures, and completed a questionnaire and written competency test. COs were the practitioners of interest since they are the primary cadre of non-physician clinicians in Malawi and provide the majority of Malawian pediatric hospital care.

We retrospectively assessed CD4 performance at multiple percentage and absolute CD4 count thresholds. Infants with values below the CD4 threshold were classified “positive” and those with values above the threshold were classified as “negative.” We assessed CD4 percentages because they are preferred for HIV management in infants, and absolute counts because point-of-care technology is available currently, though not used in this assessment.

Analytic Methods

Normally distributed continuous covariates were described using means and standard deviations and categorical characteristics were presented as proportions. Level of agreement in assignment of overall WHO algorithm status and individual algorithm conditions were compared between COs and the pediatrician using proportion of overall agreement and Cohen’s kappa statistic.

The reference standard used for HIV infection was a positive HIV DNA PCR or RNA PCR with >10,000 copies/ml. The performance of the pediatrician, COs, and CD4 thresholds (both absolute count and percentages) were compared to this standard. We also compared each individual WHO algorithm condition to this standard. Sensitivity and specificity were calculated, along with 95% confidence intervals (CI). Given these sensitivities and specificities, the positive predictive value (PPV), negative predictive value (NPV) and corresponding 95% CIs were calculated at HIV prevalence levels from 0% to 100%, including the prevalence in this population. Additionally, we calculated the total number of errors expected (false positives plus false negatives) in a population of 1000 infants at each CD4 count threshold. We varied two sets of assumptions. First we varied the relative weight of a false negative and false positive result (i.e. that a false positive and a false negative result were equal or that a false negative result would be three times worse than a false positive result). We also varied the prevalence of HIV infection in the population from 5% (the projected prevalence under improved PMTCT policies) and 37% (the prevalence in this sub-study).

All pediatric data were analyzed using SAS 9.3 (SAS Institute, Cary NC). CO characteristics were analyzed in Microsoft Excel. Completeness and accuracy in conduct and reporting of this study was assessed using the STARD initiative checklist. ¹⁷

Ethical Approval

We received ethical approval from the Malawi National Health Sciences Research Committee and the Institutional Review Boards at The University of North Carolina at Chapel Hill and Baylor College of Medicine.

Results

Of all hospitalized infants <12 months 13.1% (323/2465) were HIV-exposed (Figure 1). Research staff enrolled 300 infants into the parent study, of whom 237 (79%) tested HIV antibody-positive and were eligible for this sub-study. Both the study pediatrician and one CO examined 70.0% (166/237) of these HIV antibody-positive infants. COs were not available to examine all 237 eligible infants due to logistical conflicts.

Figure 1. Study Population.

Figure 1 displays the proportion of infants included and excluded from the parent trial and this sub-analysis. The proportions of infants who were HIV-infected and HIV-uninfected were determined with HIV RNA or HIV DNA PCR.

Table 1 presents patient characteristics. Exactly half (83/166) of study subjects were female with a median age of 3.3 months (interquartile range=1.5–7.4 months) irrespective of gender. The majority of infants (84%) were breast feeding, of whom 64% (89/139) were breastfeeding exclusively. Most infants received at least partial PMTCT as 66% of mothers reported antenatal ART prophylaxis and 59% reported that their infant received ART prophylaxis after birth.

Table 1.

Patient Characteristics

	N=166 (100%)
Gender
Male	83 (50%)
Female	83 (50%)
Age
<2 months	57 (34%)
2–6 months	58 (35%)
>6 months	51 (31%)
Maternal PMTCT
Yes	107 (66%)
No	56 (33%)
Infant PMTCT
Yes	98 (59%)
No	68 (41%)
Mother alive
Yes	161 (98%)
No	4 (2%)
Breastfeeding status
Exclusive	89 (54%)
Mixed	50 (30%)
No	27 (16%)

PMTCT indicates prevention of mother-to-child transmission.

Participating COs were generally young (mean age 25.4 years, range 21–32 years) and without substantial clinical or pediatric HIV experience (mean 2.3 years clinical experience; mean 1.6 years pediatric HIV experience [Table 2]). Prior to this study only two of seven COs were both nationally certified in HIV and also trained in pediatric HIV. COs and pediatricians arrived at the same diagnosis on 67% (112/166) of the infants. An overall “fair” agreement level¹⁸ in the assignment of WHO clinical algorithm status was achieved between COs and the study pediatrician (mean kappa 0.35, 95% CI: 0.21, 0.49), ranging from 0.18 to 0.59. We also assessed overall agreement levels for the diagnosis of specific WHO clinical algorithm conditions between participating COs and the study pediatrician using the kappa statistic, where 0 is no agreement, 1 is complete agreement (data not shown), and negative values reflect less agreement than would be expected at random. Except for almost no agreement found for the diagnosis of severe malnutrition (kappa=0.03), kappa agreement levels ranged from 0.28 for sepsis to 0.62 for oral Candida.

Table 2.

Clinical Officer Demographics and Agreement with Pediatrician in allocation of WHO-PD

	CO #1	CO #2	CO #3	CO #4	CO #5	CO #6	CO #7
Male gender	Y	Y	Y	Y	Y	Y	Y
Age	26	25	32	21	23	23	28
Years of Pediatric HIV Experience	1	3	3	0	1	1	2
Years of Clinical Experience	1	3	4	1	3	1	3
National IMCI certification	N	N	Y	N	Y	N	Y
National HIV certification	N	Y	Y	N	Y	N	N
Pediatric HIV trained	N	Y	Y	Y	Y	N	N
Children evaluated with WHO algorithm	29	4	7	38	13	52	23
Observed agreement	79%	75%	71%	71%	62%	56%	74%
Kappa statistic	0.59 (0.31–0.87)	0.50 (−0.24–1.00)	0.30 (−0.47–1.00)	0.42 (0.14–0.71)	0.25 (−0.21–0.72)	0.18 (−0.04–0.40)	0.41 (0.04–0.77)

HIV indicates human immunodeficiency virus; IMCI, Integrated Management of Childhood Illnesses; WHO, World Health Organization.

Table 2 displays the characteristics of clinical officers as well as their level of agreement with the pediatrician on each diagnosis. Observed agreement is the proportion of cases in which the clinical officer and pediatrician indicated the same response on the algorithm.

The kappa statistic reflects observed agreement relative to expected agreement.

We next examined the sensitivity of the WHO algorithm when applied by clinicians and CD4 thresholds as compared to gold standard DNA or RNA molecular testing (Table 3). Overall, 61/166 infants (37%) were HIV-infected. The sensitivity of the WHO algorithm was lower when used by pediatricians (57%) versus COs (71%). However, 5/43 cases classified as algorithm-positive by COs did not actually meet criteria according to their own evaluation. For example, some checked “HIV-positive” even if the infant only had one HIV-related condition. If COs had correctly interpreted the WHO algorithm for all patients, the sensitivity of the algorithm would have decreased to 62%. For both pediatricians and COs, the most sensitive clinical condition was severe or very severe pneumonia (71% for pediatricians and 57% for COs). Sensitivity increased with both CD4 percentages (28% sensitive at CD4 <15% to 85% sensitive at CD4<30%) and CD4 count (25% sensitive at CD4 <500 cells/mm³ to 95% sensitive at CD4 <2000 cells/mm³).

Table 3.

Performance of WHO Clinical Criteria for Symptomatic HIV Infection in children <12 months old

	True +	False −	True −	False +	Sensitivity (95% CI)	Specificity (95% CI)
PCR as the gold standard
Pediatrician
WHO algorithm-positive	35	26	58	47	57.4 (44.1, 70.0)	55.2 (45.2, 65.0)
Sepsis	5	56	94	11	8.2 (2.7, 18.1)	89.5 (82.0, 94.7)
Severe or very severe pneumonia	43	18	30	75	70.5 (57.4, 81.5)	28.6 (20.2, 38.2)
Oral candida	21	40	95	10	34.4 (22.7, 47.7)	90.5 (83.2, 95.3)
Any two HIV conditions	15	46	89	16	24.6 (14.5, 37.3)	84.8 (76.4, 91.0)
Pneumocystis jirovecci pneumonia	22	39	70	35	36.1 (24.2, 49.4)	66.7 (56.8, 75.6)
Esophageal candida	8	53	103	2	13.1 (5.8, 24.2)	98.1 (93.3, 99.8)
Severe malnutrition	2	59	103	2	3.3 (0.4, 11.4)	98.1 (93.3, 99.8)
Clinical Officer
WHO algorithm- positive	43	18	52	53	70.5 (57.4, 81.5)	49.5 (39.6, 59.5)
WHO PD by algorithm conditions	38	23	68	37	62.3 (49.0, 74.4)	64.8 (54.8, 73.8)
Sepsis	20	41	78	27	32.8 (21.3, 46.0)	74.3 (64.8, 82.3)
Severe or very severe Pneumonia	35	26	36	69	57.4 (44.1, 70.0)	34.3 (25.3, 44.2)
Oral candida	21	40	99	6	34.4 (22.7, 47.7)	94.3 (88.0, 97.9)
Any two HIV conditions	19	42	88	17	31.2 (19.9, 44.3)	83.8 (75.4, 90.3)
Pneumocystis jirovecci pneumonia	18	43	81	24	29.5 (18.5, 42.6)	77.1 (67.9, 84.8)
Esophageal candida	5	56	105	0	8.2 (2.7, 18.1)	100 (96.6, 100.0)
Severe malnutrition	5	56	103	2	8.2 (2.7, 18.1)	98.1 (93.3, 99.8)
CD4 percentage
<15%	17	44	104	1	27.9 (17.2, 40.8)	99.0 (94.8,100.0)
<20%	40	21	98	7	65.6 (52.3, 77.3)	93.3 (86.8, 97.3)
<25%	46	15	88	17	75.4 (62.7, 85.5)	83.8 (75.4, 90.3)
<30%	52	9	77	28	85.2 (73.8, 93.0)	73.3 (63.8, 81.5)
CD4 absolute count
<500	15	46	103	2	24.6 (14.5, 37.3)	98.1 (93.3, 99.8)
<1000	37	24	92	13	60.7 (47.3, 72.9)	87.6 (79.8, 93.2)
<1500	49	12	71	34	80.3 (68.2, 89.4)	67.6 (57.8, 76.4)
<2000	58	3	52	53	95.1 (86.3, 99.0)	49.5 (39.6, 59.5)

HIV, human immunodeficiency virus; IMCI, Integrated Management of Childhood Illnesses; WHO, World Health Organization

Table 3 displays the number of individuals with true positive, false negative, true negative, and false positive results, as well as sensitivity, specificity, and corresponding 95% confidence intervals. WHO clinical criteria for symptomatic HIV infection in infants include HIV antibody positive, and ≥2 severe HIV-associated clinical conditions or ≥1 AIDS-specific condition. HIV-associated conditions included sepsis, severe or very severe pneumonia, and oral candida. Cryptococcol meningitis, KS, TB, and other stage 4 conditions were not identified in any children by a pediatrician during this study and are therefore excluded from the table, even though they are AIDS-defining conditions.

In addition to studying the sensitivity of the WHO algorithm and CD4 thresholds, we also studied specificity. In all, 105/166 infants (63%) were HIV-uninfected. Pediatricians achieved a higher specificity (55%) with use of the WHO algorithm as compared to COs (50%). When algorithm performance was corrected for 16 patients misclassified by COs as algorithm-positive, despite their own evaluation indicating otherwise, the specificity of the algorithm increased from 50% to 65%. Compared to HIV PCR testing, the specificity decreased with higher CD4 percentage and absolute count thresholds. Specifically, specificity was 99% at a CD4 percentage <15%, and 73% at CD4<30%. Specificity was 98% at an absolute CD4 count <500 cells/mm³ and 50% at a CD4 <2000 cells/mm³.

At the 37% prevalence, the PPV for CD4 percentage thresholds ranged from 94% at CD4 <15%, to 65% at CD4 <30% (Figure 2C). For absolute CD4 count testing, the PPV ranged from 88% at CD4 <500 cells/mm³ to 52% at CD4 <2000 cells/mm³ (Figure 2B). NPV ranged from 72% at CD4 <15%, to 90% at CD4 <30% and 69% at CD4 count <500 cells/mm³ to 95% at CD4 <2000 cells/mm³.

Figure 2.

Figure 2A. Positive and Negative Predictive Values of the WHO Algorithm as a Function of HIV Prevalence

Figure 2B. Positive and Negative Predictive Values of CD4 Count Thresholds as a Function of HIV Prevalence

Figure 2C. Positive and Negative Predictive Values of CD4 Percentage Thresholds as a Function of HIV Prevalence

Several trends emerged when assessing PPV and NPV using the WHO algorithm, and all CD4 thresholds over a spectrum of prevalence levels (Figure 2). First, at any prevalence, both PPV and NPV of CD4 <1000, CD4 <1500, and CD4 <2000 were better than PPV and NPV among either of the clinician cadres. At the study prevalence (37%) PPV was best (88%) at CD4 <500, and at this value CD4 NPV was almost identical to that of both clinician cadres. CD4 <2000 was the worst CD4 count threshold for PPV (52%), but still better than use of the WHO algorithm by either type of clinician.

Because CD4 count thresholds outperformed the WHO algorithm and could become available at the point of care, we explored which thresholds would minimize the number of errors (Table 3). At 37% prevalence, when false negatives and false positives were weighted equally, the total number of errors increased monotonically from 29 per 100 infants screened at a CD4 threshold of <500 cells/mm³ to 34 per 100 infants screened at a threshold of <2000 cells/mm³. When a false negative was weighted three times more heavily, the opposite trend emerged: the total number of errors decreased monotonically from 85 per 100 infants screened at <500 cells/mm³ to 37 per 100 infants screened at <2000 cells/mm³. At a 10% prevalence when false negatives and false positives were weighted equally, the total number of errors increased monotonically from 9 per 100 infants screened at <500 cells/mm³ to 46 per 100 infants screened <2000 cells/mm³. But when a false negative was weighted three times more heavily, the relationship between CD4 count threshold and number of errors per 100 infants screened was not monotonic: 24 errors at CD4 <500 cells/mm³, 23 errors at CD4 <1000 cells/mm³, 35 errors at CD4 <1500 cells/mm³, and 47 errors at <2000 cells/mm³(Table 4). The choice of the optimal CD4 count threshold is quite sensitive to both the relative weight of false negative and false positive result and the prevalence.

Table 4.

Total expected errors at different prevalence levels, CD4 count thresholds, and false negative weights assuming a population of 1000 infants

Prevalence	CD4 Threshold	HIV Positive	HIV Negative	Sensitivity	Specificity	FP	FN	FN x 3 (weighted)	Total Errors	Total Errors (x3)
0.37	<500	370	630	0.25	0.98	12	279	837	291	849
0.37	<1000	370	630	0.61	0.88	78	145	436	224	514
0.37	<1500	370	630	0.80	0.68	204	73	219	277	423
0.37	<2000	370	630	0.95	0.50	318	18	54	336	373
0.2	<500	200	800	0.25	0.98	15	151	452	166	468
0.2	<1000	200	800	0.61	0.88	99	79	236	178	335
0.2	<1500	200	800	0.80	0.68	259	39	118	299	377
0.2	<2000	200	800	0.95	0.50	404	10	29	414	433
0.1	<500	100	900	0.25	0.98	17	75	226	93	243
0.1	<1000	100	900	0.61	0.88	112	39	118	151	230
0.1	<1500	100	900	0.80	0.68	292	20	59	311	351
0.1	<2000	100	900	0.95	0.50	455	5	15	459	469
0.05	<500	50	950	0.25	0.98	18	38	113	56	131
0.05	<1000	50	950	0.61	0.88	118	20	59	137	177
0.05	<1500	50	950	0.80	0.68	308	10	30	318	337
0.05	<2000	50	950	0.95	0.50	480	2	7	482	487

HIV, human immunodeficiency virus; FP, false positive; FN, false negative

Table 4 displays total number of expected errors (FP plus FN) in a hypothetical population of 1000 infants at different prevalence levels of HIV and different CD4 thresholds. It further displays number of expected error with false negatives (FN) weighed three times more heavily than false positives (FP)

Discussion

Unacceptably high rates of mother-to-child HIV transmission and pediatric mortality continue in part because a reliable point-of-care HIV diagnostic test for infants is not yet routinely available in resource-constrained African countries with epidemic HIV. This study examined two types of existing point-of-care diagnostic approaches that could be utilized for infant HIV diagnosis until an acceptable molecular test is available, the WHO clinical algorithm for symptomatic HIV infection in infants and the CD4 test. Overall we found the accuracy of infant HIV diagnosis using the WHO clinical algorithm or CD4 to be inferior to gold standard molecular HIV testing (DNA or RNA PCR) in hospitalized Malawian infants. Both COs and pediatricians missed a large share of HIV-infected infants using the WHO algorithm. Higher CD4 count thresholds (>1500 cells/mm³ or >25%), though also sub-optimal, performed markedly better at identifying HIV-infected infants in our patient population.

This study addressed an important knowledge gap regarding the use of the WHO algorithm by COs, a cadre of non-physician clinicians that provide the majority of pediatric HIV care in Malawi and are common throughout sub-Saharan Africa.¹⁹ The performance of the WHO algorithm by the pediatrician in this study was comparable to that observed in WHO algorithm validation studies done in Kenya¹² and Zambia¹⁴ (low sensitivity) and Rwanda¹¹ (low specificity). Our study found that there was only fair agreement with algorithm use between COs and pediatricians (Kappa = 0.35) on an individual patient basis. However, COs and pediatricians had similar performance overall with respect to the sensitivity and specificity of the algorithm when compared to molecular HIV testing. COs displayed a trend towards better sensitivity and pediatricians displayed a trend towards better specificity, though these differences were not statistically significant at an alpha level of 0.05. In summary, using the WHO algorithm COs were able to assess infant HIV status as well as pediatricians in this setting.

Although the two cadres performed similarly to each other, the WHO algorithm is poorly sensitive and specific for detecting infant HIV in this setting. Using the WHO algorithm at least 30% of HIV-infected infants were not identified by either clinician cadre and the majority of those identified as HIV-infected by both cadres were actually HIV-uninfected. Similar performance of the WHO algorithm was also observed in other studies assessing this algorithm.^11–14

Since point-of-care molecular testing is not yet available, this study explored the performance of different CD4 thresholds in an effort to identify an existing alternative to the WHO algorithm. CD4 count testing could be made available at the point of care. In some settings CD4 count is comparable to CD4 percentages, the preferred metric in infants.^20–22 We have shown that using CD4 count thresholds can improve sensitivity over clinical diagnosis without compromising PPV at any given prevalence. Previous studies have assessed the performance of the CD4 <25% threshold in similar infant populations and found sensitivities between 55%¹¹ and 72%¹² and specificities from 39%¹² to 88%¹³. Other studies have assessed clinical criteria performance in combination with CD4<25% and demonstrated improved algorithm performance^11–14. Choosing an optimal CD4 count threshold is challenging and requires careful consideration. Initiating HIV-uninfected infants on ART is the compromise of using a less specific HIV diagnostic test and should be done with caution. Falsely identifying infants as HIV-infected may result in medication side effects, misdirected use of scarce health care resources, and psychosocial strain on the family. To avoid these potential issues, a highly specific diagnostic is desirable. On the other hand, maximizing sensitivity is likely to minimize mortality because more HIV-infected infants can be initiated on ART in a timely fashion. Thus we consider weighing a false negative more heavily than a false positive to be reasonable. A CD4 count of 2000 cells/mm³ identified nearly all the HIV-infected infants (95%). At 37% prevalence, assuming a false negative is three times worse than a false positive, a CD4 count threshold of 2000 cells/mm³ optimizes the potential diagnostic algorithm.

But the optimal CD4 threshold will vary depending on the prevalence of HIV among infants, and should therefore be individually tailored to each setting’s HIV prevalence prior to implementation and reviewed over time as HIV prevalence changes. For example, in Malawi HIV prevalence is likely to decline over time due to Option B+, a PMTCT program started in 2011 that provides free lifetime ART to all HIV-infected pregnant and breastfeeding women.²³ In our setting, if prevalence among hospitalized infants declines to 10%, the optimal CD4 count threshold would be <1000 cells/mm³. Since our study relied on data preceding Option B+ implementation, additional modeling or a pilot study assessing different CD4 thresholds at the point-of-care for HIV-exposed Malawian infants in this new context could be informative.

There are several reasons why the WHO algorithm may have underperformed compared to molecular testing. First, the individual conditions used in the algorithm themselves may not be good predictors of HIV in this population. For example, sepsis and pneumonia are prevalent among both HIV-infected and HIV-uninfected infants in this population (Table 3), and thus poorly specific for predicting HIV. Second, individual algorithm conditions may have been inaccurately classified, thereby limiting accuracy of clinicians who adhere to the algorithm. Third, COs misclassified some infants by not adhering to the algorithm. Previous evidence has reported poor adherence to clinical criteria for both pneumonia diagnosis and oxygen eligibility by Malawian COs.¹⁹ In this study we similarly report inconsistent CO adherence to the WHO algorithm for infant HIV diagnosis.

The results observed in this hospital-based study may not be generalizable to outpatient Malawian clinics due to several factors. Hospitalized infants are generally sicker than HIV exposed-infants presenting to outpatient care and HIV prevalence is also higher in the hospital compared to the community. Therefore, fewer infants presenting for routine outpatient care are likely to meet algorithm criteria for an HIV-positive diagnosis. Only one pediatrician and seven COs participated in this single facility study. A larger multicenter study in diverse settings with more practitioners is needed to confirm our findings.

In conclusion, accurate, timely HIV infant diagnosis is critical for access to life-saving ART but is not occurring in the current Malawian infant HIV diagnosis system that relies on slowly processed DNA PCR tests. Ideally, an infant HIV point-of-care diagnostic that is highly sensitive, specific, and affordable is desired. Until such a test is available, we recommend implementing point-of-care CD4 testing as an infant diagnostic, with thresholds based on local HIV prevalence levels.