Summary
Background
The HIV Infant Tracking System (HITSystem) is a web-based intervention linking providers of early infant diagnosis, laboratory technicians, and mothers and infants to improve outcomes for HIV-exposed infants. We aimed to evaluate the efficacy of the HITSystem on key outcomes of early infant diagnosis.
Methods
We did a cluster-randomised trial at six hospitals in Kenya, which were matched on geographic region, resource level, and volume of patients (high, medium, and low). We randomly allocated hospitals within a matched pair to either the HITSystem (intervention; n=3) or standard of care (control; n=3). A random number generator was used to assign clusters. Investigators were unaware of the randomisation process. Eligible participants were mothers aged 18 years or older with an infant younger than 24 weeks presenting for their first early infant diagnosis appointment. The primary outcome was complete early infant diagnosis retention, which was defined as receipt of all indicated age-specific interventions until 18 months post partum (for HIV-negative infants) or antiretroviral therapy initiation (for HIV-positive infants). Analysis was per protocol in all randomised pairs judged eligible, excluding infant deaths and those who moved or were transferred to another health facility. Modified intention-to-treat sensitivity analyses judged all infant deaths and transfers as incomplete early infant diagnosis retention. Separate multivariable logistic regression analyses were done with intervention group, hospital volume, and significant covariates as fixed effects. This trial is registered with ClinicalTrials.gov, number NCT02072603; the trial has been completed.
Findings
Between Feb 16, 2014, and Dec 31, 2015, 895 mother–infant pairs were enrolled. Of these, 87 were judged ineligible for analysis, 26 infants died, and 92 pairs moved or were transferred to another health facility. Thus, data from 690 mother–infant pairs were analysed, of whom 392 were allocated to the HITSystem and 298 to standard of care. Mother–infant pairs were followed up to Sept 30, 2017. Infants diagnosed as HIV-positive were followed up for a median of 2·1 months (IQR 1·6–4·8) and HIV-negative infants were followed up for a median of 17·0 months (IQR 16·6–17·6). Infants enrolled in the HITSystem were significantly more likely to receive complete early infant diagnosis services compared with those assigned standard of care (334 of 392 [85%] vs 180 of 298 [60%]; adjusted odds ratio [OR] 3·7, 95% CI 2·5–5·5; p<0·0001). No intervention effect was recorded at high-volume hospitals, but strong effects were seen at medium-volume and low-volume hospitals. Modified intention-to-treat analyses for complete early infant diagnosis were also significant (334 of 474 [70%] vs 180 of 334 [54%]; adjusted OR 2·0, 95% CI 1·4–2·7; p<0·0001). No adverse events related to study participation were reported.
Interpretation
The HITSystem intervention is effective and feasible to implement in low-resource settings. The HITSystem algorithms have been modified to include HIV testing at birth, and an adapted HITSystem 2.0 version is supporting HIV-positive pregnant women to prevent perinatal transmission and optimise maternal and infant outcomes.
Introduction
For HIV-exposed infants aged from 6 weeks up to 18 months, early infant diagnosis is a critical and time-sensitive service to identify HIV-positive infants for rapid initiation of lifesaving antiretroviral therapy (ART). In Kenya, the early infant diagnosis cascade of care includes a series of age-specific interventions to clinically manage, test, and retest infants until 18 months of age and initiate ART rapidly when indicated. These interventions include collection of a dried bloodspot sample for HIV DNA PCR testing at 6 weeks of age, sample shipment to and processing at a central laboratory, return of the paper-based PCR result back to the health facility, notifying the mother of the infant’s PCR result, and either ART initiation for HIV-positive infants or antibody retests at 9 months and 18 months for HIV-negative infants.1 In 2012, early infant diagnosis reached approximately 39% of HIV-exposed infants in Kenya,2 with only 38% of HIV-positive infants and children identified through these services initiating ART.3 Data from 2016 show improvements, with 53% of HIV-exposed infants receiving early infant diagnosis and 65% of HIV-positive infants and children identified initiating ART.4 Nevertheless, early infant diagnosis services and outcomes still fall well short of evidence-based targets to optimise infant survival by testing all HIV-exposed infants by 6 weeks of age and initiating ART for HIV-positive infants by 12 weeks of age.5 The quality and efficiency of early infant diagnosis services in Kenya and other low-resource settings remain hampered by system and structural barriers, including late presentation for care,6 long turnaround times for PCR results to be returned to the hospital then communicated to mothers,7 suboptimum retention of infants up to 18 months of age,7 and delayed ART initiation for HIV-positive infants.8
In 2011, Global Health Innovations—a non-governmental organisation based in the USA and Kenya—developed the HIV Infant Tracking System (HITSystem) intervention in partnership with the digital marketing firm OnTarget Interactive (Kansas City, MO, USA), to address the limitations of early infant diagnosis services in low-resource settings. The HITSystem uses available technology (internet and short message service [SMS] texting) to improve communication and accountability between early infant diagnosis stakeholders (providers, laboratories, and parents), to optimise outcomes for HIV-exposed infants. The HITSystem aims to expedite PCR test results, facilitate rapid initiation of ART, and increase retention. Infants are tracked until either they are ascertained conclusively to be HIV-negative at age 18 months and discharged from early infant diagnosis or they are established to be HIV-positive and initiated on ART.9,10
At the time we had the idea for our study in 2012, the rate of mobile phone use in Kenya was estimated at 75%,11 and evidence for mobile health (mHealth) interventions for HIV was promising but scant. Findings of two Kenyan randomised controlled trials showed significantly improved HIV treatment outcomes with weekly text messages to adult patients.12,13 The first efforts to apply mHealth strategies to improve early infant diagnosis focused solely on reducing turnaround time for infant PCR results,14 and electronic product design printers (typically referred to as SMS printers) providing one-way communication (from laboratory to health facility) had been deployed in African countries.15,16 Nevertheless, no data from randomised controlled trials assessing mHealth interventions had been published on early infant diagnosis outcomes.
We designed a cluster-randomised trial to assess the efficacy of the HITSystem to improve key measures of early infant diagnosis quality and efficiency in Kenya. The HITSystem targets multiple clinical, laboratory, and participant engagement outcomes to assess comprehensively complete early infant diagnosis—ie, completion of all eligible services throughout the early infant diagnosis cascade of services. We also aimed to assess the turnaround time of multiple time-sensitive early infant diagnosis services.
Methods
Study design and participants
We did a parallel group, cluster-randomised study at six government hospitals in Kenya. We matched hospitals on geographic region, resource level, and volume of patients to produce three matched pairs. Matched hospitals met the following criteria: government funded; provided prevention of mother-to-child transmission (PMTCT), early infant diagnosis, and ART services; had at least one dedicated early infant diagnosis provider; and maintained medical files for HIV-exposed infants. Risk of contamination between intervention and control hospitals was small owing to the geographic distance between sites and because a login and password was required to access the HITSystem. Intervention sites were provided with a wireless modem, so wired internet connectivity was not a requirement. The appendix (p 1) provides additional details about study sites.
We judged mothers eligible to enrol in our trial if they met the following criteria: they were HIV-positive; aged 18 years or older; had an HIV-exposed infant presenting for their first early infant diagnosis appointment through the maternal and child health department; and presented for early infant diagnosis care when their infant was younger than 24 weeks. Mothers enrolled in early infant diagnosis services at intervention sites who did not meet our eligbility criteria were still offered the HITSystem intervention, but their data were excluded from analyses.
We established a data safety monitoring board (DSMB) at the beginning of the study, which consisted of three people with expertise in epidemiology and paediatric HIV, to ensure safety of participants and to undertake a prespecified interim analysis planned at the midpoint of data collection. Study protocol and informed consent forms (available in English and Swahili) were approved by ethics review committees at the Kenya Medical Research Institute (KEMRI) and the University of Kansas Medical Center. The study protocol has been published10 and is available online.
Randomisation and masking
All individuals enrolled in the trial at each hospital comprised a cluster for randomisation. Hospitals within a matched pair were assigned to either control (standard of care) or intervention (HITSystem) using a random number generator. The study statistician was unaware of the identity of study sites. Similarly, all research staff were unaware of the randomisation process.
Procedures
At both intervention and control sites, eligible mothers were informed about the trial by trained research or clinical staff; those choosing to participate provided written informed consent before enrolment. Mothers completed a baseline survey at the time of early infant diagnosis enrolment, which included information on demographics, HIV disclosure status, and early infant diagnosis knowledge. The baseline survey was administered by trained research staff (intervention sites) or trained clinical staff (control sites). Study staff entered survey data into a protected Microsoft Excel 2016 (version 1708) spreadsheet, and we merged these data with each infant’s clinical data for final analyses.
At intervention sites, we captured clinical data in the HITSystem and stored them securely on Ubuntu LAMP (Linux, Apache, MySQL, and PHP) servers, which we backed up daily to preserve the integrity of entered data. When an eligible mother–infant pair first presented for early infant diagnosis care, we created a new infant record in the HITSystem and entered data for the mother’s antenatal care, post-partum infant prophylaxis, and the infant’s date of birth. We also added the mother’s mobile phone number to the HITSystem to enable automated messages when her infant required follow-up services. The HITSystem identifies receipt of text messages and absorbs all SMS costs. Messages do not relay sensitive information that could compromise confidentiality but use the field-tested message in Swahili, “Tafadhali mlete mtoto kwenye kliniki” (the English translation is “please bring baby to the clinic”). If a mother did not have a mobile phone, we obtained physical tracing information. At completion of each subsequent early infant diagnosis service, we accessed the infant’s HITSystem record using a unique numeric identifier and updated the record. We exported HITSystem data directly to an Excel spreadsheet. In response to calls for improved reporting of implementation strategies for interventions,17 the appendix (p 2) specifies the key actors, actions, timing, dose, and targeted behavioural or clinical outcome of each intervention component.
At control sites, participants received services comprising the standard of care—ie, paper-based tracking of HIV-exposed infants with no enhanced or automated follow-up. Providers recorded clinical data in the HIV-exposed infant register, which is the standard paper-based register that clinicians use to record all early infant diagnosis services. A study staff member populated an Excel spreadsheet with datapoints captured in the HIV-exposed infant register to assess early infant diagnosis outcomes at control sites. Study coordinators visited control sites once every 2 weeks and reviewed the HIV-exposed infant register for services received by mother–infant pairs, and they updated the spreadsheet accordingly. We obtained all data from control sites using a unique numeric identifier for each mother–infant pair.
At both intervention and control sites, study personnel did quarterly reviews of additional documentation, including participants’ files, ART-specific records for HIV-positive infants, and records from the hospital’s internal laboratory to fill in data missing or recorded incorrectly in the HIV-exposed infant register or the HITSystem. Participants in both intervention and control clusters were tracked from time of enrolment to: HIV-positive diagnosis and ART initiation; completion of 18-month services (we followed up infants to age 21 months to allow for complete reporting of 18-month services for infants who were brought to hospital late for early infant diagnosis); or discharge from the HITSystem (infant death or documented transfer to another health facility).
Outcomes
The primary outcome was complete early infant diagnosis retention, defined as completion of all eligible services, which comprised: initiation of co-trimoxazole (sulfamethoxazole and trimethoprim) for opportunistic infection prophylaxis; PCR testing; return of the PCR result to the hospital; notifying the mother of the PCR result; initiation of ART (HIV-positive infants only); and retesting at 9 months and 18 months (HIV-negative infants only). We scored this binary measure of complete early infant diagnosis as 1 if all indicated services were received or 0 if one or more service was missed. Consistent with the emerging standard for retention analyses in PMTCT, early infant diagnosis, and paediatric ART,7,18,19 we accounted for deaths and recorded clinic transfers and excluded these from analyses.
Secondary outcomes included turnaround time for key time-sensitive steps in the early infant diagnosis cascade of care. We defined turnaround time for PCR test results as the number of days between sample collection and receipt of the PCR result at the hospital. We calculated turnaround time for notifying the mother of the PCR result as the number of days between receipt of infant PCR results at the hospital and the mother being notified of the PCR results. We measured the turnaround time for ART initiation among HIV-positive infants as the number of days between the mother being notified of the PCR result and ART initiation. We ascertained age at ART by the number of weeks between the infant’s date of birth and the date the infant initiated ART. We established the age at repeated testing—targeted at 9 months and 18 months—by the number of months between the infant’s date of birth and the date of each retest.
Additional secondary outcomes in the study protocol included the provider’s perceptions of HITSystem usefulness and ease of use, the mothers’ experiences with the HITSystem, and cost-effectiveness analyses. These outcomes will be reported separately.
Statistical analysis
We based sample size calculations on unadjusted odds ratios (ORs) obtained in our pilot work, comparing historical control data with intervention data for early infant diagnosis outcomes, ranging from PCR testing by 6 weeks of age (47·4% vs 68·2%; OR 2·4 [effect size 0·42]) to retention at 9-month retesting (43·2% vs 94·1%; OR 21·0 [effect size 1·22]).9 We chose a fairly conservative effect size estimate within this range (0·44) to guarantee adequate power to assess the primary outcome of complete early infant diagnosis at the longer retention period of 18 months post partum. Using Murray’s methods to conduct power for randomised trials,20 we accounted for the number of groups, participants in each group, one-tailed test of significance, and intraclass correlation coefficient (ICC) ranging between 0·01 and 0·05. Using an ICC estimate of 0·05, we calculated that a sample of 720 infants (360 each from intervention and control sites [120 per hospital]) was sufficient to detect with 80% power an effect size of 0·44. Because of multiple planned comparisons along the early infant diagnosis cascade, we used a more conservative α of 0·01 to adjust for possible inflated type I error. After data analyses were done, we became aware of an error in the initial sample size calculation, resulting in a study design with half the number of clusters necessary for detecting small effects. Ultimately, we recorded differences between groups that were much larger than the conservative effect size estimates and variables used to calculate power. Thus, despite being underpowered, significant differences between intervention and control sites were noted.
To maintain consistency in the period being compared, each pair of matched hospitals continued enrolment until the target at both sites was met. Thus, some sites enrolled more than the targeted 120 participants. Among the matched high-volume sites, enrolment of infants for early infant diagnosis was significantly greater at the intervention site. Slower than expected enrolment among matched low-volume sites lengthened the enrolment period for this cluster.
We analysed the primary outcome of complete early infant diagnosis retention per protocol, excluding infant deaths and those who transferred to another health facility from both the intervention and control clusters. We identified participants’ characteristics associated with PMTCT and early infant diagnosis retention using χ2 tests and independent sample t tests to ascertain significant baseline differences between groups. We compared the proportion of infants receiving each intervention across the early infant diagnosis cascade of care, and we treated the aggregated primary outcome of complete early infant diagnosis retention as a response variable in a multivariable logistic regression model. Similarly, we analysed turnaround times in separate Poisson regression models adjusted for overdispersion to produce a robust model fit. For all regression models, we included the intervention as the main effect, and site-level variables—eg, volume of patients—were included as fixed effects. In the models, we controlled for theoretically supported individual-level covariates that have been associated with the outcome in previous research and differed significantly between intervention and control sites—ie, maternal age, level of education, and HIV disclosure status. We present the primary outcome as the proportion of participants with complete early infant diagnosis services, by group, with an adjusted OR, 95% CI, and p value. Because of the small number of HIV-positive infants and skewed distribution of ART-related data, we did non-parametric Mann-Whitney U tests to compare median values specific to HIV-positive infants (ART initiation turnaround time and age at ART initiation) between study groups. We also calculated mean infant ages at the targeted 9-month and 18-month retesting timepoints and compared these by group using an independent sample t test. In effect-modification analyses, we assessed whether the intervention effect differed across type of hospital by including an interaction term between the intervention and hospital volume (high, medium, and low). We report adjusted ORs (with 95% CIs) and p values for the interaction test. Finally, we did a modified intention-to-treat sensitivity analysis, which included all randomised infants with at least 18 months’ post-partum follow-up. This intention-to-treat analysis counted all infant deaths and transfers or moves to a new health facility as incomplete early infant diagnosis, thus establishing the most rigorous standard to measure an intervention effect. We judged significance at an α of 0·05. We did all analyses with SAS version 9.4.
The DSMB did one, prespecified, interim analysis in June, 2016, and ascertained that there were significant findings indicating clinically meaningful advantages for participants receiving the intervention. The DSMB recommended stopping enrolment and proceeding with study analyses once all enrolled infants had been followed up for 21 months (18 months post partum plus an additional 3 months to capture any late testing), to ensure that receipt of all services that make up the primary outcome of complete early infant diagnosis could be recorded. This curtailment meant that the low-volume sites, which accrued participants at a slower pace than high-volume sites, did not enrol their original target samples. In July, 2016, Kenyan early infant diagnosis testing guidelines were changed unexpectedly, removing 9-month HIV antibody testing and adding PCR tests at birth, 6 months, and 12 months.21 We monitored all study sites closely for evidence of adoption of these new testing guidelines, with two study sites showing implemention in June, 2017. This change in practice meant that infants born after Dec 31, 2015, at these two sites would not have the same testing timepoints as the rest of the participants in the cluster, thus precluding a consistent definition of the main outcome of complete early infant diagnosis. As such, we excluded all infants enrolled after Dec 31, 2015, from the final analysis set. We analysed the final dataset after Sept 30, 2017.
This trial is registered with ClinicalTrials.gov, number NCT02072603.
Results
Between Feb 12, 2014, and Dec 31, 2016, we screened 1019 mother–infant pairs for inclusion in our trial, of whom 895 were enrolled and randomised. 87 pairs were judged ineligible for analysis and were excluded (40 at intervention sites and 47 at control sites), of whom five were mothers younger than 18 years, 20 were infants who had their first dried bloodspot test at more than 24 weeks, and 62 were infants born after Dec 31, 2015. Another 26 infants died (17 at intervention sites and nine at control sites) and 92 pairs were transferred or moved to another health facility (65 from intervention sites and 27 at control sites), including three mothers from intervention sites who refused follow-up care for their infants. Therefore, 690 mother–infant pairs were included in analyses, 392 at intervention sites and 298 at control sites (figure 1). No significant differences between intervention and control sites were noted in key participants’ characteristics (ie, maternal age, education level, and HIV disclosure status) among mothers or infants deemed ineligible or pairs who were transferred or moved to another health facility. Only 15 (1%) mothers declined to participate in the trial, and no one withdrew from the study. Median infant age at enrolment was 6·4 weeks (IQR 6·1–7·3) and 344 (50%) of 690 infants were male (table 1).
Table 1:
Intervention (n=392) | Control (n=298) | |
---|---|---|
Study sites | ||
High volume | 226 (58%) | 115 (39%) |
Medium volume | 109 (28%) | 99 (33%) |
Low volume | 57 (15%) | 84 (28%) |
Maternal age (years) | 30·2 (6·0) | 29·0 (5·4) |
Median (IQR) | 29·0 (26·0–35·0) | 29·0 (25·0–32·0) |
Education level | ||
No formal education | 35 (9%) | 7 (2%) |
Partial education up to completion of primary education | 199 (51%) | 155 (52%) |
Partial secondary education and beyond | 151 (39%) | 129 (43%) |
Missing data | 7 | 7 |
Partner status | ||
Yes, live together | 274 (70%) | 211 (71%) |
No current partner | 110 (28%) | 70 (23%) |
Missing data | 8 | 17 |
Disclosure of HIV status to anyone | 252 (64%) | 212 (71%) |
Income level* | ||
<750 KSH | 211 (54%) | 162 (54%) |
≥750 KSH | 153 (39%) | 116 (39%) |
Missing data | 28 | 20 |
Travel costs to hospital (KSH)* | 159·1 (107·9) | 141·15 (151·8) |
Missing data | 8 | 10 |
Infant sex | ||
Male | 193 (49%) | 151 (51%) |
Female | 199 (51%) | 145 (49%) |
Missing data | 0 | 2 |
Infant age at first PCR (weeks) | 6·3 (6·0–6·7) | 6·4 (6·1–7·4) |
Missing data | 0 | 1 |
Data are n (%), mean (SD), or median (IQR).
100 KSH is roughly US$1.
392 (100%) participants at intervention sites were enrolled in the HITSystem. Infants diagnosed as HIVpositive received the intervention for a median of 2·1 months (IQR 1·6–4·8) between early infant diagnosis enrolment and ART initiation, and HIV-negative infants received intervention follow-up for a median of 17·0 months (IQR 16·6–17·6) between early infant diagnosis enrolment and the 18-month retest. Text messages were sent directly to 373 (95%) of 392 mothers; six women provided a phone number for their designated community health-care worker to receive notifications, and 12 mothers did not provide phone numbers. 1495 text messages were sent in total, of which 1096 (73%) were confirmed received by the recipients’ phones; a median of 3·0 texts (range 1–10) were received per participant. No disruptions were reported to daily access of the HITSystem during the study period; however, no text messages were sent during a 1-week period in January, 2015, and an error in programming resulted in missed provider alerts and automated text messages for 9-month retesting for 57 infants who reached 9 months of age before November, 2014, when the issue was identified and corrected. Complete early infant diagnosis among these 57 infants did not differ significantly from the remainder of the infants at intervention sites (p=0·06).
Compared with control sites, mothers at intervention sites were older, had lower education, and were less likely to have disclosed their HIV status to anyone (table 1). These variables were all controlled for in subsequent analyses.
The HITSystem intervention had a significant effect on receipt of complete early infant diagnosis services compared with standard of care (figure 2). A higher proportion of infants enrolled in the HITSystem received complete early infant diagnosis compared with controls (334 [85%] of 392 vs 180 [60%] of 298; adjusted OR 3·7, 95% CI 2·5–5·5; p<0–0001). Hospital volume, maternal age, education level, HIV disclosure status, and an interaction effect between study arm and hospital volume were treated as fixed effects in this model. Within this aggregate measure of complete early infant diagnosis services, 391 (100%) of 392 participants at intervention sites received prophylaxis for opportunistic infections (vs 267 [90%] of 298 at control sites), 392 (100%) of 392 had initial dried bloodspot sample collection (vs 296 [99%] of 298), 392 (100%) of 392 had PCR results returned to hospital (vs 289 [97%] of 298), 388 (99%) of 392 mothers were notified of the the PCR result (vs 266 [89%] of 298), 360 (97%) of 372 HIV-negative infants were retested at 9 months (vs 265 [91%] of 291), 315 (85%) of 372 HIV-negative infants were retested at 18 months (vs 201 [69%] of 290), and 21 (100%) of 21 HIV-positive infants at intervention sites initiated ART (vs eight [73%] of 11 HIV-positive infants at control sites).
When analysed by volume of patients at each hospital, a significant interaction between hospital volume and study group was noted. The effect of the HITSystem on complete early infant diagnosis at high-volume hospitals in the intervention and control groups was not significant; however, the odds of receiving complete early infant diagnosis services increased significantly at medium-volume and low-volume hospitals implementing the HITSystem (table 2). When analysed as a modified intention-to-treat approach, with all infant deaths and transfers to other facilities regarded as not achieving complete early infant diagnosis, the HITSystem intervention effect on complete early infant diagnosis retention remained significant.
Table 2:
Intervention (n=392) | Control (n=298) | Adjusted odds ratio (95% CI)* | p value | |
---|---|---|---|---|
Hospital volume | ||||
High volume (n=341) | 184/226 (81%, 76–87) | 99/115 (86%, 80–93) | 0·7 (0·4–1·3) | 0·28 |
Medium volume (n=208) | 104/109 (95%, 92–99) | 55/99 (56%, 46–66) | 16·8 (6·3–44·9) | <0·0001 |
Low volume (n=141) | 46/57 (81%, 71–91) | 26/84 (31%, 21–41) | 9·4 (4·2–21·4) | <0·0001 |
Analysis strategy | ||||
Standard | 334/392 (85%, 82–89) | 180/298 (60%, 55–66) | 3·7 (2·5–5·5) | <0·0001 |
Intention to treat† | 334/474 (70%, 66–75) | 180/334 (54%, 49–59) | 2·0 (1·6–2·8) | <0·0001 |
Data are n (%, 95% CI). Table shows the differential effect of the HITSystem by hospital volume and analysis strategy.
Adjusted for maternal age, education level, and HIV disclosure status.
Infant deaths and transfers regarded as incomplete early infant diagnosis.
HIV-exposed infants enrolled in the HITSystem received more efficient early infant diagnosis services compared with those receiving standard of care at control hospitals. Results from the initial PCR test were returned faster for infants at intervention sites compared with control sites (median 20·0 days [IQR 15·0–32·0] vs 38·5 days [26·0–66·0]; incident rate 2·0, 95% CI 1·8–2–2; p<0·0001), and mothers were notified of their infants’ test results faster (median 14·0 days [IQR 7·0–24·0] vs 23·0 days [13·0–33·0]; incident rate 1·6, 95% CI 1·3–2·0; p<0·0001), controlling for maternal age, education level, and HIV disclosure status. Modified intention-to-treat analyses for these same outcomes retained significant interaction effects for PCR turnaround time (incident rate 2·0, 95% CI 1·8–2·2; p<0·0001) and turnaround time for notifying mothers of results (1·5, 1·2–1·8; p=0·0002).
No difference was noted in mean infant age at 9-month and 18-month retesting between intervention and control sites (mean age 9·4 months [SD 0·8] vs 9·4 months [1·2]; p=0·62; and mean 18·7 months [1·5] vs 18·5 months [1·6]; p=0·13). Turnaround time for ART initiation was longer at intervention hospitals compared with control sites (median 21·0 days [IQR 4·0–66·0] vs 0·5 days [0·0–6·5]; p=0·;03).
27 (84%) of 32 HIV-positive infants were diagnosed at their initial PCR test (20 of 21 at intervention sites and seven of 11 at control sites), with three infants diagnosed at the 9-month retest and two at the 18-month retest. One infant from a control site was diagnosed at 6 weeks but never started ART. Of the other 26 infants diagnosed HIV-positive at their first test, age at ART initiation was younger at intervention sites compared with control sites (20 infants, median age 17·5 weeks [IQR 13·7–21·8] vs six infants, median age 25·1 weeks [21·0–26·9]; p=0·04; figure 3). HIV-positive infants at control sites were older at the time of sample collection for their first HIV test compared with those at intervention sites (median age 11·4 weeks [IQR 8·7–15·4] vs median age 6·4 weeks [6·0–7·7]). Few infants who were diagnosed after their first 6-week test initiated ART within the targeted first 12 weeks of life (three [15%] of 20 at intervention sites and none at control sites). However, ten (50%) of 20 mothers of HIV-positive infants at intervention sites were notified of their infant’s result before 12 weeks of age (median age at notification 12·5 weeks [IQR 10·6–15·4] vs 24·7 weeks [21·0–25·9] at control sites), making ART initiation before age 12 weeks feasible if same-day or next-day ART initiation had been facilitated (PCR results were available at median age 10·5 weeks [IQR 9·1–12·0] at intervention sites vs 17·7 weeks [13·1–23·6] at control sites).
No adverse events were recorded that were related to participation in the HITSystem intervention or this research study. During the study, 26 infant deaths were reported as adverse events to the institutional review boards (17 at intervention sites and nine at control sites), although none were related to study participation. In intention-to-treat analyses, more deaths were reported among HIV-positive infants (three [8%] of 38) than among HIV-negative infants (23 [3%] of 770; p=0·09). The number of deaths among HIV-positive infants at intervention sites (two [8%] of 25) was similar to the number at control sites (one [8%] of 13).
Discussion
The results of our cluster-randomised trial show significant improvements in early infant diagnosis quality and efficiency achieved by the HITSystem intervention. Infants enrolled at hospitals implementing the HITSystem were significantly more likely to receive complete early infant diagnosis services. Hospitals of medium and lower patient volume using the HITSystem had substantially increased odds of receiving complete early infant diagnosis services, whereas no difference was noted between high-volume sites. In modified intention-to-treat analyses, in which all infant deaths and transfers were judged not to have completed early infant diagnosis rather than regarded as exclusions, the intervention effect of the HITSystem on the primary outcome of complete early infant diagnosis remained significant. Several significant efficiencies in early infant diagnosis services were achieved at sites implementing the HITSystem; turnaround times for infant PCR test results were 2·6 weeks faster than at control sites, and mothers were notified of their infants’ results 1·3 weeks faster than at control sites. The turnaround time for infant PCR test results in our trial (20 days) was similar to that reported in an assessment of programmatic HITSystem implementation (24·7 days).22 The turnaround time for ART initiation among infants diagnosed HIV-positive, an important target for efficiency, was not improved at sites implementing the HITSystem, and in fact was significantly slower. Despite these delays, median infant age at ART initiation was 7·6 weeks younger at sites implementing the HITSystem.
Updated 2014 policy guidance advised immediate ART initiation among HIV-positive infants,23 but the speed of adoption varied across study sites. By the time our trial started, all control sites had already adopted immediate ART initiation, which for many infants resulted in ART initiation at the same visit when the mother was informed of the HIV-positive results. Slower ART initiation at intervention sites was probably attributable to the practice of requiring mothers to complete multiple adherence counselling sessions before ART initiation, which was the norm at two of the three intervention hospitals. This practice typically resulted in multiweek delays in initiating ART. We had not encountered this practice of delayed ART initation in our pilot or programme sites and, thus, did not identify this important disparity during initial site selection. During our trial, one of the two intervention hospitals adapted their practice and began initiating ART after the first adherence counselling session (typically 1–2 weeks after notification). Unfortunately, the larger intervention facility, which contributed eight of 21 HIV-positive infants, did not modify their practice during the study period. We do not believe that the HITSystem accounted for this delay, because other Kenyan hospitals implementing the HITSystem during an overlapping period reported shorter turnaround times between notification of the mother and ART initiation (median 6·5 days).8 This practice revealed an important barrier to rapid ART that requires greater emphasis at the county and health-facility level to ensure updated treatment guidelines are implemented.
Most infants were diagnosed with HIV during their initial test. Although median infant age at the first test was similar across implementation and control sites (6·3 weeks vs 6·4 weeks), age at first test among HIV-positive infants at control sites was nearly twice that of HIV-positive infants at intervention sites (6·4 weeks vs 11·4 weeks). In view of the small sample size of HIV-positive infants, this finding is difficult to explain and cannot be attributed directly to the intervention, because engagement begins the day infants are brought to the hospital for testing. Even with the improved efficiencies added by the HITSystem, delays in ART initiation significantly reduced the proportion of infants initating ART within 12 weeks of age, thereby reducing the maximum benefit of ART.5
Technology solutions for health-care and HIV services are quickly becoming part of the new standard of care in Kenya, with 2016 national coverage of mobile phones estimated at 90%24 and the emergence of multiple electronic medical records systems to support HIV care at government facilities. During the period of study implementation (2013–17), findings of four randomised controlled trials using text messaging outreach to mothers showed significant improvements in PMTCT retention and uptake of early infant diagnosis services (three in Kenya25–27 and one in Mozambique).28 In a 2016 systematic review,29 the effect of mHealth interventions on improving PMTCT and early infant diagnosis was summarised from five studies, with a significant increase reported in early infant diagnosis uptake at 6 weeks post partum (pooled relative risk 1·2, 95% CI 1·1–1·3). Findings of a 2017 systematic review30 ascertained that SMS and general packet radio service (GPRS) printers reduced turnaround time for infant PCR test results by 25% compared with a paper-based courier (51·1 days vs 68·0 days). Our trial provides the most comprehensive evidence to date of the success of an mHealth intervention to improve quality and efficiency across the complete early infant diagnosis cascade of care.
Strengths of our trial include its prospective design and the long duration of infant follow-up (until 18 months post partum). In most studies reporting uptake of early infant diagnosis services, infants have been followed up until the result of their first infant PCR test (typically 8–10 weeks post partum). Few studies report ART initiation for infants diagnosed with HIV and, to our knowledge, no other Kenyan studies assessing early infant diagnosis have reported infant age at ART initiation. The aggregate measure of complete early infant diagnosis services provides a new and more comprehensive outcome to measure the quality of early infant diagnosis services across the full cascade of care for both HIV-positive and HIV-negative infants. Multiple standalone measures of early infant diagnosis efficiency are reported by us here, including turnaround time for notifying the mother of PCR results and infants’ ages at targeted retesting timepoints, which are rarely reported in studies. In Kenya, considerable investments in PMTCT and early infant diagnosis services have occurred over the study period. Establishing intervention efficacy against a continually improving standard of care presents a challenge for studies with multiyear follow-up periods. Nevertheless, significant intervention effects were achieved in our trial, even with modified intention-to-treat analyses.
Limitations of our trial include the small number of clusters and the imbalance in number of patients at the high-volume cluster. Randomisation at the level of clusters (hospital) reduces comparability between mother–infant pairs (unit of analysis) compared with randomisation at the individual level. Significant differences in maternal characteristics existed between implementation and control sites, with mothers at control sites (receiving the standard of care) reporting greater education and higher rates of HIV disclosure compared with those at implementation sites (enrolled in the HITSystem); both of these variables are associated with better adherence to PMTCT and early infant diagnosis guidelines.6,31,32 Older maternal age is also associated with better adherence to PMTCT and early infant diagnosis services,31 and although mothers at implementation sites were older than those at control sites, the difference of just over 1 year (mean age 30·2 years vs 29·0 years) is not meaningful. We included only mother–infant pairs who presented for early infant diagnosis through the maternal and child health department of the hospital before age 24 weeks. The intervention might vary in efficacy for those engaging in early infant diagnosis at a later age or who are identified through different hospital departments, reflecting a population who might be less engaged in care. Although mothers younger than 18 years (roughly 2% of all mothers enrolled in the HITSystem during the active study enrolment period) were excluded from analyses, inclusion criteria for future research will include younger mothers who might face more barriers to retention. The use of clinical staff at control sites to enrol participants, compared with a dedicated research staff at intervention sites, could account for missed enrolment opportunities, in view of multiple demands on clinical providers. Higher numbers of transfers were recorded at intervention facilities than at control sites, which probably reflects the improved documentation of data with the HITSystem compared with the standard-of-care paper-based registries. Poorer quality and incompleteness of clinical records and infant registries at control sites could have biased outcomes. However, rigorous and repeated efforts to cross-reference multiple sources of data helped to optimise the quality and accuracy of control data.
The HITSystem is an innovative public health intervention that makes the most of accessible mobile and internet technology to significantly improve the quality and efficiency of early infant diagnosis services in Kenya and other low-resource settings. Qualitative data from interviews with mothers at intervention and control sites emphasise the same themes of early infant diagnosis quality and improved efficiency gained with the HITSystem.33 User-satisfaction data from clinical and laboratory providers have described the ease of use and perceived usefulness of the HITSystem,34 which contributes to successful adoption of the system in programmatic settings. The HITSystem has been reconfigured to support Kenya’s new and more intensive algorithm for early infant diagnosis testing. As point-of-care infant HIV testing expands and eliminates the need for sample tracking features, the participant-tracking features of the HITSystem maintain relevance to ensure infant retesting and retention throughout the early infant diagnosis cascade. A cost-effectiveness analysis currently underway will inform decisions about national and potentially regional scale-up of the HITSystem. Wider use of a system-level intervention that accelerates diagnosis and supports retention among mother–infant pairs can be implemented to significantly strengthen HIV prevention and treatment for improved maternal and paediatric health outcomes.
Supplementary Material
Research in context
Evidence before this study
We searched PubMed, the Cochrane Library, and Google Scholar between Jan 1, 2008, and Dec 31, 2012, with the terms “early infant diagnosis”, “early infant diagnosis retention”, “infant ART initiation”, “HIV-exposed infants”, “turn-around time for infant HIV DNA PCR results”, “mHealth”, “eHealth”, “mobile phones”, “Internet”, and “early infant diagnosis cascade of care”. When we had the idea for the study in 2012, no published data were available from randomised controlled trials assessing paediatric HIV or early infant diagnosis outcomes with mobile phone or internet components. During the course of this study (2013–18), evidence has accumulated supporting the usefulness of mobile health (mHealth) interventions—including short message service (SMS) text messaging—to improve early infant diagnosis uptake, completion of the initial test, and turnaround time for PCR results.
Added value of this study
To the best of our knowledge, we have done the first cluster-randomised trial of an intervention using a combination of SMS and internet technology to target both clinical and laboratory outcomes across the complete 18-month early infant diagnosis cascade. The HITSystem is a novel integrated system linking multiple stakeholders, including early infant diagnosis providers, laboratory technicians, and mothers and infants. The primary outcome was an aggregate measure of complete early infant diagnosis, which is a new approach to measuring the quality of early infant diagnosis that reflects receipt of all indicated services along the cascade of care. Eligibility of participants for each early infant diagnosis intervention was assessed according to previous HIV test results, allowing quality and efficiency of services to be measured contextually at each stage. To increase the generalisability of findings, hospitals were stratified by volume of patients (high, medium, and low) to assess implementation experiences in multiple settings.
Implications of all the available evidence
Our study provides the most comprehensive evidence to date of the feasibility and utility of an mHealth early infant diagnosis intervention to significantly improve the quality (complete early infant diagnosis services) and efficiency (turnaround time for PCR results and notifying the mother of the PCR results) of early infant diagnosis services. Even in remote settings, internet-based interventions accessed by modems at hospitals and central laboratories—such as the one evaluated here—have proved feasible, and text messaging is now ubiquitous in Kenya.
For the study protocol see https://globalhealthinnovations.org/research
Acknowledgments
This work was supported by the National Institute of Child Health and Human Development (number R01HD076673, awarded to the University of Kansas Medical Center). The Kenya Medical Research Institute (KEMRI), Global Health Innovations, Children’s Mercy Kansas City, and OnTarget Interactive were collaborative partners in these efforts. We acknowledge the members of the HITSystem Study Team who had a key role in implementation: Martin Ochieng, Shadrack Babu, Elizabeth Nyambura Muchoki, and Eric Muriithi.We also thank Beryne Odeny, who did an exhaustive literature review before publication, and Michael Sweat and Andrea Ruff, who provided strategic guidance throughout the study. We are grateful for implementation support from mentor mothers and clinical staff, and we thank the mother–infant pairs who participated in this research. We also acknowledge the important role of our government partners at the Kenya National AIDS and STI Control Programme. We thank the Director of KEMRI for permission to publish this report. This report is solely the responsibility of the authors and does not necessarily represent the official views of the US National Institutes of Health.
Funding National Institute of Child Health and Human Development.
Declaration of interests
SF-K, BG, CW, MM, RL, MB, MS, and KG report funding from the National Institutes of Health (grant numbers R01HD076673 and R34MH107337) during the conduct of the study. BG reports non-financial support from HITSystem, outside the submitted work. HITSystem has a patent copyright issued under HITSystem LLC, of which BG is 33% owner (copyright TXu001763374). No owner, including BG, receives any royalities, salary, profit, or benefits from this position. AC, NN, SK, TAO, JKD, and NM declare no competing interests.
Footnotes
Role of the funding source
The funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report. SF-K, AC, NN, and CW had access to the final dataset. The corresponding author had final responsibility for the decision to submit for publication.
Contributor Information
Sarah Finocchario-Kessler, Department of Family Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
Brad Gautney, Global Health Innovations, Dallas, TX, USA.
AnLin Cheng, University of Missouri-Kansas City, School of Medicine, Kansas City, MO, USA.
Catherine Wexler, Department of Family Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
May Maloba, Global Health Innovations, Nairobi, Kenya.
Niaman Nazir, Department of Preventive Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
Samoel Khamadi, Center for Virus Research, Kenya Medical Research Institute, Nairobi, Kenya.
Raphael Lwembe, Center for Virus Research, Kenya Medical Research Institute, Nairobi, Kenya.
Melinda Brown, Department of Family Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
Thomas A Odeny, University of Missouri-Kansas City, School of Medicine, Kansas City, MO, USA; Center for Microbiology Research, Kenya Medical Research Institute, Nairobi, Kenya.
Jacinda K Dariotis, College of Education, Criminal Justice, and Human Services, University of Cincinatti, Cincinatti, OH, USA.
Matthew Sandbulte, Department of Family Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
Natabhona Mabachi, Department of Family Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
Kathy Goggin, University of Missouri-Kansas City, School of Medicine, Kansas City, MO, USA; Children’s Mercy Kansas City, Health Services and Outcomes Research, Kansas City, MO, USA.
References
- 1.Ministry of Health. Guidelines for prevention of mother to child transmission (PMTCT) of HIV/AiDs in Kenya. January 1, 2012. http://guidelines.health.go.ke/#/category/27/64/meta (accessed May 25, 2018).
- 2.National AIDS Control Council. Kenya AIDS response progress report 2014: progress towards zero. March, 2014. http://www.unaids.org/sites/default/files/country/documents/KEN_narrative_report_2014.pdf (accessed May 25, 2018).
- 3.UNAIDS. 2013 progress report on the Global Plan: towards the elimination of new HIV infections among children by 2015 and keeping their mothers alive. June 25, 2013. http://www.unaids.org/en/resources/documents/2013/20130625_progress_global_plan_en.pdf (accessed May 25, 2018).
- 4.UNAIDS. Country factsheets: Kenya 2016. 2016. http://www.unaids.org/en/regionscountries/countries/kenya (accessed May 25, 2018).
- 5.Violari A, Cotton MF, Gibb DM, et al. Early antiretroviral therapy and mortality among HIV-infected infants. N Engl J Med 2008; 359: 2233–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Goggin K, Wexler C, Nazir N, et al. Predictors of infant age at enrollment in early infant diagnosis services in Kenya. AIDS Behav 2016; 20: 2141–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hassan AS, Sakwa EM, Nabwera HM, et al. Dynamics and constraints of early infant diagnosis of HIV infection in rural Kenya. AIDS Behav 2012; 16: 5–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wexler C, Nazir N, Gautney B, et al. Predictors of timely ART initiation among HIV-positive infants in Kenya. 22nd International AIDS Conference; Amsterdam, Netherlands; July 23–27, 2018: THPEE715 abstr. http://programme.aids2018.org/Abstract/Abstract/10797 (accessed Sept 11, 2018). [Google Scholar]
- 9.Finocchario-Kessler S, Gautney BJ, Khamadi S, et al. If you text them, they will come: using the HIV infant tracking system to improve early infant diagnosis quality and retention in Kenya. AIDS 2014; 28 (suppl 3): S313–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Finocchario-Kessler S, Goggin K, Khamadi S, et al. Improving early infant HIV diagnosis in Kenya: study protocol of a cluster-randomized efficacy trial of the HITSystem. Implement Sci 2015; 10: 96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Communications Commission of Kenya. Annual report 2011–2012 (CCK/CPR/AR/007). https://ca.go.ke/wp-content/uploads/2018/02/Annual-Report-for-the-Financial-Year-2011-2012.pdf (accessed May 25, 2018).
- 12.Pop-Eleches C, Thirumurthy H, Habyarimana JP, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011; 25: 825–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lester RT, Ritvo P, Mills EJ, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010; 376: 1838–45. [DOI] [PubMed] [Google Scholar]
- 14.Seidenberg P, Nicholson S, Schaefer M, et al. Early infant diagnosis of HIV infection in Zambia through mobile phone texting of blood test results. Bull World Health Organ 2012; 90: 348–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Deo S, Crea L, Quevedo J, et al. Expedited results delivery systems using GPRS technology significantly reduce early infant diagnosis test turnaround times. J Acquir Immune Defic Syndr 2015; 70: e1–e4. [DOI] [PubMed] [Google Scholar]
- 16.Anon. Assessment of EID services and SMS/GPRS printer pilot program. Ethiopia: Clinton Health Access Initiative, 2012. [Google Scholar]
- 17.Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013; 8: 139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Fox MP, Rosen S. Systematic review of retention of pediatric patients on HIV treatment in low and middle-income countries 2008–2013. AIDS 2015; 29: 493–502. [DOI] [PubMed] [Google Scholar]
- 19.Giuliano M, Liotta G, Andreotti M, et al. Retention, transfer out and loss to follow-up two years after delivery in a cohort of HIV+ pregnant women in Malawi. Int J STD AIDS 2016; 27: 462–68. [DOI] [PubMed] [Google Scholar]
- 20.Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health 2004; 94: 423–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.National AIDS and STI Control Programme, Ministry of Health, Kenya. Guidelines on use of antiretroviral drugs for treating and preventing HIV infection in Kenya: 2016 edition. https://aidsfree.usaid.gov/sites/default/files/kenya_art_2016.pdf (accessed Sept 24, 2018).
- 22.Wexler C, Cheng A-L, Gautney B, et al. Evaluating turnaround times for early infant diagnosis samples in Kenya from 2011–2014: a retrospective analysis of HITSystem program data. PLoS One 2017; 12: e0181005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.National AIDS and STI Control Programme, Ministry of Health, Kenya. Guidelines on use of antiretroviral drugs for treating and preventing HIV infection: rapid advice. June, 2014. https://aidsfree.usaid.gov/sites/default/files/tx_kenya_2014.pdf (accessed May 25, 2018).
- 24.Communications Authority of Kenya. Annual report 2015–2016. https://ca.go.ke/wp-content/uploads/2018/02/Annual-Report-for-the-Financial-Year-2015-2016.pdf (accessed May 25, 2018).
- 25.Odeny TA, Bukusi EA, Cohen CR, Yuhas K, Camlin CS, McClelland RS. Texting improves testing: a randomized trial of two-way SMS to increase postpartum prevention of mother-to-child transmission retention and infant HIV testing. AIDS 2014; 28: 2307–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kassaye SG, Ong’ech J, Sirengo M, et al. Cluster-randomized controlled study of SMS text messages for prevention of mother-to-child transmission of HIV in rural Kenya. AIDS Res Treat 2016; 2016: 1289328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Kebaya L, Nduati R, Wamalwa D, Kariuki N, Bashir A. PO-0260a: efficacy of mobile phone use on adherence to nevirapine prophylaxis and retention in care among the HIV-exposed infants in PMTCT: a randomised controlled trial. Arch Dis Child 2014; 99: A329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Joseph-Davey D, Ponce W, Augusto O, Traca D, de Palha de Sousa C. Improved uptake of institutional birth and early infant HIV diagnosis following SMS reminders among PMTCT patients in Mozambique: a randomized control trial. 7th IAS Conference on HIV Pathogenesis, Treatment, and Prevention; Kuala Lumpur, Malaysia; June 30, 2013-July 3, 2013. http://pag.ias2013.org/EPosterHandler.axd?aid=3055 (accessed Sept 11, 2018). [Google Scholar]
- 29.Ambia J, Mandala J. A systematic review of interventions to improve prevention of mother-to-child HIV transmission service delivery and promote retention. J Int AIDS Soc 2016; 19: 20309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Vojnov L, Markby J, Boeke C, et al. Impact of SMS/GPRS printers in reducing time to early infant diagnosis compared with routine result reporting: a systematic review and meta-analysis. J Acquir Immune Defic Syndr 2017; 76: 522–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Izudi J, Auma S, Alege JB. Early diagnosis of HIV among infants born to HIV-positive mothers on option-B plus in Kampala, Uganda. AIDS Res Treat 2017; 2017: 4654763. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Spangler SA, Onono M, Bukusi EA, Cohen CR, Turan JM. HIV-positive status disclosure and use of essential PMTCT and maternal health services in rural Kenya. J Acquir Immune Defic Syndr 2014; 67 (suppl 4): S235–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Brown M, Wexler C, Ochieng M, et al. The user experience: perspectives from MCH providers, laboratory providers, and HIV+ mothers enrolled in the HITSystem. 9th IAS Conference on HIV Science (IAS 2017); Paris, France; July 23–26, 2017:. http://programme.ias2017.org/Abstract/Abstract/4426 (accessed Sept 11, 2018). [Google Scholar]
- 34.Wexler C, Brown M, Hurley EA, et al. Implementing an ehealth technology to address gaps in early infant diagnosis services: a qualitative assessment of Kenyan providers’ experiences. JMIR Mhealth Uhealth 2018; 6: e169. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.