Lessons learned from implementing the HIV Infant Tracking System (HITSystem): A web-based intervention to improve early infant diagnosis in Kenya

Abstract

Objectives

Guided by the RE-AIM model, we describe preliminary data and lessons learned from multiple serial implementations of an eHealth intervention to improve early infant diagnosis (EID) of HIV in Kenya.

Methods

We describe the Reach, Effectiveness, Adoption, Implementation and Maintenance of the HITSystem, an eHealth intervention that links key stakeholders to improve retention and outcomes in EID. Our target community includes mother-infant pairs utilizing EID services and government health care providers and lab personnel. We also explore our own role as program and research personnel supporting the dissemination and scale up of the HITSystem in Kenya.

Results

Key findings illustrate the importance of continual adaptation of the HITSystem interface to accommodate varied stakeholders’ workflows in different settings. Surprisingly, technology capacity and internet connectivity posed minimal short-term challenges. Early and sustained ownership of the HITSystem among stakeholders proved critical to reach, effectiveness and successful adoption, implementation and maintenance.

Conclusions

Strong and sustained collaborations with stakeholders improve the quality and reach of eHealth public health interventions.

Introduction

Early infant diagnosis (EID) and prompt treatment of HIV-positive infants are critical program components of Prevention of Mother-to-Child Transmission (PMTCT) efforts (1,2). EID can facilitate early (before 12 weeks postnatal) initiation of antiretroviral therapy (ART) which can reduce mortality by 76% and slow the progression of HIV by 75% (3). Current challenges to quality EID include late infant testing, delayed or lost test results, passive systems for informing mothers, high loss -to-follow-up, and poor linkage to treatment for HIV-positive infants (4–6). Primary stakeholders include mothers, EID providers who collect and send samples for HIV DNA PCR testing, courier services that deliver samples, laboratories technicians who conduct and report PCR testing, and community health workers who reach out to mothers of HIV-exposed infants (7). These stakeholders span clinical and community settings and vary widely in training and capacity. In an effort to improve communication and accountability between varied EID stakeholders, we developed the HIV Infant Tracking System (HITSystem).

The HITSystem is a web-based, automated intervention designed to overcome current EID barriers by providing efficient prospective tracking of HIV-exposed infants. The HITSystem is accessed online via computer, using mobile broadband modems that respond to cellular signal rather than hardwired internet access; making this system feasible even in remote areas. The HITSystem triggers electronic action ‘alerts’ for both EID providers and lab technicians when time-sensitive EID interventions are due, see Table I. These alerts facilitate infant tracking so that those who default from care can be easily identified and quickly targeted for outreach. A built-in text messaging system sends messages to mothers’ mobile phones when test results are ready or follow up visits are needed. HITSystem data is de-identified and stored on a secure server, which facilitates reporting. A more detailed description of the HITSystem has been previously published (8). The primary goals of the HITSystem are to: a) reduce turnaround time for the PCR testing cycle, (b) facilitate early ART initiation for infants identified as HIV+, and (c) improve EID retention.

Table I.

HITSystem Targets: 8 time-sensitive EID interventions

Initiation of OI prophylaxis at 6 weeks. Collect dried blood spot (DBS) for PCR test by 6 weeks. Receipt of DBS at lab within 10 days of collection. Return of PCR results from lab within 2 weeks. Notify mother within 2 weeks of the EID provider receiving results. Initiate all HIV-infected infants on ART within 4 weeks of notifying the mother. Retest all HIV-uninfected infants at 9 months, initiate ART w/in 4 weeks if applicable. Retest all HIV-uninfected infants at 18 months, initiate ART w/in 4 weeks if applicable, complete EID.

^*For more detail, refer to Figure 1 illustrating the HITSystem process (8).

After promising pilot testing of the HITSystem in two Kenyan health facilities, we partnered with key stakeholders (detailed below) to implement the HITSystem in other health facilities across Kenya. This paper highlights experiences and lessons learned from implementing the HITSystem at ten sites in Kenya. Guided by the RE-AIM model (9), we analyze and describe our efforts to disseminate the HITSystem.

Methods

Setting

Ninety-five percent of HIV+ Kenyans who seek care and treatment receive services through government hospitals (10). As such, we maximized the representativeness of facilities by targeting ten government hospitals at the referral, county, sub-county and health center levels; including three urban and seven peri-urban hospitals, see Table II. Consistent with the size and population density of their catchment areas, the average monthly volume of EID patients varied significantly between sites and ranged from 6 to 26 mother-infant pairs. The HITSystem was implemented between April 2011 and November 2013 at these ten health facilities and the designated regional central laboratories (i.e., Kericho, Kisumu or Nairobi). Implementation and evaluation of sites was conducted between April 2011 and May 2014. Follow-up periods ranged between 6 and 38 months depending on month of initial implementation. The initial sites were selected based on perceived need, logistical feasibility, absence of other EID-related interventions, and approval from hospital administrators.

Table II.

Characteristics of Hospitals Implementing the HITSystem, Including Level of Ownership and Estimated time to program adoption.

Hospital	Region	Mo/Yr implemented	Setting	Volume May/2014 Total (monthly)	Cell phone utilization	Model of Support	Ownership Level, range 1–5 (SD)	Time to adoption, months (SD)
1. Referral Hospital A	Nairobi	Apr 2011	Urban	745 (20)	(99.5)	Med	4.25 (.96)	4.8 (4.9)
2. Health Centre B	Nairobi	Nov 2011	Urban	514 (15)	(86.8)	High	3.25 (.5)	14.3 (9.6)
3. County Hospital C	Rift Valley	Nov 2012	P-urban	319 (12)	(76.2)	Med	4.75 (.5)	2.75(2.4)
4. County Hospital D	Rift Valley	Nov 2012	P-urban	558 (15)	(68.1)	High	3.25(.29)	10.5 (3.0)
5. Sub-County Hospital E	Western	Mar 2013	P-urban	104 (<10)	(80.8)	Low	2.0 (1.15)	in progress
6. Sub-County Hospital F	Western	Mar 2013	P-urban	123 (<10)	(74.8)	Low	3.37(.75)	4.0 (.81)
7. Sub-County Hospital G	Western	Mar 2013	P-urban	103 (<10)	(67.0)	Med	1.37(.48)	in progress
8. Teach & Ref Hospital H	Nyanza	Sep 2013	Urban	306 (25)	(92.8)	Low	3.0(1.35)	7.8 (3.2)
9. Health Center I	Nairobi	Oct 2013	Urban	69 (<10)	(97.1)	High	3.0 (1.41)	in progress
10. Health Center J	Nairobi	Feb 2014	Urban	15 (<5)	(100)	High	2.87(1.31)	in progress

To increase the sense of anonymity, information regarding hospital level and geographic region has been provided rather than hospital name.

“in progress”= not yet achieved full adoption/ownership of the HITSystem by health care providers but the process is ongoing

Partners

The implementation process of the HITSystem was undertaken by a multidisciplinary group of collaborating partners that included the: NGO that developed the HITSystem (Global Health Innovations), national testing laboratories in Kenya that support PCR testing (Kenya Medical Research Institute), internet marketing company that designed the interface for the HITSystem (OnTarget), and researchers from two academic medical centers that provided design and evaluation support (University of Kansas Medical Center and Children’s Mercy Hospital Kansas City). The Walter Reed U.S. Military HIV Research Program in Kenya also supported training and implementation at health facilities within their catchment area. While not a direct collaborator, Kenya’s National AIDS & STI Control Programme has encouraged the evaluation of HITSystem implementation to inform feasibility of national scale up. Key members from each partnering organization worked collaboratively through in-person meetings and routine remote communication to develop effective strategies for approaching, introducing, training, implementing and evaluating the HITSystem.

Implementation

After gaining the appropriate facility level permissions, the team scheduled training with existing EID and laboratory staff who would utilize the HITSystem and worked with administrators to integrate these efforts as part of the routine workflow in an effort to foster ownership of the program and to reduce the perception of an increased burden. An overview of the HITSystem, which lasted approximately two hours and included a question and answer session, was provided to key personnel from involved departments (Maternal and Child Health [MCH]/EID, internal laboratory, and pediatric provider from the Comprehensive Care Center [CCC] where ART is provided). We then provided tailored training with hands-on use of the system using practice scenarios similar to what staff would encounter in everyday use. Trainings lasted one day (6 hrs) for MCH/EID providers and a half day (3 hrs) for lab technicians at the central laboratory. Technical assistance was provided during the first month of implementation and refresher/booster training was provided annually or when necessitated by program modifications or personnel changes. Clinical and laboratory staff provided ongoing intervention refinement by suggesting interface modifications.

Evaluation Model

Ideal for our purposes, the RE-AIM model provides a framework for evaluating the impact of real-world health promotion interventions on various levels (i.e., individual, organization, community) across the five key dimensions of Reach, Effectiveness, Adoption, Implementation, and Maintenance (11). As applied here, “Reach” focuses on understanding the penetration of the HITSystem into its intended target audience of all mother-infant pairs who enroll in EID at the ten targeted facilities and the range of clinical and laboratory providers involved in providing EID services. “Effectiveness” involves documenting the impact of the HITSystem on intervention outcomes. Specifically, whether implementation led to: (a) reduced turn-around time for the PCR testing cycle, (b) early ART initiation for infants identified as HIV+, and (c) improved rates of re-testing among uninfected infants at 9 months postnatal. “Adoption” is similar to “Reach” but is assessed at the level of settings (health facilities and laboratories). In this report, we assess the level of HITSystem adoption among all targeted settings and explore how representative the clinics and labs that adopted the HITSystem are compared to other government facilities. “Implementation,” or intervention fidelity, captures the extent to which different components of the HITSystem intervention are delivered as originally designed, what adaptations were made, and the costs of the program in terms of time and money. “Maintenance” is assessed at both the individual and the setting level. At the individual level, it documents the long-term effects of the HITSystem on intended outcomes. At the setting level, it indicates the extent to which the HITSystem has been integrated into standard care and staff workflows in its intended or a modified form.

Analyses

Process and outcome data captured by the HITSystem was exported to STATA intercooled version 11 for analyses. We calculated descriptive statistics including proportions for categorical variables and means (SD, range) or median (interquartile range) for continuous variables.

Results and Discussion

REACH

The HITSystem was designed to reduce barriers to use and maximize reach at the facility level. For example, it requires neither wire-based internet access nor a continuous supply of electricity, making it ideal for use in health facilities in remote areas of Kenya and other low resource countries. Use of cellular mobile broadband permits internet access through USB devices that allow for uploading of data at any time and can be inexpensively resupplied with airtime, much as one would top up mobile phone minutes. In instances when poor broadband quality did impair the speed of connection and processing, providers waited until the end-of-day to complete data entry or identified another location with better signal and speed, thus overcoming this challenge. As a system level intervention, once implemented, HITSystem utilization was nearly universal (<1% refusal rate at pilot sites) for mother-infant pairs enrolling in EID at the facilities. Because both mobile phone use and literacy are high among the target population (between 67% to 100% utilization among mothers at facilities, see Table 2), 67% literacy rate among adult women (12)), we were able to reach the vast majority of HIV+ mothers using the HITSystem’s mobile phone text messaging approach. For mothers without mobile phones, we asked for permission to make contact through their catchment area community health workers (CHW). Patient tracing maps drawn by mothers at enrollment provide CHWs with clear directions from the hospital to their home. For mothers with limited literacy or concerns about disclosure, a pre-identified numeric code was used to request that they return to the hospital.

EFFECTIVENESS

Our team conducted a 12 month pilot of the HITSystem in two Kenyan hospitals, one urban and one peri-urban, comparing data from historical controls to HITSystem participants on key EID outcomes. Data from two distinct settings demonstrated a reduction in turn-around time from sample collection to notifying the mother of the results (median 6.3 vs. 5.0 weeks; urban and 8.1 vs. 3.4 weeks; peri-urban), a significant increase in the proportion of HIV-infected infants initiated on antiretroviral therapy (ART) (14% to 100% urban; 64% to 100% peri-urban), and improved EID retention at 9 postnatal months (45.1% to 93.0% urban; 43.2% to 94.1% peri-urban) (8).

In the absence of historical control data prior to initiation at the other eight implementation sites, we assessed effectiveness by comparing HITSystem outcomes with national averages for the specified EID outcomes. Where national level statistics were not available, we compared HITSystem outcomes to those reported by studies conducted in Kenya during a similar period of time. Table III details hospital specific and combined averages from the ten hospitals for each of the three primary outcomes of EID effectiveness: turn-around time for test results, ART initiation among HIV-infected infants and retention in care demonstrated by 9 month retesting. Historical data documents turn-around time from dried blood spot (DBS) collection to return of HIV DNA PCR test result to the health facilities in Kenya requires an estimated average of 7 weeks (6) compared to an average of 3.3 weeks (SD= 1.7, Range 2.2–7.8) at HITSystem sites. National estimates indicate that only 38% of HIV+ infants/children eligible for ART actually receive it (13). In contrast, 90.7% (118/130, SD=28.8, Range 0–100) of eligible HIV-infected infants (excludes transfers to different facility or infant death prior to notification of result) initiated ART in facilities that implemented the HITSystem. Lastly, retention is one of the most significant challenges for EID services in Kenya, with estimates for 9 month retesting ranging from 19.3% to 45% (6,14). The ease of contact and improved services provided by sites that deployed the HITSystem evidenced 9 month re-testing rates of 86.7% among HIV-exposed infants enrolled in EID (SD=8.6, Range 71.2–100).

Table III.

Effectiveness of the HITSystem on primary EID outcomes: turn-around time for test results, ART initiation among HIV-infected infants, and re-testing at 9 months postnatal.

Hospital	TAT (SD) sample to results	HIV+ infants n (%)	ART initiation among eligible infants1 n/n (%)	Proportion of eligible infants2 re-tested at 9m
1. Referral Hospital A	4.0 (2.3)	20 (2.7)	20/20 (100)	459/502 (91.4)
2. Health Centre B	3.2 (2.3)	39 (7.8)	30/32 (93.7)	260/330 (78.8)
3. County Hospital C	2.2 (1.7)	30 (9.5)	27/29 (90)	218/246 (93.1)
4. County Hospital D	2.9 (3.6)	32 (5.9)	26/28 (92.3)	361/403 (93.7)
5. Sub-County Hospital E	2.2 (1.4)	5 (5.1)	3/3 (100)	37/52 (71.2)
6. Sub-County Hospital F	2.5 (1.6)	6 (4.9)	4/5 (80.0)	68/79 (86.1)
7. Sub-County Hospital G	3.0 (2.6)	5 (5.0)	4/5 (80.0)	56/69 (79.7)
8. Teach & Ref Hospital H	7.8 (4.8)	6 (3.0)	0/4 (0)	41/49 (92.7)
9. Health Center I	2.7 (1.8)	3 (4.7)	3/3 (100)	1/1 (100)
10. Health Center J	2.5 (1.6)	1 (7.1)	1/1 (100)	N/A3
TOTAL	3.3 (1.7)	147 (5.6)	118/130 (90.7)	1501/1731 (86.7)

Note. To safeguard confidentiality, information regarding hospital level and geographic region has been provided rather than hospital name.

TAT= turn-around time

¹Eligible infants exclude those who transferred health facilities or died prior to notification of results.

²Infants retest among those eligible by age, excluding infants previously diagnosed HIV-positive and those discharged early (transferred or relocated to a different facility/region or infant deceased).

³Duration of implementation has not yet been long enough for 9 month re-testing

ADOPTION

As displayed in Table II, the health facilities utilizing the HITSystem represent a range of resource levels, geographic region, and pediatric HIV prevalence between 2.7 to 9.5% (SD=2.0). These facilities are largely representative of the range of government hospitals in Kenya. All ten health facilities approached have adopted or are in the process of adopting the HITSystem. The model of support for adoption varied among sites ranging from a one-day training with follow up support provided from a distance (low), to one-day training plus 2–3 site visits in the first 2–3 months for refresher training and continued remote support (medium), to the routine presence of a HITSystem facilitator at sites where the EID process was less integrated (high), see Table II. Clinical staff at some sites mastered the HITSystem within weeks and displayed very strong ownership of the system, while the adoption curve for staff at other sites ranged from 2.8 to 14.3 months. At the end of this review period, four sites had yet to achieve complete and independent adoption of the system. (Table II) Differences in ownership level and speed of adoption were more a reflection of staff motivation rather than size or resource-level of hospital. The importance of initial computer literacy proved to be far less relevant in adoption than anticipated. This is likely because even novice computer users were able to master the HITSystem technology within one to two months.

As an initial first step in assessing the level of program ownership among clinic staff and time required to fully adopt the HITSystem as the new standard of care for EID, we independently surveyed the four program members most directly involved with HITSystem implementation and alert monitoring at each of the 10 sites. Perceived level of ownership was measured on a 5 point scale (1=minimal ownership, 5= full ownership) taking into account provider interest, consistent enrollment of patients, and number of unresolved action alerts. Similarly, the same clinic staff members were asked to estimate the number of months required at each site before full adoption/integration of the system, including an option of “in progress” for hospitals that had not yet fully owned the system. We report the average of their responses by site in Table II.

IMPLEMENTATION

The fidelity of the HITSystem intervention is maintained across sites because the algorithm for EID care is embedded in the system itself and tailored to established country-specific guidelines. The strategies for patient follow-up were always intended to support existing services, thus this is the area where we observe the most variation. For example, some sites rely largely on text messaging, others utilize a community health care worker to support patient tracing, some prefer to call women directly, and many utilize a combination of strategies. One of the key strengths of the system is its flexibility to adapt to evolving health needs or changing national guidelines For example, recent changes in Kenyan guidelines required the addition of new medication options and additional lab values. Furthermore, EID and laboratory providers have freely offered valuable recommendations to collect additional data or re-word/re-order existing data points to improve the flow and completeness of data entry. Responsive incorporation of end-user recommendations serves to strengthen the system and foster a sense of ownership. Feedback from various stakeholders has also reshaped the vision for the tool’s design, leading to an expansion to address components of PMTCT.

Because the HITSystem is a web-based tool, such changes are instantaneously disseminated to all sites, avoiding the need for slow and expensive technical support at each and every location. This feature has been well received by sites accustomed to using locally-installed systems that require on-site maintenance for any updates or changes. Direct implementation costs were kept low at approximately $350 USD per month per hospital, including mobile broadband minutes for Internet connection and use, support for patient tracing, and a monthly operations fee (includes texting costs and secure data storage) (15). One-time start-up costs per hospital ($100–400 USD) included training, quality assurance, and a computer and modem, if needed. These costs make widespread implementation of the HITSystem both feasible and sustainable, even in low-resource settings. A detailed evaluation of the HITSystem’s cost-effectiveness is ongoing (R01 HD076673).

MAINTENANCE

At the individual level, maintenance is demonstrated by strong EID outcomes (e.g. ART initiation and retention) that persist after multiple years of implementation. As displayed in Table III, hospitals A through D evidence continued strong EID outcomes 9 months after initial training. Data collection is ongoing to assess outcomes over longer time periods. Maintenance at the facility level is achieved when hospitals consistently utilized the HITSystem for EID services and maintain it with limited to no additional support from the implementation team. By November 2013, eight of the 10 hospitals reported on here had been independently implementing the HITSystem for between 12–40 months. One of the greatest challenges to maintenance at the facility level is frequent rotation of health care providers between and among clinical departments. When providers who are well trained and committed to using the HITSystem are transferred, it creates a disruption in utilization and requires increased support. Maintenance of the system’s organization or human infrastructure is a constant process that demands responsiveness to the growth and changes in the areas of PMCTC, EID, information communication technology (ICT), and the evolution of national strategies for HIV-related services that effect patient volume and staff capacity and continuity in the countries in which the HITSystem operates (Kenya, Tanzania and Malawi). The emergence of new tools and devices such as cheaper and faster smart phones has also led to rethinking the role of the operating platform and the development of a HITSystem that can run smoothly on Android and iOS devices. As Kenya moves toward a computer-based health information system supported by the MOH (16), the HITSystem’s programming can be integrated to complement national electronic medical records (EMR) systems, including Open MRS (17) and other complementary ICT programs targeting HIV-related services (18,19).

Limitations

There are limitations to this evaluation of the HITSystem implementation. At this point in time, efficacy data from eight of the ten sites are limited by the lack of matched control data, and thus are compared with the best available national estimates. Given the serial start dates, there is significant variation in implementation period, which complicates the interpretation of data by site. A more rigorously designed ongoing RCT is evaluating the efficacy and effectiveness of the HITSystem. The assessment of program ownership is somewhat subjective; based on direct observation, communication with healthcare providers, and level of additional technical assistance required. We are developing systematic implementation science measures to assess level of adoption, length of time to adoption, and level of programmatic support for future efforts. The HITSystem has successfully reached nearly all mother-infant pairs enrolled in EID services at the targeted health facilities, but an estimated 40% of HIV-exposed infants in Kenya are never enrolled in EID (6). Thus, we recognize the current version of the HITSystem does not address uptake of EID services. We are working to expand the HITSystem to support antenatal prevention of maternal-to-child transmission (PMTCT) in an effort to directly impact pediatric transmission and integrate pre and postnatal services to maximize enrollment and retention among HIV-exposed infants.

Conclusion

This system-level intervention has been feasibly implemented in low-resource settings, reaching nearly all mother-infant pairs enrolling in EID in the targeted government health centers. HITSystem outcomes in Kenya consistently exceed EID outcomes from national reports or other research studies assessing current EID practices and outcomes. A sense of HITSystem ownership among providers was fostered by responsive integration of user feedback, which facilitated adoption at facilities. Lessons learned through the implementation of the HITSystem in Kenya emphasize the need for dynamic and easily adapted eHealth innovations responsive to changing national care guidelines and evolving technology. System maintenance is facilitated by the web-based format, yet requires continual reassessment of hospital and regional support for EID services. The evidence of improved outcomes is encouraging and has led to interest to scale up the HITSystem in a sustainable manner.