Computer-Generated Reminders and Quality of Pediatric HIV Care in a Resource-Limited Setting

Abstract

OBJECTIVES:

To evaluate the impact of clinician-targeted computer-generated reminders on compliance with HIV care guidelines in a resource-limited setting.

METHODS:

We conducted this randomized, controlled trial in an HIV referral clinic in Kenya caring for HIV-infected and HIV-exposed children (<14 years of age). For children randomly assigned to the intervention group, printed patient summaries containing computer-generated patient-specific reminders for overdue care recommendations were provided to the clinician at the time of the child’s clinic visit. For children in the control group, clinicians received the summaries, but no computer-generated reminders. We compared differences between the intervention and control groups in completion of overdue tasks, including HIV testing, laboratory monitoring, initiating antiretroviral therapy, and making referrals.

RESULTS:

During the 5-month study period, 1611 patients (49% female, 70% HIV-infected) were eligible to receive at least 1 computer-generated reminder (ie, had an overdue clinical task). We observed a fourfold increase in the completion of overdue clinical tasks when reminders were availed to providers over the course of the study (68% intervention vs 18% control, P < .001). Orders also occurred earlier for the intervention group (77 days, SD 2.4 days) compared with the control group (104 days, SD 1.2 days) (P < .001). Response rates to reminders varied significantly by type of reminder and between clinicians.

CONCLUSIONS:

Clinician-targeted, computer-generated clinical reminders are associated with a significant increase in completion of overdue clinical tasks for HIV-infected and exposed children in a resource-limited setting.

What’s Known on This Subject:

Of more than 2 million children infected with HIV, almost 90% live in resource-limited settings where pediatric HIV care is often suboptimal. Implementing electronic health records with computerized decision support offers a potential tool for improving care.

What This Study Adds:

This randomized, controlled trial demonstrates that computer-generated clinical reminders can significantly improve clinician compliance with HIV care guidelines for children in a resource-limited setting. This intervention is scalable as developing countries implement electronic health record systems.

The World Health Organization estimates that 2.3 million children live with HIV worldwide,¹ with almost 90% of affected children found in Sub-Saharan Africa.² Few studies document the extent to which HIV care programs in these resource-limited settings are able to meet the pediatric care needs,³ but the limited evidence available suggests that there is significant room for improvement. Care programs report difficulties in establishing systems for early infant diagnosis of HIV, initiating antiretroviral therapy (ART) promptly, ordering baseline and follow-up laboratory investigations, detecting tuberculosis, maintaining prophylaxis for opportunistic infections, and managing child malnutrition.^4–8 These challenges occur at multiple levels, ranging from individual clinical care systems to the country level.

Scalable and evidence-based approaches are needed to improve quality of care for HIV-infected children in resource-limited settings. Many of these settings are implementing electronic health records (EHRs) for this purpose.⁹ However, little rigorous evidence exists to demonstrate whether EHRs will effectively improve HIV care. In resource-rich settings, Clinical Decision Support Systems (CDSS), which use data stored in EHRs to provide care suggestions and reminders to clinicians, generally improve clinician behaviors and quality of care.^10,11 For HIV care in the United States, computer-based alerts and reminders significantly improved clinicians’ adherence to HIV care guidelines for adults.¹² In Kenya, a comparative study between 2 clinics showed that clinical summaries with computer-generated reminders improved clinicians’ compliance with CD4-testing guidelines for adults, but did not evaluate other aspects of HIV care.¹³ Rigorously controlled trials of CDSS’ effectiveness in resource-limited settings could not be found. Whether CDSS might address the particular challenges associated with pediatric HIV care or improve quality of care remains to be seen. In this clinic-level study, set in a resource-limited HIV care system in western Kenya, we used a randomized, controlled design to rigorously evaluate the impact of delivering computer-generated, patient-specific reminders to clinicians on their rates of compliance with pediatric HIV care guidelines.

Methods

Study Site

This study was conducted in a large pediatric HIV clinic at the Moi Teaching and Referral Hospital in Eldoret, Kenya. This clinic is the referral site for the US Agency for International Development (USAID)-funded Academic Model Providing Access to Healthcare (AMPATH) in western Kenya.¹⁴ At the time this study was initiated (February 2011), this pediatric HIV clinic had enrolled 5140 HIV-exposed and HIV-infected children aged 0 to 14 years since its inception in 2003 and had started 1670 HIV-infected children on ART. HIV-infected children on ART are typically seen on a monthly basis in AMPATH, and those not yet meeting criteria to start ART are seen every 3 to 6 months. HIV-exposed children are followed monthly until 18 months of age. Those found to be uninfected are followed every 6 months until their fifth birthday.

Electronic Health Record System

Since 2004, USAID-AMPATH clinics have used the AMPATH Medical Record System (AMRS) to store comprehensive, longitudinal, electronic patient records for all enrolled patients.¹⁵ AMRS is the original implementation of OpenMRS, an open-source EHRs deployed widely in the developing world.^16,17 Patient records in the system contain demographic information, historical and physical examination data, problem lists, medications, diagnostic test results, and visit data. Clinical information is stored as coded concepts (as opposed to free text) for easy retrieval and analysis.¹⁸ Clinicians caring for AMPATH patients do not enter data directly into AMRS but rather complete paper encounter forms (see Supplemental Appendix A). This is largely due to the lack of physical space and technical infrastructure to support computer access in every examination room. Clerks enter data from the completed paper forms into AMRS, with a random 10% subset of the data entered checked for accuracy. The encounter forms are then placed in the patient’s paper clinic chart, which is available to clinicians during patient care. Laboratory results are transferred electronically from the central laboratory information system into AMRS, with a paper copy of the same results placed into the patient’s chart.

Clinical Summaries and Reminders

We developed a module within OpenMRS that generated a patient-specific clinical summary. This summary, tailored for pediatric care, displayed selected information from the patient’s EHR to provide a quick reference to the most relevant data needed by clinicians (Supplemental Appendix B). The module also contained CDSS functionality that appended patient-specific care reminders to the bottom of the clinical summary.¹⁹ New summaries, with or without reminders, were generated for all patients in a portable document format every 4 days as part of a batch-generation process. Summaries could also be generated on demand as needed. For patients with a scheduled visit date, the generated summaries were typically printed by clinic intake personnel on the afternoon before the scheduled visit. Summaries for unscheduled patients were printed when the patient presented at the clinic. All summaries were attached to the relevant patients’ paper charts for clinicians to review during a patient’s clinic visit.

Study Intervention

For this study, we implemented CDSS consisting of care reminders for overdue tests and treatments that were indicated based on standard pediatric HIV care protocols. These protocols were based on recommendations by the World Health Organization²⁰ and the Kenya Ministry of Health.²¹ We implemented reminders to order overdue 6-week HIV DNA polymerase chain reaction tests, 18-month enzyme-linked immunosorbent assay (ELISA) antibody tests, CD4 tests, routine laboratory studies, and chest radiographs; to initiate ART; and to refer malnourished children for nutritional evaluation and assistance (Table 1).

TABLE 1.

Implemented Pediatric Reminders

Indication for Care Suggestion		Care Suggestion Generated
PCR tests	- No baseline PCR for HIV-exposed child, aged 6 wk to 18 mo.	- Please order DNA PCR. Patient between 6 wks and 18 mo with no DNA PCR
	- Repeat PCR needed. Child has one +ve PCR, aged 6 wk to 18 mo.	- Please order DNA PCR. Patient between 6 wks and 18 mo with only 1 DNA PCR.
CD4 tests	- No baseline CD4 count & percent for HIV exposed child with +ve 6-wk DNA PCR test or +ve 18-mo ELISA	- Please order CD4 panel. +ve PCR or ELISA but no CD4 test.
	- No repeat CD4 count and percent in 6 mo for HIV-exposed child with +ve 6-wk DNA PCR test and/or 18-mo ELISA	- Please order CD4 panel. +ve PCR or ELISA but no CD4 for > 6 mo.
Baseline chemistry & hematology studies	Child with +ve 6-wk DNA PCR test or +ve 18-mo ELISA result, with the following baseline tests not ordered:
	- SGPT (liver test)	- Please order SGPT. +ve PCR or ELISA but no SGPT.
	- Creatinine (kidney test)	- Please order creatinine. +ve PCR or ELISA but no creatinine.
	- Hemogram test	- Please order hemogram. +ve PCR or ELISA but no hemogram.
Baseline chest x-ray	Child with +ve 6-wk DNA PCR test or +ve 18-mo ELISA result, with no baseline CXR result or order.	- Please order chest x-ray. +ve PCR or ELISA but no x-ray.
ARV reminders	Child 18 mo to 5 y old, with +ve 6-wk DNA PCR test or +ve 18-mo ELISA result, and CD4% <25% on no antiretroviral medications.	Consider starting ARV medications. Patient 18 mo to 5 y with positive || PCR or ELISA || AND || CD4% <25a
	Child >5 y, with +ve 6-wk DNA PCR test or +ve 18-mo ELISA result, and CD4% <20% or CD4 count <500, on no antiretroviral medication.	Consider starting ARV medications. Patient >5 y with positive ELISA AND CD4% <25.
	Child is <18 mo of age, and has 1 positive DNA PCR test, on antiretroviral medication.	Consider starting ARV medications. Patient with positive HIV PCR.
Malnutrition reminders	Child’s z score is < −3	Hospitalize for malnutrition. Last Z score in AMRS −3 or less.
	Child’s z score is < −1.5 but > −3	Refer for nutrition support. Last Z score in AMRS −1.5 or less.

ARV, antiretroviral; PCR, polymerase chain reaction; SGPT, alanine aminotransferase.

^aReminder only shows items where the criteria are met for PCR and/or ELISA and CD4% <25.

Randomization and Study Population

We assessed the effect of the patient-specific, computer-generated pediatric reminders on compliance with pediatric HIV care guidelines by using a randomized, controlled trial. All patients, both HIV-exposed and HIV-infected, previously enrolled at the study site were randomly assigned to either the intervention or control group in a 1:1 ratio by using a 4-block randomization scheme. Study patients were HIV-infected or HIV-exposed children <14 years of age presenting for return clinical visits at the outpatient HIV clinic. New patients enrolling in the clinic during the 5-month evaluation were excluded from the study. Patients stayed in the same study group throughout the 5-month duration of the study. We randomized by patient instead of by clinician, because patients typically saw whichever clinician was first available at the time of their visit, and it was not possible to tell in advance which patient a clinician would see. We understood that this could sensitize clinicians to order the indicated care for control patients, which might bias our study against finding a significant effect for the reminders.

Study Implementation

If an intervention patient was overdue for 1 or more of the care interventions described above, the reminder to complete that overdue task was printed at the bottom of the patient’s paper clinical summary when they presented to the clinic (Supplemental Appendix B). An example of a printed reminder was: “Please order Chest X-ray. +ve polymerase chain reaction or ELISA but no X-ray.” Clinicians were asked to document their response to each reminder within the clinical summary by selecting among 7 responses, namely: (1) ordered today; (2) not applicable (explain); (3) previously ordered; (4) patient allergic; (5) patient refused; (6) I disagree with reminder (explain); or (7) other (explain). Each type of reminder was assigned a unique number that appeared at the top of the printed patient clinical summary (Supplemental Appendix B: in this example, the number assigned for “order chest x-ray” reminder type is “6”). No more than 5 reminders were displayed for each patient per visit, so as not to overwhelm clinicians. Reminders were displayed in numerical order, with no preference or prioritization to any particular reminder. For intervention patients, if the recommended action was not taken, the reminder for that task would again be printed on the clinical summary at the next visit. Reminders were repeated at subsequent visits as long as the action remained overdue. For control patients, reminders were not printed on the paper summaries when the patients had overdue orders at any of the visits.

For both intervention and control patients, clinicians recorded actions taken as part of routine care (including test and treatment orders) on the paper encounter forms (Supplemental Appendix A: Sections 24–26). All clinical summaries and encounter forms were subsequently reviewed by 2 research assistants who extracted information on reminder type, study group, clinician, completion of targeted overdue tasks, and documented reasons in the clinical summaries when care suggestions were not implemented. Completion of overdue tasks was considered to have occurred if there was documentation of ordering an investigation, medication, or referral either within the patient’s paper chart or within the EHR. We also queried the EHRs 3 months after study closure (October 7, 2011) to determine if any other relevant results had returned.

Before our study, clinicians had received paper clinical summaries without any reminders on patients’ charts for ∼1 year. In the first 2 weeks of the training period (February 1–13, 2011), only reminder tracking numbers were printed on the top of each clinical summary for all patients. Clinicians were informed that these numbers were for internal use by the clinical decision support team. During the next 2 weeks (February 14–28, 2011), reminders were added to printed summaries, and clinicians were trained on how to respond to them in accordance with pediatric care protocols. Random selection of patients was done on February 28, 2011, and the study ran until August 8, 2011. The study was approved by the institutional review boards at Indiana University School of Medicine in Indianapolis, Indiana, and by the Institutional Review and Ethics Committee at Moi University School of Medicine, Kenya. The review boards did not require informed consent from patients or providers, because reminders have been considered routine components of care in the HIV clinics at AMPATH.

Statistical Methods

Continuous baseline variables are summarized by mean and SD, with age summarized by using median and interquartile range. Categorical variables are summarized by frequency and percentage. Student t tests were used to compare means for continuous variables. χ² and Fisher exact tests were used to compare distributions for categorical variables. Demographic data used were obtained as part of the routine procedures for clinic registration and patient care.

The primary outcome of interest was the number of visits (inclusive) before a recommended action had been fulfilled and documented by a provider in the patient’s chart. Toward this goal, the analysis unit was the response to a specific recommended action at a particular visit, which was dichotomized as “Ordered” or “Not Ordered” at each visit. A care suggestion was considered ordered only when explicit documentation of the relevant order existed within the patient’s clinical encounter form, or when the requisite laboratory result was found within the EHR. We used a discrete survival model to test for differences in parameters that govern the cumulative rate of correcting overdue tasks for control and intervention groups (see Supplemental Appendix C for details of analysis). This was done for each type of reminder and for all reminders combined. Therefore, the primary hypothesis was tested through tests of equality of the model parameters between intervention and control groups. In our model, we included health care provider as a random effect.

All analyses were performed by using SAS 9.3 (SAS Institute, Cary, NC). A 2-sided P value of .05 was considered statistically significant. Bonferroni correction of the P values was made to adjust for multiple comparisons for the primary end point.

Results

Patient and Provider Demographics

A total of 3993 HIV-positive or -exposed children were randomly assigned to the study. Of these, 1619 patients (41%) were eligible to receive at least 1 computer-generated reminder (ie, had an overdue order or referral) during the study period (Table 2). Eight of the 1619 were subsequently excluded from the analysis because a clinician involved in the study saw them. Median age was 6.17 years, with an interquartile range of 7.91 (p25 = 2.67, p75 = 10.58). Seventy-one percent of the eligible children were HIV-infected, and 784 (49%) were on ART. There were no statistically significant differences in the demographic characteristics between the control and intervention groups. A total of 1142 children (609 control and 533 intervention) were seen more than once during the study period.

TABLE 2.

Patient Characteristics

	Control (n = 786)	Intervention (n = 825)	P
Age, y	7.07 (5.04)	6.88 (4.70)	.4318
Gender
Female	379 (48.2)	416 (50.4)	.3962
Male	407 (51.8)	409 (49.6)
Mother deceased
No	608 (77.5)	650 (78.9)	.7853
Yes	173 (22.0)	170 (20.6)
Unknown	4 (0.5)	4 (0.5)
Father deceased
No	584 (74.4)	615 (74.6)	.4826
Yes	185 (23.6)	185 (22.5)
Unknown	16 (2.0)	24 (2.9)
HIV status (at enrollment)
Exposed	113 (14.5)	117 (14.2)	.0841
Negative	99 (12.7)	139 (16.9)
Positive	570 (72.9)	569 (69.0)
ARTa
No	293 (42.7)	296 (43.2)	.8701
Yes	394 (57.4)	390 (56.9)
WHO stageb
1	113 (20.0)	111 (19.7)	.7200
2	114 (20.1)	129 (22.9)
3	295 (52.1)	280 (49.7)
4	44 (7.8)	43 (7.6)

Values are frequency (percent) for categorical variables and means (SDs) for continuous variables; P values are from χ² tests for categorical variables and Student t test for continuous variables. ARV, antiretroviral; WHO, World Health Organization.

^aHIV-negative children were excluded in this analysis.

^bHIV-negative and HIV-exposed children were excluded from this analysis.

Thirty clinicians saw the study patients during clinic visits, and, of these, 22 provided demographic information. The clinicians were primarily clinical officers (equivalent to physician assistants) (71%), but 1 attending physician (5%) and 5 nurses (24%) also provided care. The clinicians were mostly women (68%) and had worked with AMPATH for a mean of 52 months (SD 34) and at the study clinical site for a mean of 40 months (SD 34). Clinicians spent a mean of 4.4 days per week in the study site (SD 1.2).

Response to Care Reminders

In the intervention group, 825 patients (43%) had at least 1 unmet care protocol, resulting in computer generation and delivery of 2153 unique care reminders on their clinical summaries. In the control group, 786 patients (40%) had an unmet care protocol, generating 2843 unique reminders, but these were not printed on the clinical summaries. Table 3 shows the cumulative rates for correcting overdue clinical tasks, by visit, for control and intervention group patients. The intervention group had significantly higher rates of completing overdue clinical tasks at the first visit in the study period and across the subsequent visits. Over the course of the study, there was a fourfold increase in completing overdue clinical tasks when the patient with an overdue task had a reminder printed on their clinical summary sheet (68% completion in intervention group vs 18% in control group, P < .001) (Table 3).

TABLE 3.

Cumulative Rates of Correcting Overdue Clinical Care Tasks at Each Visit for Control and Intervention Groups

Targeted Clinical Care Tasks		1st visit	2nd visit	3rd visit	4th visit	5th visit	6th visit
Order chest x-ray	Control (n = 1236)	0.02	0.04	0.05	0.06	0.08	0.09
(P < .001)	Intervention (n = 843)	0.25	0.39	0.47	0.53	0.58	0.63
Order 18-mo HIV ELISA	Control (n = 739)	0.09	0.12	0.13	0.13	0.13	0.14
(P < .001)	Intervention (n = 553)	0.20	0.28	0.32	0.35	0.38	0.40
Order other laboratory testsa	Control (n = 318)	0.18	0.22	0.26	0.30	0.34	0.37
(P < .001)	Intervention (n = 147)	0.27	0.56	0.62	0.67	0.72	0.76
Begin ART	Control (n = 192)	0.09	0.21	0.34	0.45	0.55	0.62
(P < .28)	Intervention (n = 247)	0.07	0.29	0.48	0.62	0.72	0.79
Refer to nutritional support	Control (n = 358)	0.10	0.14	0.18	0.21	0.24	0.27
(P < .001)	Intervention (n = 363)	0.31	0.48	0.65	0.77	0.84	0.89
All overdue care tasks	Control (n = 2843)	0.07	0.11	0.13	0.14	0.16	0.18
(P < .001)	Intervention (n = 2153)	0.22	0.38	0.48	0.56	0.63	0.68

Notes: (1) Values indicate the cumulative rates for overdue clinical tasks were corrected, by visit, for control and intervention group patients. The outcomes were considered significant at the .0083 level, instead of at the .05 level, to reflect the Bonferroni correction. (2) The proportions reported in Table 3 are adjusted for correlation within the same care provider and patient. See Supplemental Appendix C for details of analysis. (3) The numbers (n) reported in Table 3 represent the total empirical numbers of patient-reminder pairs at the end of the study.

^aOther laboratory tests include HIV DNA polymerase chain reaction, CD4 count, chemistries, and blood count.

Overdue clinical tasks were also corrected earlier when reminders were presented to clinicians. For the intervention group, the mean time from when a patient had an overdue clinical task to the completion of that task was 77 days (SE 2.4 days), whereas the mean time to correction was 104 days (SE 1.2 days) for the control group (P < .001 based on test for equality over strata) (Fig 1).

FIGURE 1.

Proportion of overdue tasks completed as a function of time.

The impact of reminders varied by type of care suggestion (Table 3). Rates of correcting overdue clinical tasks were significantly greater for the intervention group compared with the control group in response to reminders for ordering HIV ELISA tests (P < .001); chest radiographs (P < .001); other tests including chemistries and blood counts (P < .001); and for making referrals for malnutrition (P < .001). Rates of initiating ART for qualifying children were higher in the intervention group than in the control group, but this difference did not reach statistical significance (79% vs 62%, P = .28).

Response rates to clinical reminders also varied by clinician. Of the 30 clinicians who saw patients during the study period, 8 clinical officers saw >50 reminders for intervention patients. The cutoff of 50 reminders was chosen to identify providers who had enough exposure and familiarity with the reminder system beyond the learning phase. Among these 8 clinical officers, rates of correcting overdue clinical tasks in response to a reminder ranged from 11% to 32% per encounter. Seven of the 8 clinical officers had significantly higher rates of correcting overdue clinical tasks when reminders were presented in comparison with when there were no reminders (P < .001). The 1 clinician who did not have a significant increase still had higher rates of correction per encounter with presentation of reminders (4.6%–11.2%, P = .0621) (Table 4).

TABLE 4.

Rates of Completing Overdue Tasks During an Encounter by Clinicians Who Saw >50 Reminders in the Study Period

Clinician	Control	Intervention	Pa
1	6.4% (32/500)	28.9% (120/415)	<.0001
2	1.7% (3/176)	14.2% (23/162)	<.0001
3	5.2% (30/577)	32.4%(145/448)	<.0001
4	4.6% (6/130)	11.2% (14/125)	.0621
5	2.3% (7/304)	12.2% (22/180)	<.0001
6	4.3% (9/209)	22.9% (36/157)	<.0001
7	6.0% (22/367)	18.8% (46/245)	<.0001
8	7.4% (33/446)	21.8% (76/349)	<.0001

^aP values are from the Fisher exact test.

When reminders did not result in performing the suggested task during a visit for an intervention patient, the following reasons were provided by the clinicians: test was previously ordered (28%); clinician planned to order at next visit, patient refused, or other deferral not specified (25%); or provider disagreed with reminder or considered it not applicable (13%). In 13% of encounters where a reminder for an intervention patient did not result in correction of the overdue task, no reason was provided as to why the reminder was not followed.

Discussion

In this randomized, controlled trial of CDSS to improve pediatric HIV care in a resource-limited setting, we found that including patient-specific, computer-generated reminders on a paper-based clinical summary significantly improved adherence to pediatric HIV care protocols. Presenting the clinicians with these reminders also significantly decreased the time to correction of overdue clinical tasks, with the observed improvements seen throughout the duration of the study.

These findings could have particular clinical importance for the pediatric HIV population, in which early mortality and morbidity are high. Early diagnosis of HIV and prompt initiation of ART can reduce children’s high mortality by as much as 76%,²² but our findings suggest that many of the recommendations for pediatric HIV care are not performed in a timely fashion. Clinicians may have difficulty adhering to care protocols because they care for large numbers of patients, they might not recall guideline recommendations, or they cannot readily determine whether a given patient needs a particular test or therapy. A CDSS intervention that addresses these challenges and significantly improves the quality of pediatric HIV care could yield real clinical improvements.

Despite the improvements in care practices for the intervention group, the response rates did vary by type of reminder. This suggests that clinicians found some reminders more relevant or critical. In addition, reminders never resulted in perfect compliance with care protocols for reasons similar to those found in other settings.²³ The recommended orders were sometimes appropriately deferred because they were not applicable, the EHR data generating the reminder was deficient, or because other patient factors made it appropriate to wait. Allowing clinicians to give reasons for not following the reminder reveals whether the continued noncompliance with protocols is appropriate. Although reminders did not result in perfect adherence with care protocols, significant improvements in care practices were seen across a variety of reminder types, ranging from test ordering to referral patterns. The findings suggest that CDSS can have broad applicability across multiple care scenarios.

This study underscores that completeness and quality of data within an EHR are crucial for the successful implementation of CDSS. Without access to complete and high-quality data, CDSS cannot offer relevant, timely, and accurate reminders. For example, in 28% of the clinical encounters where clinicians were given a reminder for an overdue task but did not complete this task at that visit, clinicians reported that the test, medication, or referral had previously been ordered. In these cases, the relevant information was likely not reflected in the EHR, suggesting a breakdown in the system connecting the clinician, patient, laboratory, and EHR. Such data quality issues deserve additional attention, because clinicians can become indifferent to reminders that are consistently inaccurate.²⁴

This study has some limitations. It was conducted in a single HIV clinic with a limited number of clinicians, potentially limiting its generalizability. Nonetheless, these constraints enabled conduct of a rigorous, randomized, controlled trial to provide high-quality evidence. In addition, the study included a large sample of 1611 children, and this may improve its applicability to other resource-limited settings. Another limitation is that the study was conducted in a setting where patients sometimes saw different clinicians on different visits, and the study findings might not necessarily translate to institutions where clinicians routinely follow up the same patients. It should also be emphasized that computer-generated reminders are only as good as the rules programmed into the computer. Although some health care systems may not have a robust EHR in place, the OpenMRS system used at the study site has been widely implemented in over 20 African countries and identified for broad rollout across countries like Kenya and Rwanda. The widespread and expanding use of OpenMRS means that our findings will be directly translatable to a large number of sites. In fact, our CDSS system has now been rolled out to all 51 AMPATH clinical sites. We hope to evaluate the impact of CDSS on clinical outcomes in the future.

Conclusions

This randomized, controlled trial provides robust evidence that patient-specific, computer-generated clinical reminders are associated with improved completion of overdue tasks relevant to pediatric HIV care in a resource-limited setting. This technology could be widely applied to improve compliance with care protocols in other resource-limited settings.

Glossary

AMPATH

Academic Model Providing Access to Healthcare

AMRS

AMPATH Medical Record System

ART

antiretroviral therapy

CDSS

clinical decision support systems

EHR

electronic health record

ELISA

enzyme-linked immunosorbent assay

USAID

US Agency for International Development

+ve

positive