Evaluation of the effectiveness of an outreach clinical mentoring programme in support of paediatric HIV care scale-up in Botswana

Abstract

Clinical mentoring by providers skilled in HIV management has been identified as a cornerstone of scaling-up antiretroviral treatment in Africa, particularly in settings where expertise is limited. However, little data exist on its effectiveness and impact on improving the quality-of-care and clinical outcomes, especially for HIV-infected children. Since 2008, the Botswana-Baylor Children’s Clinical Centre of Excellence (COE) has operated an outreach mentoring programme at clinical sites around Botswana. This study is a retrospective review of 374 paediatric charts at four outreach mentoring sites (Mochudi, Phutadikobo, Molepolole, Thamaga) evaluating the effectiveness of the programme as reflected in a number of clinically-relevant areas. Charts from one visit prior to initiation of mentoring and from one visit after approximately one year of mentoring were assessed for statistically-significant differences (p<0.05) in the documentation of clinically-relevant indicators. Mochudi showed notable improvements in all indicators analyzed, with particular improvements in documentation of pill count, viral load (VL) results, correct laboratory monitoring and correct antiretroviral therapy (ART) dosing (p<0.0001, p<0.0001, p<0.0001 and p<0.0001, respectively). Broad and substantial improvements were also seen in Molepolole, with the most improvement in disclosure documentation of all four sites. At Thamaga, improvements were restricted to CD4 documentation(p<0.001), recent VL and documented pill count (p<0.05 and p<0.05, respectively). Phuthadikobo showed the least amount of improvement across indicators, with only VL documentation and correct ART dosing showing statistically-significant improvements (p<0.05 and p<0.0001, respectively). These findings suggest that clinical mentoring may assist improvements in a number of important areas, including ART dosing and monitoring; adherence assessment and assurance; and disclosure. Clinical mentoring may be a valuable tool in scale-up of quality paediatric HIV care-and-treatment outside specialized centres. Further study will help refine approaches to clinical mentoring, including assuring mentoring translates into improved clinical outcomes for HIV-infected children.

Introduction

In almost every setting globally, children remain underrepresented in HIV care-and-treatment programmes compared to adults, posing a distinct challenge to the global goal of universal access to antiretroviral therapy (ART) (Prendergast, Tudor-Williams, Jeena, Burchett & Goulder, 2007; World Health Organization [WHO], 2010). This is particularly true in resource-limited settings, where comprehensive paediatric HIV/AIDS care and treatment tends to be centralized in larger urban areas and dedicated paediatric treatment centres (Fredlund & Nash, 2007; Sutcliffe et al., 2010; WHO, 2010). In addition to challenges regarding access to care, there are concerns over the quality of comprehensive care delivered as programmes scale-up (Philips, Zachariah & Venis, 2008).

Quality paediatric HIV care and treatment relies strongly on proper prescription of cotrimoxazole and ART, as well as regular and comprehensive monitoring of parameters indicative of treatment success, including growth and development; response of CD4 count and viral load (VL) to treatment; and attention to psychosocial aspects of ART provision, such as adherence to therapy and awareness of disease status (disclosure), upon which successful clinical outcomes substantially depend (Ahoua et al., 2011; Fetzer et al., 2011; Gray, 2009; Palombi et al., 2009; Staehelin et al., 2012; Vreeman et al., 2009; Vreeman et al., 2010).

Along with training and supervision, clinical mentoring by providers skilled in HIV management has been identified as a cornerstone of scaling-up antiretroviral treatment in Africa (Zolfo et al., 2010), including for children, and is part of many national scale-up approaches(Cameron et al., 2012; Rustein et al.,1998; Morris et al., 2009; Scherzer et al., 2010; Tolleet al.,2008). While evidence of a direct effect of clinical mentoring on clinical outcomes is lacking, there is ample evidence for improved HIV outcomes in cohorts of patients managed by specialized or more experienced providers compared to those with less HIV-specific training and experience (Landon et al., 2003; Stone, Mansourati, Poses & Mayer, 2001; Wood et al., 2003), including for nurse-managed cohorts (Callaghan, Ford & Schneider, 2010; Sanne et al., 2010). There is great interest, then, in the evaluation of clinical mentoring approaches that support the development of skills and expertise in mentees with respect to paediatric HIV management in Africa.

Botswana has one of the world’s highest HIV prevalence rates – an estimated 25.0% amongst adults ages 15–49 (Joint United Nations Program on HIV/AIDS [UNAIDS], 2010) and a proportionally large number of HIV-infected children (19,000 ages 0–14yrs), with nearly 9,000 children on ART (Botswana Ministry of Health [MOH], 2011). Since the launch of Botswana’s national ART programme in 2002, clinical mentorship has been a key component of the national approach to scale-up of HIV care and treatment services. There has been a focus on skills-and confidence-building amongst new ART providers, targeting expansion of high-quality care beyond specialized centres in urban areas to district hospitals and primary health centres across Botswana where HIV outcomes such as mortality rate and rate of viral suppression are less optimal, particularly for children (Botswana MOH, 2011).

This pilot study reports the first data from Botswana on the effectiveness of a clinical mentoring programme at decentralized ART sites dedicated to promoting the scale-up of quality paediatric HIV care and treatment.

Methods

A principal partner to the Government of Botswana in paediatric HIV care and treatment, training and clinical mentoring(Kline, 2006), the Botswana-Baylor Children’s Clinical Centre of Excellence (COE) cares for over 2,000 HIV-infected children, more than 1,800of whom receive ART (Kirk et al., 2010). Since April 2008, the COE has operated an outreach programme in support of Botswana’s efforts to scale-up paediatric HIV care, in which COE physicians and nurses visit ART sites outside Gaborone (Workneh et al., 2010), including at primary health centre level (Scherzer et al., 2010). Sites are visited monthly by either a paediatrician or a medical officer experienced in provision of comprehensive paediatric HIV care, as well as an experienced nurse mentor, and focus on supporting Botswana’s National ART Guidelines, which are followed by all public health facilities, including the COE, and delineate standard ART regimens and protocols for the treatment and monitoring of HIV-infected patients.

During the visits, the COE team conducts side-by-side mentoring of medical officers, nurse prescribers, and nurses from the local clinic as they see patients. In addition, didactic sessions that are structured around the fundamentals of paediatric HIV care and treatment are offered. Since its inception, the project has grown to sustain 21 sites across Botswana, with an average of more than 250 mentored patient visits monthly, and a total outreach site paediatric enrollment of more than 4,000 HIV-infected children (Workneh et al., 2010).

To measure effectiveness of the COE’s clinical mentoring programme, we used the metric of comparing documentation of several clinical aspects of a comprehensive paediatric HIV visit relevant to quality paediatric HIV care and treatment, pre-and post-mentoring.

A retrospective chart review was conducted by three COE paediatricians of 374 charts at four of these outreach mentoring sites: Molepolole, Mochudi, Phuthadikobo, and Thamaga [Figure 1]. The sites were selected based on length of mentoring (all selected sites had received structured mentoring since at least April 2008) and size of the paediatric ART population (sites had to have at least 39 active paediatric ART patients per sample size calculations). A sample size of 39 charts from one outreach site will have 80% power to detect a difference in proportions of 0.350 when the proportion of discordant pairs is expected to be 0.650 and the method of analysis is a McNemar’s test of equality of paired proportions with a 0.05 two-sided significance level. Based on this sample size calculation, it was attempted to review a minimum of 39 paediatric charts at each site (this number was exceeded at all sites studied). Relevant site characteristics are detailed in Table 1.

Figure 1.

Map of southeastern Botswana: Baylor Centre and Outreach sites studied

- Botswana-Baylor Children’s Clinical Centre of Excellence

- Outreach sites studied

Table 1.

Characteristics of outreach sites

Characteristics of Sites	Mochudi	Phuthadikobo	Thamaga	Moleps
Number of patients
Total	3000	2500	2000	6000
Pediatric	330	60	100	500
Staff * A set of physician rotates through the clinic on a weekly basis ** A set of physicians rotates through the clinic on a daily basis
Dedicated MO (MD)	8*	6**	5**	1
Dedicated NP	3	N/A	3	5
Dedicated general nurses	7	N/A	5	4
KITSO Trained staff A national week-long health care provider training course on paediatric HIV
Medical officers	8*	N/A		1
Nurse prescribers	3	N/A		5
Registered nurses	7	N/A		4
Laboratory capabilities Full: CD4/VL/other done at the facility Partial: VL sent to Gaborone to be handled by a specialized lab (Harvard Lab) Limited: Both VL and CD4 sent to Gaborone to for processing in a specialized lab (Harvard Lab) None: No labs at facility, specimen sent off- site	Patial	None	Limited	Full
Facility District Hospital: serves the general population including patients on ARVs a district in Botswana (population size…) Primary hospital: serves the general population including patients on ARVs in the surrounding villages (appx population size.. ) Free standing local clinic: only serves patient on ARVs.	District	Free standing clinic	Primary	District

At each site a list of all active paediatric patients 18 years and younger was obtained from medical records staff. To be included in the study, patients had to be registered before April 2008 and remain registered one year later (April 2009). Those patients whose charts could not be found, those enrolled at the site after April 2008, those transferred out of that site prior to April 2009, and those who were no longer in care at the time of review were excluded. For each patient two visits were assessed. The pre-and post-mentoring data were collected for analysis only if the pre-mentoring visit occurred no more than two months before April 2008 and the post-mentoring visit occurred no more than 2 months after April 2009.

The paper-based charts at each site were reviewed by one of three paediatric HIV specialists, all with more than one year of experience in clinical mentoring at ART sites around Botswana. Each visit entry was assessed for completion of the following clinical indicators relevant to quality paediatric HIV care and treatment:weight, height, plotting of weight and height, whether visit was done by a physician or a nurse prescriber, pill count, assessment of adherence to ART, physical exam, disclosure, recent CD4, recent viral load (VL), whether VL was detectable and if so if it was handled correctly, correct lab monitoring, appropriate use of cotrimoxazole prophylaxis, and correct ART dosing. Completion was assessed by examining if the clinical indicators were written in the chart notes, or, in the case of laboratory outcomes, if the output results were present in the chart, thereby accounting for providers who may have looked at results, but not written them into the patient note. For a given encounter, the presence of each indicator was given a “yes,” “no,” “unknown,” or “not applicable.” “Unknown” was used for things that could not be determined from the chart -i.e. whether a patient was seen by a physician or a nurse prescriber or determining if a patient’s VL was detectable when there was no VL on record. “Not applicable” was used when the indicator did not apply to the patient – i.e. appropriate handling of VL was not applicable to patients who did not have a detectable VL.

For the purpose of our study, “disclosure” refers to documentation of where in the disclosure process the child was. The COE has created a disclosure flipchart, which providers use to teach a child the reasons for taking their medications. Beginning around age 5, children are taught that they take their medication to stay healthy and protect their “soldiers” (CD4 cells) from the “bad guy” (HIV). As they age, they are taught more about their disease until they are ready for full disclosure (naming the virus). This disclosure process is taught to the local clinics as part of the clinical mentoring programme and providers are asked to document what stage of the process the child is in (e.g., “knows about soldiers”).

Data was analyzed using McNemar’s test to identify statistically significant differences in documentation of these indicators between pre-and post –mentoring discordant pairs. Charts with an “unknown” or “not applicable” response for a given indicator were omitted from the analysis. Data was analyzed using Stata 10.1 Special Edition software (StataCorp; 2009. College Station, TX, USA) and sample size was calculated using nQuery Advisor^® 7.0 (Statistical Solutions Ltd., Cork, Ireland).

Ethical Approval

This study was approved by the Health Research and Development Committee (HRDC), Ministry of Health, Botswana, and the Institutional Review Board, Baylor College of Medicine, USA.

Results

A total of 374 patient records were assessed. Only six of the clinical indicators had enough discordant pairs and sufficient documentation to make a valid comparison: pill count, correct lab monitoring, correct ART dosing, CD4 count within the last 7 months, VL within the last 4 months, and disclosure. Botswana guidelines recommend monitoring CD4 counts every 6 months in stable paediatric patients. We allowed an additional month for the common occurrence that 6 month follow-up visits may be scheduled chronologically just beyond 6 months. Botswana guidelines recommend monitoring the VL every 3 months in stable paediatric patients. Thus, a similar allowance for visit scheduling as described for CD4 documentation was also applied to VL documentation. The remaining indicators were excluded from the analysis. Table 2 outlines results for the indicators by site as the percentage of all charts that had proper documentation pre-and post-mentoring.

Table 2.

Proportion of records at each outreach site with documentation of clinically-relevant indicators

Site	Pill Count	Disclosure	Viral Load	CD4	Correct lab Monitoring	Correct ARV Dosing
Mochudi (N=156)	9% / 77%**	2% / 10%*	19% / 77%**	87% / 96%*	12% / 47%**	54% / 92%**
Phuthadikobo (N=41)	0% / 0%	6% / 3%	22% / 49%*	78% / 73%	13% / 23%	35% / 93%**
Molepolole (N=126)	3% / 26%**	5% / 21%*	50% / 66%*	83%/ 83%	20% / 48%**	45% / 92%**
Thamaga (N=51)	0% / 25%*	0% / 7%	51% / 77%*	53% / 96%**	24% / 29%	83% / 85%

Pre / Post-mentoring documentation of indicators -% of discordant charts with successful documentation

^*p<0.05;

^**p<0.0001

Mochudi showed notable, statistically-significant improvements in all indicators analyzed, with particular improvements in documentation of pill count, viral load results, correct laboratory monitoring and correct ARV dosing (p<0.0001, p<0.0001, p<0.0001 and p<0.0001, respectively).

Broad and substantial improvements were also seen in Molepolole, with the most improvement in disclosure documentation of all four sites (p=0.0004 versus p=0.0075 at Mochudi and p>0.05 at both Phutadikobo and Thamaga). Pill count, VL, correct lab monitoring, and correct ARV dosing also showed statistically significant results (p<0.0001, p<0.05, p<0.0001 and p<0.0001, respectively).

Statistically significant differences were less in Thamaga, with only CD4 documentation (p<0.001), recent VL, and documented pill count (p<0.05 and p<0.05, respectively) showing improvements after mentoring.

Among the four sites, Phuthadikobo showed the least amount of improvement across indicators, with only VL documentation and correct ARV dosing showing statistically significant improvements (p<0.05 and p<0.0001, respectively).

Discussion

This study demonstrates improvement in the completion, by mentored health care providers, of several clinically-relevant indicators of comprehensive paediatric HIV care and treatment after the initial year of specialized clinical mentoring at four Botswana Ministry of Health ART sites outside Gaborone. While this pilot study demonstrates changes in practice pre-and post-mentoring at mentored sites, it was not designed to explore the relationship between improved documentation of care delivered and the quality of that care or the correlation of documentation improvements with patient clinical outcomes. Follow-up studies along these lines hold great interest and are planned.

Nonetheless, this study suggests important conclusions, which add to the very limited current body of knowledge of clinical mentoring’s impact on paediatric HIV care in Africa. As the chief source of information about both the process and outcomes of clinical care, the medical record may be used as a guide to assessing quality of care (Donabedian, 1988), with process measures or clinical indicators and their documentation predicting quality of care and service delivery (English et al., 2007; Eriksen et al., 2007; Mwakyusa, 2006; Nemes et al., 2009). Data from a Kenyan study of a structured paediatric admission record’s implementation supports this notion, concluding that helping providers improve documentation proved an important step in improving the care of children, with substantial improvements in the evaluation of patients by providers noted after the record’s implementation (Mwakyusa et al., 2006).

The indicators in our study for which there were valid pre-and post-mentoring comparisons (Table 2) are all associated with important aspects of paediatric ART management. Correct ART dosing and correct laboratory monitoring are essential skills in managing ART, allowing longitudinal assessment of treatment efficacy and toxicity. Pill counts are a standard method of assessing a child’s adherence to ART. Associated with virologic suppression, immunologic recovery and reduced mortality risk, adherence to ART is the foundation of treatment success (Ahoua et al., 2011; Palombi et al., 2009; Staehelin et al., 2012); particularly for children, early detection and correction of non-adherence is critical to good ART outcomes (Vreeman, 2008). As programme data in Botswana show substantially higher rates of paediatric ART failure at sites outside the COE (Botswana MOH, 2011), developing robust capacity at sites in the detection of early non-adherence is of high priority.

The importance of adherence to treatment success highlights the issue of disclosure, an important focus of our mentoring. Disclosure as a gradual and continuous process over multiple visits for children – rather than as a single event - is well supported (Brown et al., 2011; Butler, 2009; Haberer, 2011; Myer, Moodley, Hendricks & Cotton, 2006; Vaz, 2010; Vreeman et al., 2010), with clinicians playing a crucial role in the process, assisting caregivers in “partial” disclosure by helping the child’s understanding to gradually increase (Klitzman, Marhefka, Mellins & Wiener, 2008). While fear of stigmatization and discrimination make disclosure a challenging aspect of paediatric HIV management for families and caregivers alike (Gray, 2009), children who are aware of their HIV status experience elevated self-esteem, enhanced willingness to accept treatment, better ART adherence and less frustration with treatment than non-disclosed children (Fetzer et al., 2011; Gray, 2009; Lester, 2002; Mellins, Brackis-Cott, Dolezal & Abrams,2004; Vreeman et al., 2009; Vreeman et al., 2010). Disclosure issues are heightened for adolescents, for whom disclosure concerns can pose substantial barriers to psychosocial functioning and ultimate transition to adult care (Wiener, Kohrt, Battles & Pao, 2011).

While in no sites was discussion of disclosure well-documented, even post-mentoring, at Mochudi and Molepolole the observed improvement in this area from very low baselines was statistically-significant. In addition, in many sites disclosure is performed by lay counselors or social workers -who have been trained by COE physicians and nurses -and as such not documented in clinic notes. As a result, there may be a greater actual improvement in performance of disclosure than measured in this study.

There were not sufficient numbers of records where detectable viral load was managed to be able to make statistically-relevant conclusions regarding whether this very important aspect of care improved after mentoring, nor was a comparison of relative performance between physicians and nurse prescribers possible. These outcomes, particularly the latter consideration regarding nurse prescribers, which a larger study may be able to evaluate, are very much of interest in settings such as Botswana’s, where physician-oriented models of care are unlikely to be able to provide universal access to care for HIV-infected children (Hulela et al., 2008; Monyatsi et al., 2012; WHO, UNAIDS & President’s Emergency Plan for AIDS Relief [PEPFAR], 2008).

The improvements at sites were not uniform, with Mochudi and Molepolole showing more improvement, and in more areas, than Thamaga, and Phutadikobo showing the least improvement. In part, this could be explained by Mochudi and Molepolole ART clinics being based in district hospitals with more consistent and specialized staff (physicians, nurses, and pharmacists) and more comprehensive laboratory facilities. Better laboratory facilities may account for swifter turn-around times on laboratory results with less loss of results, influencing the results for documentation of viral load, CD4, and other monitoring results. Given heterogeneity amongst sites – a feature not restricted to Botswana -a single approach to clinical mentoring may not be effective at all sites, and individual site factors should be taken into consideration when planning the nature of mentoring at a particular site.

Consistent presence of staff available for mentoring is likely a key factor in a site’s development under the influence of clinical mentoring. Clearly, the impact of mentoring will be less when health care workers are irregularly present for mentoring opportunities or where there is frequent staff turnover. Indeed, rotation of staff from one clinical site to another is a feature of many national health systems in our region, including Botswana, and probably contributes to the lack of traction experienced by paediatric HIV programmes and other vital health programmes in many settings.

In addition to the limitations mentioned above, our pilot study is further limited by its retrospective design and potential confounding factors. While none of the sites studied had other formal, longitudinal clinical mentoring support similar to ours during the study period, our study was not designed to assess the presence or impact of other relevant training site personnel may have received over the study period. Botswana has a national weeklong health care provider-training course on paediatric HIV (Paediatric KITSO) that some site personnel may have attended, and there are other paediatric-specialized programmes that visit sites irregularly. As well, the effects of improvement through self-study, or with experience, cannot be determined by our study. There is also the possibility that what we measured is not a true change in practice, but merely an improvement in documentation of practice, and that documentation may vary from visit to visit depending on whether a mentor was present.

A final limitation concerns the quality of data being reviewed. A lack of a standardized charting system at sites made evaluation of the records challenging. Some handwritten notes were difficult to decipher; there was a lack of consistency among provider documentation of certain information that may have been covered in the visit but not documented; and some of the charting templates prompted providers to document certain information while others did not. The subjectivity generated by these hurdles in reviewers’ deciding whether or not an indicator was charted in a given record was, however, likely mollified to a considerable degree by the background of the three reviewers who performed the study as paediatricians with extensive experience in the nuances of paediatric HIV management in Botswana’s national health system. Designing and implementing comprehensive charting systems that prompt full evaluation and documentation (e.g., adherence, disclosure, pill count) would make future assessments more robust, and, importantly, also likely improve patient care.

As well as a larger prospective evaluation of clinical mentoring in support of paediatric HIV scale-up in Botswana, focused on mentoring’s impact on clinical outcomes, studies of mentored providers’ abilities and interest in managing paediatric HIV outside specialized centres would also be useful in enriching the current understanding of the role of mentoring in paediatric ART programme development and support.

Conclusion

This study begins to evaluate the effectiveness of clinical mentoring, suggesting a role for clinical mentoring by experienced providers as ART programmes in resource-limited settings scale-up paediatric HIV care and treatment programmes. Clinical mentoring may assist improvements in a number of important areas, including proper ART dosing and monitoring; adherence assessment and assurance; and disclosure. It may be particularly important in settings where nurses and other non-physician clinicians are assuming task-shifted roles in HIV management and depend on in-service training and mentoring for the acquisition of important skills. Further study of existing methods will help refine approaches to clinical mentoring, with focus on assuring mentoring results in improved clinical outcomes for HIV-infected children.