Skip to main content
Trials logoLink to Trials
. 2017 Dec 28;18:619. doi: 10.1186/s13063-017-2266-z

Simulated patient encounters to improve adolescent retention in HIV care in Kenya: study protocol of a stepped-wedge randomized controlled trial

Kate S Wilson 1,, Cyrus Mugo 2, David Bukusi 3, Irene Inwani 2, Anjuli D Wagner 1, Helen Moraa 2, Tamara Owens 4, Joseph B Babigumira 1, Barbra A Richardson 5, Grace C John-Stewart 1,6,7,9, Jennifer A Slyker 1, Dalton C Wamalwa 2, Pamela K Kohler 1,8
PMCID: PMC5745919  PMID: 29282109

Abstract

Background

Adolescent-friendly policies aim to tailor HIV services for adolescents and young adults aged 10–24 years (AYA) to promote health outcomes and improve retention in HIV care and treatment. However, few interventions focus on improving healthcare worker (HCW) competencies and skills for provision of high-quality adolescent care. Standardized patients (SPs) are trained actors who work with HCWs in mock clinical encounters to improve clinical assessment, communication, and empathy skills. This stepped-wedge randomized controlled trial will evaluate a clinical training intervention utilizing SPs to improve HCW skills in caring for HIV-positive AYA, resulting in increased retention in care.

Methods/design

The trial will utilize a stepped-wedge design to evaluate a training intervention using SPs to train HCWs in assessment, communication, and empathy skills for AYA HIV care. We will recruit 24 clinics in Kenya with an active electronic medical record (EMR) system and at least 40 adolescents enrolled in HIV care per site. Stratified randomization by county will be used to assign clinics to one of four waves – time periods when they receive the intervention – with each wave including six clinics. From each clinic, up to 10 HCWs will participate in the training intervention. SP training includes didactic sessions in adolescent health, current guidelines, communication skills, and motivational interviewing techniques. HCW participants will rotate through seven standardized SP scenarios, followed by SP feedback, group debriefing, and remote expert evaluation. AYA outcomes will be assessed using routine clinic data. The primary outcome is AYA retention in HIV care, defined as returning for first follow-up visit within 6 months of presenting to care, or returning for a first follow-up visit after re-engagement in care in AYA with a previous history of being lost to follow-up. Secondary outcomes include HCW competency scores, AYA satisfaction with care, and AYA clinical outcomes including CD4 and viral load. Additional analyses will determine cost-effectiveness of the intervention.

Discussion

This trial will contribute valuable information to HIV programs in Kenya and other low-resource settings, providing a potentially scalable strategy to improve quality of care and retention in critical HIV services in this population.

Trial registration

ClinicalTrials.gov, ID: NCT02928900. Registered 26 August 2016.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2266-z) contains supplementary material, which is available to authorized users.

Keywords: Adolescents and young adults, Retention in HIV care, Clinical training intervention, Standardized patients, Stepped-wedge trial

Background

Inadequate provision of accessible and acceptable HIV testing, counseling, and treatment services is a barrier to uptake of, and retention in, HIV care among adolescents and young adults aged 10–24 years (AYA) [13]. Studies from Africa have reported loss to follow-up from HIV care, with varying definitions, among AYA ranging from 15 to 57% [46]. Poor engagement in care may increase risk of HIV transmission and early death [7, 8]. Reducing HIV-related mortality and onward transmission requires the identification of evidence-based interventions that promote adolescent uptake of, and retention in, HIV care [1].

Quality of care and patient satisfaction with HIV services have been shown to affect retention in care [912]. In particular, AYA have expressed reluctance to seek care for fear of judgment by HCW, perceived HIV-stigma and discrimination by HCWs, and concerns over lack of confidentiality in healthcare settings [1315]. AYA experience unique physiological, developmental, and psychosocial changes that require services appropriate to their developmental stage [16, 17]. Initiatives are underway to implement the World Health Organization guidelines for “adolescent-friendly” health services [18], with the premise that offering care that is accessible, acceptable, appropriate, equitable, and effective will improve AYA engagement in health services and clinical outcomes. Healthcare workers (HCWs) are encouraged to be “non-judgmental and considerate” and have the “competencies to deliver the right health services in the right way” [19]. The Kenyan government has developed ambitious guidelines for adolescent-friendly HIV services in Kenya. However, there is a gap between advertised “adolescent-friendly services” and HCWs’ reported skills and training to deliver such care [14, 20].

Standardized patients (SPs) are trained actors who work with HCWs in simulated clinical encounters for training and evaluation [21]. Clinical training using SPs is an acceptable and effective approach to improve HCW skills in patient-centered care, including empathy, communication skills, and adherence to clinical guidelines [22, 23]. Studies in low-resource settings have used SPs to evaluate quality of health service delivery [20, 2426], but there are few examples of SP training interventions. Emerging evidence suggests that this methodology not only improves service delivery, but can also improve patient outcomes [27, 28]. Given the success of SP programs in other settings, it is plausible that high-quality, patient-centered approaches could fill this “know-do gap” in quality, and increase retention in care among HIV-positive AYA.

There is a lack of scalable, evidence-based interventions that build adolescent-friendly capacities in HCWs and that have been linked to improved AYA care outcomes [29, 30]. To address this need, the University of Nairobi and the University of Washington have partnered with the Kenya Ministry of Health’s National AIDS and STI Control Program (NASCOP) to evaluate the effectiveness of an innovative clinical training intervention targeting adherence to clinical guidelines, as well as communication and interpersonal skills among HCW.

The intervention will be evaluated using a stepped-wedge randomized controlled trial (RCT) design. We hypothesize that SP training will improve HCW competency and AYA satisfaction, increase AYA retention in HIV care, and will be cost-effective. Results from this study could demonstrate a scalable intervention to improve AYA services, retention, and programmatic impact in HIV care clinics.

Methods/design

Study design

The Simulated Patient Encounters to promote Early Detection and engagement in care for adolescents (SPEED) trial aims to evaluate the effectiveness of the SP training intervention using a stepped-wedge RCT design in 24 HIV care clinics in Kenya. This pragmatic study design was chosen because this training intervention occurs at the clinic, rather than at the individual level, uses routine clinic data for primary outcome measurement, and would not be feasible to implement simultaneously at all facilities [31]. In this modified, one-way, cross-over design, all facilities eventually receive the intervention, which was important for government partners. Facilities are randomized to when the intervention is introduced, which allows for benefits of randomization [32]. All facilities will contribute intervention and control periods. The RCT consists of four intervention waves approximately 9 months apart. Each wave includes HCW participants from six facilities (Fig. 1). Once facilities receive the training intervention, they are considered “exposed” until the end of the trial.

Fig. 1.

Fig. 1

Adapted CONSORT Diagram for trial

Study sites and population

We engaged NASCOP and county leadership during the planning phase of this study to identify eligible health facilities. We also established a study Community Advisory Board (CAB), including AYA leaders from networks of people living with HIV, to provide guidance on recruitment and retention of facilities and participants. The sampling frame included secondary and tertiary HIV care facilities from four counties in Kenya that were purposefully selected to represent different HIV epidemic patterns and services to enhance generalizability by geographic region and HIV prevalence. Eligible clinics had at least 40 AYA enrolled in care, a current electronic medical record (EMR) system, and no other special adolescent interventions. Facilities were excluded if anything would prevent the complete conduct of the training intervention at that site and/or the collection of outcome measures (e.g., discontinuation of the EMR system). Among 30 eligible facilities, we selected 24 facilities ensuring a balanced distribution of counties and facility sizes in each wave.

Electronic medical records from all AYA patients ages 10–24 years enrolled in HIV care at participating sites are eligible for data abstraction. A sample of AYA presenting for HIV care at enrolled sites are further eligible to complete patient satisfaction surveys. Eligible HCW participants are 18 years of age or older, employed at trial sites, and provide clinical and/or counseling services to adolescents. Individuals are excluded if they experience any conditions that would place them at increased risk of harm or preclude the individual’s completion of the study.

Randomization

Facilities were randomized to one of four intervention waves (Table 1), stratified by county: Nairobi, Kiambu, Kisumu, Homa Bay and by facility size. County was used for stratification because it is a reasonable marker of differences in AYA HIV prevalence and HIV care services across regions [33]. All possible combinations that yielded an allocation of two Nairobi, two Kiambu, one Homa Bay, and one Kisumu site, half high (more than 73 AYA) and half medium volume (less than 73 AYA, based on the median number of AYA enrolled in all sites) in each wave were generated in Excel. One of the possible combinations was randomly chosen using a random-number generator for the randomization allocation.

Table 1.

Overview of the SPEED Intervention Timing, by Wave

graphic file with name 13063_2017_2266_Tab1_HTML.jpg

Key: white boxes = unexposed periods; grey boxes= exposed periods

Blinding

Because this is a clinical training intervention using a pragmatic trial design, it is impossible to blind all members of the study team or trial participants to the randomization assignments. However, we have implemented procedures to minimize the number of individuals who are unblinded. Only the biostatistician has access to the randomization assignments of the 24 facilities. One month before each wave, the biostatistician provides a list of the six facilities in that wave to the study lead and the coordinator. All other facilities are informed that training will occur at a later wave. An independent External Advisory Panel reviews recruitment, enrollment, and potential for social harms annually during the trial. Because of the stepped-wedge design, there are no interim analyses or stopping rules.

Training of standardized patients

We contracted with a casting agency in Kenya to recruit professional actors to be trained as SPs, hiring 14 Kenyan actors over age 18 years who could believably role-play AYA patients between 14 and 21 years. Before piloting the intervention, actors received a 5-day training facilitated by a consultant expert in the SP methodology. The training included review of seven case scripts, individualized coaching on role playing cases believably without over dramatizing, video-taped practice of the simulated patient encounters with actual HCWs followed by group discussion to refine the SPs’ portrayal of the cases, and training on providing verbal feedback to HCWs and completing the SP actor feedback forms. At the end of the training, the consultant and study team determined the final case assignments (Additional file 1).

Recruitment and enrollment: HCWs and AYA clients

Clinic leadership personnel are consulted to provide a list of up to 10 HCWs who provide adolescent services and to negotiate release from work to participate in the training and survey data collection. A study interviewer then approaches those HCWs to invite them to learn more about the study. HCWs provide written informed consent. HCWs are tracked over time to assess provider retention at clinical intervention sites, using a facility survey to monitor staff turnover. All AYA seeking HIV care on a given day are offered referral by a clinical staff member to our study team for survey participation. We schedule facility visits to “adolescent clinic” days or during school breaks to ensure adequate volumes of eligible patients. Oral consent or assent is obtained from AYA based on their status as emancipated minors. AYA aged 18 years and older, and those aged 14–17 years who are emancipated minors, provide oral consent to participate in surveys. Otherwise, parents/caregivers provide oral informed consent for AYA ages 10–17 years, and those adolescents provide oral assent.

SP training intervention

The SP intervention is based on an adaptation of Kolb’s model of Experiential Learning [34], Bandura’s model of Social Learning Theory [35], and an adaptation of Andersen’s Behavioral Model of Health Services Use [36]. The 2-day training intervention focuses on skill-building through practice, with a series of SP encounters, individual feedback by SPs, and group debriefings. HCW participants receive orientation to the SP methodology, instructions for taking part in the encounters, and short didactic sessions on adolescent-friendly services in Kenya and motivational interviewing techniques [37]. After the didactic session, each participant rotates through seven SP encounters. Encounters are video-recorded and occur at a training venue with stations set up to resemble counseling rooms, separated by a wall or curtain for privacy. At the end of each encounter, the SP provides non-technical, emotional feedback. Facilitated group debriefings consist of review of videos and discussion about strengths and areas for improvement in each encounter.

Standardized case scripts

Each case script follows a standard format, with a chief complaint, relevant medical and social history, and actor prompts (Additional file 2). Qualitative analysis of in-depth interviews with HIV-positive AYA enrolled in a study on HIV-testing services in Nairobi, Kenya informed the range of cases and chief complaints [38]. We also consulted World Health Organization (WHO) and Kenyan guidelines on “adolescent-friendly” services to ensure that the cases covered major AYA health concerns, including contraception and fertility desire (whether the client wants children in the future) [5, 13], sexual identity [39], mental health [40], HIV-stigma and disclosure [14], and gender-based violence [41]. These issues also are associated with poor retention in health services [5, 13, 42, 43]. Case scripts were reviewed by Kenyan general practitioners, pediatricians, and a psychiatrist with experience in AYA care; the training consultant; and SP actors for accuracy and relevance to local context. There are seven cases in total, including both male and female roles, and one case with an older sibling present as the caregiver (Table 2).

Table 2.

SPEED Training Intervention Case Scripts

Case Name Key features and health concerns
Alex, Lisa (Case “0”) 17 year old male (Alex) or female (Lisa) with no complaints, good adherence, disclosed to family
Teddy 15 year old male, cognitive delay that can be mistaken for depression
Rehema 17 year old female, needs family planning, had bad experiences with health care providers in the past. She has a history of tuberculosis.
Nick, accompanied by older sister Emily 13 year old male whose is unaware of his HIV status and thinks he is taking medicine for a lung infection. He is in need of disclosure counseling. His older sister, who is also HIV-positive, is his primary caregiver. She has been hiding his HIV status from him all these years. She distrusts medical providers and HIV treatments, and sometimes gives her brother herbal treatments.
Lucy 18 year old female with STI symptoms, an older “sponsor” who is abusive, and an alcohol problem related to being in an abusive relationship.
Ian 16 year old gay-identified who is afraid of coming out to his friends and family. His HIV status is not his primary concern. He experiences suicidal ideation because he cannot be himself around anyone. For this reason, he has poor adherence and alcohol problems.
Ashley 19 year old female in need of counseling about fertility desire. She has mixed feelings about having a child. Her boyfriend knows her HIV-status and is pressuring her to get pregnant.

Intervention waves

All training materials and data collection tools were pilot tested. The SP training intervention is implemented at the 24 facilities over four waves (Fig. 2, Additional file 3: Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Checklist). Each intervention wave trains HCWs from six clinics, approximately every 9 months. We anticipate that training all HCW participants in each wave will take up to 4 weeks. Each wave includes HCW participants from all four counties. We repeat the training with groups of participants from each facility or county until all participants are trained; timeframes vary based on HCW availability and travel. The intervention training schedule is planned to minimize disruption to clinic operations and reflect a “real-world” in-service training.

Fig. 2.

Fig. 2

Adapted Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Diagram for trial

Monitoring intervention fidelity

Intervention fidelity is monitored throughout the trial. The SP training included practice-simulated encounters with actual HCWs, which helped to increase the realism and relevance of the case and identify areas where the actor needed additional training. We review a random sample of video-taped encounters in each intervention wave and rate actor fidelity according to a standardized checklist that has been adapted for this study [44]. We conduct refresher trainings before each intervention wave, and follow-up refresher trainings for actors who need additional support. We use standard operating procedures and standard training materials for the didactic sessions, track HCW retention in the training intervention, and track retention of SPEED-trained HCWs at study facilities over time.

Outcome measures using EMR data

The primary outcome for this study is AYA retention in HIV care at the individual level, defined as the probability of returning for first follow-up visit within 3 months of enrollment or returning for the first follow-up visit within 3 months after a “re-engagement visit,” among AYA who had been previously lost to follow-up (LTFU) (Table 3). A “re-engagement in care” visit is defined as a clinic visit after at least a 3-month absence from care. Because AYA who come for a re-engagement visit are effectively treated as new clients, this group is likely similar to newly enrolled AYA clients, and their inclusion will increase study power. Because there is no universal definition of LTFU [45] and the patterns of return visits are not well documented in AYA [4, 46], we will test an alternative definition of our primary outcome in a sensitivity analysis as return for any follow-up visit within 3 months among AYA currently enrolled in HIV care (i.e., patients “actively enrolled”).

Table 3.

Summary of Primary, Process, and Exploratory Outcomes, Indicators, and Sources

Indicator Source
Primary Outcomes
 AYA retention in care Follow-up visit within 3 months for newly enrolled AYA or those Lost to Follow Up (LTFU) (i.e. out of care for more than 3 months) Clinic records
Process Outcomes
 HCW competency Mean HCW scores per clinic Faculty graded scores
Exit interviews with trained HCWs
 HCW satisfaction Mean HCW score per clinic HCW survey
 AYA patient satisfaction Mean AYA score per clinic Tablet-based questionnaire
Exploratory Outcomes
 Adherence Refills within 1 week of scheduled return visit; 95% or one or fewer missed doses in past month/dosing period Clinic/pharmacy records
 Viral suppression Viral load (VL) <1000 copies per ml and a secondary analysis using VL <400 copies/ml Clinic records
 Linkage to APS services STI/TB screening, Contraception uptake, mental health referral Clinic records
 Opportunistic Infections (OI) Indication of at least one OI Clinic records
 Hospitalizations Number of hospitalizations Clinic records
 AIDS defining illness Diagnosis of at least one AIDS defining illness per national guidelines Clinic record
 Mortality Death during follow up Clinic records
Cost effectiveness and Utility Outcomes
 Cost effectiveness Cost per additional HIV-positive AYA retained in care Tablet-based questionnaire, published literature
 Cost utility Cost per life year saved and disability-adjusted life (DALY) averted Tablet-based questionnaire, published literature

APS Adolescent Package of Services, HCW Healthcare Worker

Secondary outcomes

Process and exploratory outcomes for the trial are described in Table 3. Process measures are collected using surveys to understand mechanisms of intervention effectiveness: HCW satisfaction, HCW competency, and AYA patient satisfaction with care. Exploratory outcomes to evaluate additional intervention effects on AYA outcomes are abstracted from EMR data. They include: antiretroviral therapy (ART) adherence (≥95% based on number of missed doses in past month by self-report. Refills within 1 week of scheduled return visit.); virologic suppression (<1000 copies/ml); linkage to Adolescent Package of Care services (e.g., family planning); number of hospitalizations; number of opportunistic infections; AIDS-defining illness; and mortality.

Costing, cost-effectiveness, and cost-utility outcomes

Outcomes for the health economic analysis are: cost-effectiveness, defined as the cost per additional HIV-positive AYA retained in care and, cost-utility of the intervention, defined as cost per disability-adjusted life-year (DALY) averted and cost per life-year saved. Additional exploratory outcomes for the cost-effectiveness analysis will include: cost per additional HIV-positive AYA with good adherence as defined in the EMR (≥95% or one or fewer missed dose in past month/dosing period. Refills within 1 week of scheduled return visit); cost per additional HIV-positive AYA virally suppressed; and cost per additional HIV-positive AYA linked to adolescent package of services defined by the Kenyan Ministry of Health (e.g., referrals for mental health counseling, family planning, gender-based violence care). Cost data will be estimated from AYA surveys, program records, and published literature, and include direct medical costs, direct non-medical costs, and indirect costs.

Data collection

Eligible de-identified AYA clinic records from the EMR system are abstracted from each facility at baseline, and at the end of each wave to assess primary and exploratory outcomes (Table 3). Data for the primary outcome will be limited to records from newly enrolled and recently re-engaged AYA clients. The time period for data abstraction is 15 months before baseline up until 15 months after the wave 1-4 training intervention. Each data abstraction period covers 15 months to allow sufficient time to observe AYA retention outcomes or approximately two school breaks (about 3–6 months apart) when AYA typically come for HIV care. Data are abstracted and transferred to an encrypted external hard drive. Survey data are collected on electronic tablets using the Open Data Kit (ODK) software (https://opendatakit.org/) [47]. The software provides an intuitive interface for validated data entry, automated export procedures for data downloads, and procedures for importing external data. Survey data are automatically uploaded to the secure study server when the device is connected to the Internet. Video-recordings are made of each HCW-SP interaction for debriefing purposes. Videos are reviewed and scored by a team of Kenyan and US-based experts to provide technical feedback on clinical competencies. Semi-structured exit interviews are conducted with a purposeful sample of up to 30 HCWs approximately 9–24 months after the SPEED intervention training to assess durability of new skills over time.

Sample size calculation

Sample size and power calculations used methods described by Hussey and Hughes [32]. Assuming 75% of AYA in the control period will return to clinic after first visit [6], a 15% dropout without replacement, a coefficient of variation (CV) of 0.25, and a two-tailed test with α = 0.05, 24 clusters and five time points, we would have 80% power to detect a 15% difference, or achievement of 90% retained in care, between control and intervention periods. The lower bound of the sample size required is estimated to be 720 clinic records (6 records/site × 24 sites × 5 time points). In the optimal scenario, with an average of 40 newly enrolled AYA clients at each facility per wave, and five data collection points, our estimated sample size would be 4800 clinic records. Under the same assumptions as above, we would have > 99% power to detect a 15% difference in retention between the intervention and control periods.

Statistical methods and analysis

The primary analysis is intent-to-treat (ITT), assuming that AYA who receive care at clinics after the SPEED training will be “exposed” to HCWs trained in the SPEED intervention until the end of the study. No replacement of facilities will be used once the trial has begun. Baseline values of primary and process outcomes, facility characteristics, and socio-demographic characteristics of AYA and HCWs will be presented in descriptive tables.

We will use generalized linear mixed models (GLMM) to compare the probability of AYA retention in HIV care between the exposed (i.e., intervention) and unexposed (i.e., control) periods [48]. To estimate the effect of the intervention (Xij) on the individual level, we will use a GLMM model with a binomial distribution and log link, allowing for random effects for clusters (u i) and fixed effects for time (j). This approach models individual-level outcomes, adjusts for temporal trends, and accounts for correlation of outcomes within multiple levels of clustering (clinic, individual). We will estimate adjusted risk ratios (ARR) and 95% confidence intervals at an α = 0.05 level (two-sided). We will estimate the effect of the intervention on each secondary outcome using GLMM to generate ARRs. All models will be adjusted for clustering and time (c). The equation for a generic model is:

Yijk=β0+βjtime+βeffectXij,+βcXij,ui+εij

where i indexes the cluster (facility), j indexes time, and k indexes the individual.

We will conduct exploratory analyses to evaluate intervention mechanisms, non-constant temporal trends, and lag effects. We will examine whether an observed intervention effect may be explained through HCW post-training competency and AYA patient satisfaction scores. In that analysis, mean AYA satisfaction scores will be evaluated as predictors of AYA retention in care from EMR data. Temporal trends that change over time can confound the effect estimate in a stepped-wedge trial design [32, 48]. For example, new AYA HIV treatment guidelines may be introduced nationwide during the intervention that affect AYA retention. Time-varying temporal trends will be evaluated using methods recommended for stepped-wedge trials [48, 49]. The effect of the intervention may not be immediate or it may diminish. Intervention lag will be evaluated by including an interaction term between the intervention status (fixed effect) and time [48], and, separately, by using a fractional term for the coefficient of primary-effect estimate, which will reduce in magnitude at each time step [49].

Interview data will be analyzed using a constant comparative approach by two researchers [50]. They will code transcripts separately and compare results, resolving differences in application of codes through discussion. A codebook will be developed based on preliminary reading of the transcripts, and revised based on this iterative process. Patterns identified in the coding within and across interviews will be used to identify major themes around HCW uptake and application of skills learned in the training intervention. Results will be triangulated with the quantitative results in the study.

We will estimate the cost-effectiveness of the SP intervention in terms of cost per additional HIV-positive AYA identified and retained in care. The primary effectiveness estimate from the trial will be used for the number of additional AYA retained in care as a result of the intervention. Direct medical costs include laboratory tests, medication costs, other medical procedure costs, and personnel salaries. Direct non-medical costs include AYA transportation, food, and childcare costs. Indirect costs include opportunity costs from lost wages in AYA paid and unpaid work. We will also model the cost-utility of the intervention in terms of cost per life-year saved and DALY averted by maintaining HIV-positive AYA in care, including subsequent decreases in HIV transmission/acquisition. This will require linking effectiveness data to a mathematical model that estimates DALYs in AYA HIV progression, using a model adapted for AYA populations.

We will examine how these estimates vary through one-way and probabilistic sensitivity analyses in settings with varying AYA HIV prevalence, HCW wages, patient volume, and linkage to care rates. This will assist in identifying scenarios where the intervention is most cost-effective, allowing policy-makers to determine incremental costs and net benefits of the intervention. We will perform this analysis both from a limited societal perspective and from a Ministry of Health (healthcare sector) perspective. All benefits and costs will be discounted at 3% per year [51].

Discussion

With global commitment to combat the HIV epidemic among AYA, and limited evidence of what works, it is important to evaluate novel and potentially scalable strategies to improve HIV outcomes in this population in Sub-Saharan Africa. Importantly, there is a lack of rigorous studies of any HCW training programs in resource-limited settings to improve adolescent-friendly services [29, 30]. This study fills several important gaps in AYA HIV research. We are unaware of any other intervention that uses SPs to improve provider skills in working with HIV-positive AYA. While SPs are standard in pre-service medical education in high-resource settings, our use of SPs for an in-service training intervention is novel. This study will measure intervention effectiveness on AYA health outcomes using a rigorous RCT design. This study leverages Kenya’s new EMR system for study outcomes, which is resource-efficient. This study is responsive to the Kenyan Ministry of Health request for novel interventions to inform the implementation of new adolescent-friendly HIV service guidelines [52, 53]. In addition, this study will provide data on core AYA HIV indicators for the UN “90-90-90” targets from a range of geographic and epidemiologic settings in Kenya.

Understanding the cost-effectiveness of HIV interventions for AYA is important for policy-makers to allocate resources effectively [54]. If the SP intervention is effective, we expect to generate novel and useful information on the cost-effectiveness of this intervention. To minimize the costs associated with this intervention, we designed the training to align with the local facility infrastructure, staffing, and training schedules. If the intervention is not effective, we will nonetheless generate estimates of the total cost of this training, as well as the direct and indirect costs to AYA for seeking HIV care, which will be useful for the Kenyan Ministry of Health and future economic analyses of AYA HIV interventions.

Successful implementation of the SP intervention requires careful consideration of several contextual factors. First, strong engagement of stakeholders from NASCOP, county governments, community members, and non-governmental organizations that support facilities is essential. Our established partnerships with NASCOP and county leadership has facilitated access to study facilities and EMR data, and enabled us to stay current on the evolving landscape of adolescent-friendly services in Kenya. Youth perspectives are necessary to include in adolescent health interventions [19]. Our CAB offers important insights into AYA attitudes towards HCWs and challenges of living with HIV. In addition, AYA case scripts, which are the foundation of the training intervention, require multiple revisions to ensure that they are tailored to the local clinical context. Cases were refined through pilot testing with HCWs from a range of cadres and through feedback from SPs, faculty experts, and AYA. Finally, EMR data abstraction is a complex process that likely will vary by partner organization and facility. The EMR system is new in Kenya, and facilities have different capacities for data entry and retrieval. We have established relationships with each partner organization that supports EMR systems at each facility. We can also offer sites support in the data abstraction process.

This study also has limitations. A 3-month strike by public-sector physicians and ensuing staff shortages in public facilities resulted in delays in baseline data collection, and a nursing strike is ongoing. Our ability to recruit AYA for the surveys depends on their school breaks and facilities’ schedules for this patient population. As with any in-service training, turnover of SPEED-trained HCWs is possible, and would dilute the intervention effectiveness. A limitation of a stepped-wedge design is the potential bias introduced by contextual factors that vary over time [32]. We will monitor these changes with the facility survey and evaluate them in the analysis. This intervention targets HCWs to improve AYA retention in HIV care. Other community factors, including HIV-stigma, family and peer support, and school schedules, may also affect retention rates, which we do not capture [4, 11, 14]. Finally, our primary outcomes will depend on the quality of EMR data.

A standardized patient intervention for HCW has the potential to promote high-quality patient-centered HIV services for AYA by providing HCWs with technical skills and pragmatic experiences. This improvement in delivery of care can, in turn, address barriers cited by AYA and improve retention in essential HIV services. By increasing engagement in care, the intervention can ultimately improve quality of life, survival, and decrease subsequent HIV transmission among AYA in Kenya, closing the gap in the “90-90-90” targets.

Trial registration

Pilot testing of the intervention was completed in October 2016. Participant recruitment is ongoing, and the trial is expected to end in 2020. The trial is registered at ClinicalTrials.gov (NCT02928900).

Additional files

Additional file 1: (658.5KB, eps)

Example case script. (EPS 658 kb)

Additional file 2: (171.6KB, pdf)

Example Consent Form, HCW. (PDF 171 kb)

Additional file 3: (122KB, doc)

SPIRIT Checklist. (DOC 122 kb)

Acknowledgements

The authors would like to thank our colleagues, Dr. Martin Sirengo, Dr. Laura Oyiengo, and Dr. Maureen Kimani, from NASCOP for their feedback during the development of the study protocol. We thank the Kenyan Ministry of Health, county health leadership, members of our Community Advisory Board, and facility partner organizations for their support in planning and implementation of this study. We also acknowledge the contributions of our External Advisory Panel on planned analyses. This research is supported by a grant from the US National Institutes of Health (NIH), 1R01HD085807. Preliminary data for this study were supported by National Institutes of Health (NIH) implementation science supplement to the University of Washington Center for AIDS Research (CFAR), an NIH-funded program (P30 AI027757). The funding agencies played no role in the study conception or design, writing of the manuscript, or decision to submit the manuscript for publication.

Funding

Funding is provided by the National Institutes of Health (NIH) (1R01HD085807, PI: Kohler).

Availability of data and materials

Not applicable at this time

Abbreviations

APS

Adolescent package of services

AYA

Adolescents and Young Adults

AYA

Adolescents and young adults aged 10–24 years

CAB

Community Advisory Board

DALY

Disability-adjusted life year

EAP

External Advisory Panel

EMR

Electronic medical record

GLMM

Generalized linear mixed models

HCW

Healthcare worker

ITT

Intent-to-treat

LTFU

Lost to follow-up

MOH

Ministry of Health

NASCOP

National AIDS and STI Control Program

ODK

Open Data Kit

SP

Standardized patients

SPEED

Simulated Patient Encounters to promote Early Detection and engagement in care for adolescents

Authors’ contributions

PK and DW are the principal investigators and supervised the trial protocol development and implementation. PK, DW, GJS, KW, AW, JS, CM, BR, II, JB, and DB participated in designing the study. TO trained the standardized patients for the trial. BR, KW, PK are responsible for the statistical design of the trial and data analysis. CM and HM implement specific aspects of the trial. KW and PK wrote the initial draft of the paper. All authors critically revised, read, and approved the final manuscript.

Ethics approval and consent to participate

This study was approved by University of Nairobi/Kenyatta National Hospital Ethics Review Committee (KNH ERC) (P476/06/2016) and the University of Washington Human Subjects Research Committee (51926 E/J). All HCW participants provide standard written informed consent. AYA aged 18 years and older and those aged 14–17 years, who are emancipated minors, provide oral consent to participate in surveys. Otherwise, AYA ages 10–17 years who participate in the surveys provide assent and caregivers provide oral informed consent. All SPs signed a confidentiality agreement and completed protection of human subjects training before any contact with participants. Important protocol amendments will be communicated to both ethics committees, the EAP, and trial registries.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Footnotes

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2266-z) contains supplementary material, which is available to authorized users.

Contributor Information

Kate S. Wilson, Phone: 206-685-0583, Email: ksw@uw.edu

Cyrus Mugo, Email: cyrusmugodr@gmail.com.

David Bukusi, Email: davidbukusi@gmail.com.

Irene Inwani, Email: iinwani@yahoo.com.

Anjuli D. Wagner, Email: anjuliw@uw.edu

Helen Moraa, Email: moraahelen38@gmail.com.

Tamara Owens, Email: tamowens713@gmail.com.

Joseph B. Babigumira, Email: babijo@uw.edu

Barbra A. Richardson, Email: barbrar@uw.edu

Grace C. John-Stewart, Email: gjohn@uw.edu

Jennifer A. Slyker, Email: jslyker@uw.edu

Dalton C. Wamalwa, Email: dalton@africaonline.co.ke

Pamela K. Kohler, Email: pkohler2@uw.edu

References

  • 1.UNAIDS. Get on the Fast Track: The life-cycle approach to HIV In Edited by Joint_United_Nations_Programme_on_HIV/AIDS. Geneva; UNAIDS; 2016.
  • 2.Mavhu W, Berwick J, Chirawu P, Makamba M, Copas A, Dirawo J, et al. Enhancing psychosocial support for HIV positive adolescents in Harare, Zimbabwe. PLoS One. 2013;8:e70254. doi: 10.1371/journal.pone.0070254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Philbin MM, Tanner AE, DuVal A, Ellen JM, Xu J, Kapogiannis B, et al. Factors affecting linkage to care and engagement in care for newly diagnosed HIV-positive adolescents within fifteen adolescent medicine clinics in the United States. AIDS Behav. 2014;18:1501–10. doi: 10.1007/s10461-013-0650-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Teasdale CA, Alwar T, Chege D, Fayorsey R, Hawken MP, Abrams EJ. Impact of youth and adolescent friendly services on retention of 10-24-year-olds in HIV care and treatment programs in Nyanza, Kenya. J Acquir Immune Defic Syndr. 2016;71:e56–9. doi: 10.1097/QAI.0000000000000877. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ojwang VO, Penner J, Blat C, Agot K, Bukusi EA, Cohen CR. Loss to follow-up among youth accessing outpatient HIV care and treatment services in Kisumu, Kenya. AIDS Care. 2016;28:500–7. doi: 10.1080/09540121.2015.1110234. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Koech E, Teasdale CA, Wang C, Fayorsey R, Alwar T, Mukui IN, et al. Characteristics and outcomes of HIV-infected youth and young adolescents enrolled in HIV care in Kenya. AIDS. 2014;28:2729–38. doi: 10.1097/QAD.0000000000000473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Kasedde S, Luo C, McClure C, Chandan U. Reducing HIV and AIDS in adolescents: opportunities and challenges. Curr HIV/AIDS Rep. 2013;10:159–68. doi: 10.1007/s11904-013-0159-7. [DOI] [PubMed] [Google Scholar]
  • 8.Kasedde S, Kapogiannis BG, McClure C, Luo C. Executive summary: opportunities for action and impact to address HIV and AIDS in adolescents. J Acquir Immune Defic Syndr. 2014;66(Suppl 2):S139–43. doi: 10.1097/QAI.0000000000000206. [DOI] [PubMed] [Google Scholar]
  • 9.Wachira J, Naanyu V, Genberg B, Koech B, Akinyi J, Kamene R, et al. Health facility barriers to HIV linkage and retention in Western Kenya. BMC Health Serv Res. 2014;14:646. doi: 10.1186/s12913-014-0646-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Godia PM, Olenja JM, Lavussa JA, Quinney D, Hofman JJ, van den Broek N. Sexual reproductive health service provision to young people in Kenya; health service providers’ experiences. BMC Health Serv Res. 2013;13:476. doi: 10.1186/1472-6963-13-476. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Kidia KK, Mupambireyi Z, Cluver L, Ndhlovu CE, Borok M, Ferrand RA. HIV status disclosure to perinatally-infected adolescents in Zimbabwe: a qualitative study of adolescent and healthcare worker perspectives. PLoS One. 2014;9:e87322. doi: 10.1371/journal.pone.0087322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Dang BN, Westbrook RA, Hartman CM, Retaining GTP, HIV Patients in cre: the role of initial patient care experiences. AIDS Behav. 2016;20:2477–87. doi: 10.1007/s10461-016-1340-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Godia PM, Olenja JM, Hofman JJ, van den Broek N. Young people’s perception of sexual and reproductive health services in Kenya. BMC Health Serv Res. 2014;14:172. doi: 10.1186/1472-6963-14-172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Wolf HT, Halpern-Felsher BL, Bukusi EA, Agot KE, Cohen CR, Auerswald CL. “It is all about the fear of being discriminated [against]…the person suffering from HIV will not be accepted”: a qualitative study exploring the reasons for loss to follow-up among HIV-positive youth in Kisumu, Kenya. BMC Public Health. 2014;14:1154. doi: 10.1186/1471-2458-14-1154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Mburu G, Hodgson I, Kalibala S, Haamujompa C, Cataldo F, Lowenthal ED, et al. Adolescent HIV disclosure in Zambia: barriers, facilitators and outcomes. J Int AIDS Soc. 2014;17:18866. doi: 10.7448/IAS.17.1.18866. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Resnick MD, Catalano RF, Sawyer SM, Viner R, Patton GC. Seizing the opportunities of adolescent health. Lancet. 2012;379:1564–7. doi: 10.1016/S0140-6736(12)60472-3. [DOI] [PubMed] [Google Scholar]
  • 17.Catalano RF, Fagan AA, Gavin LE, Greenberg MT, Irwin CE, Jr, Ross DA, et al. Worldwide application of prevention science in adolescent health. Lancet. 2012;379:1653–64. doi: 10.1016/S0140-6736(12)60238-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.UNAIDS. In: UNAIDS, editor. All In to #EndAdolescentAIDS. 2015.
  • 19.World_Health_Organization. Making health services adolescent friendly: developing national quality standards for adolescent-friendly services. Geneva: WHO Press; 2012.
  • 20.Mathews C, Guttmacher SJ, Flisher AJ, Mtshizana YY, Nelson T, McCarthy J, et al. The quality of HIV testing services for adolescents in Cape Town, South Africa: do adolescent-friendly services make a difference? J Adolesc Health. 2009;44:188–90. doi: 10.1016/j.jadohealth.2008.05.009. [DOI] [PubMed] [Google Scholar]
  • 21.Makoul G. Essential elements of communication in medical encounters: the Kalamazoo consensus statement. Acad Med. 2001;76:390–3. doi: 10.1097/00001888-200104000-00021. [DOI] [PubMed] [Google Scholar]
  • 22.Elley CR, Clinick T, Wong C, Arroll B, Kennelly J, Doerr H, et al. Effectiveness of simulated clinical teaching in general practice: randomised controlled trial. J Prim Health Care. 2012;4:281–7. [PubMed] [Google Scholar]
  • 23.Weaver M, Erby L. Standardized patients: a promising tool for health education and health promotion. Health Promot Pract. 2012;13:169–74. doi: 10.1177/1524839911432006. [DOI] [PubMed] [Google Scholar]
  • 24.Das J, Kwan A, Daniels B, Satyanarayana S, Subbaraman R, Bergkvist S, et al. Use of standardised patients to assess quality of tuberculosis care: a pilot, cross-sectional study. Lancet Infect Dis. 2015;15:1305–13. doi: 10.1016/S1473-3099(15)00077-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Garcia PJ, Holmes KK, Carcamo CP, Garnett GP, Hughes JP, Campos PE, et al. Prevention of sexually transmitted infections in urban communities (Peru PREVEN): a multicomponent community-randomised controlled trial. Lancet. 2012;379:1120–8. doi: 10.1016/S0140-6736(11)61846-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Kohler PK, Marumo E, Jed SL, Mema G, Galagan S, Tapia K, et al. A national evaluation using standardised patient actors to assess STI services in public sector clinical sentinel surveillance facilities in South Africa. Sex Transm Infect. 2017;93:247–52. doi: 10.1136/sextrans-2016-052930. [DOI] [PubMed] [Google Scholar]
  • 27.Walker DM, Cohen SR, Fritz J, Olvera-Garcia M, Zelek ST, Fahey JO, et al. Impact Evaluation of PRONTO Mexico: a simulation-based program in obstetric and neonatal emergencies and team training. Simul Healthc. 2016;11:1–9. doi: 10.1097/SIH.0000000000000106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Walker DM, Holme F, Zelek ST, Olvera-Garcia M, Montoya-Rodriguez A, Fritz J, et al. A process evaluation of PRONTO simulation training for obstetric and neonatal emergency response teams in Guatemala. BMC Med Educ. 2015;15:117. doi: 10.1186/s12909-015-0401-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Denno DM, Hoopes AJ, Chandra-Mouli V. Effective strategies to provide adolescent sexual and reproductive health services and to increase demand and community support. J Adolesc Health. 2015;56:S22–41. doi: 10.1016/j.jadohealth.2014.09.012. [DOI] [PubMed] [Google Scholar]
  • 30.MacPherson P, Munthali C, Ferguson J, Armstrong A, Kranzer K, Ferrand RA, et al. Service delivery interventions to improve adolescents’ linkage, retention and adherence to antiretroviral therapy and HIV care. Trop Med Int Health. 2015;20:1015–32. doi: 10.1111/tmi.12517. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391. doi: 10.1136/bmj.h391. [DOI] [PubMed] [Google Scholar]
  • 32.Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28:182–91. doi: 10.1016/j.cct.2006.05.007. [DOI] [PubMed] [Google Scholar]
  • 33.Kenya_Ministry_of_Health. Kenya_HIV_Prevention_Revolution_Road_Map. In: Edited by National_AIDS_STI_Control_Program. Nairobi: Ministry of Health; 2014.
  • 34.Kolb D. Experiential learning: experience as the source of learning and development. Englewood Cliffs: Prentice-Hall; 1984. [Google Scholar]
  • 35.Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84:191–215. doi: 10.1037/0033-295X.84.2.191. [DOI] [PubMed] [Google Scholar]
  • 36.Andersen RM. Revisiting the behavioral model and access to medical care: does it matter? J Health Soc Behav. 1995;36:1–10. doi: 10.2307/2137284. [DOI] [PubMed] [Google Scholar]
  • 37.Lundahl B, Moleni T, Burke BL, Butters R, Tollefson D, Butler C, et al. Motivational interviewing in medical care settings: a systematic review and meta-analysis of randomized controlled trials. Patient Educ Couns. 2013;93:157–68. doi: 10.1016/j.pec.2013.07.012. [DOI] [PubMed] [Google Scholar]
  • 38.Wilson KS, Beima-Sofie KM, Moraa H, Wagner AD, Mugo C, Mutiti PM, et al. “At our age, we would like to do things the way we want: ” a qualitative study of adolescent HIV testing services in Kenya. AIDS. 2017;31(Suppl 3):S213–20. doi: 10.1097/QAD.0000000000001513. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Hussen SA, Harper GW, Bauermeister JA, Hightow-Weidman LB, Adolescent Medicine Trials Network for HIVAI Psychosocial influences on engagement in care among HIV-positive young black gay/bisexual and other men who have sex with men. AIDS Patient Care STDs. 2015;29:77–85. doi: 10.1089/apc.2014.0117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Nabukeera-Barungi N, Elyanu P, Asire B, Katureebe C, Lukabwe I, Namusoke E, et al. Adherence to antiretroviral therapy and retention in care for adolescents living with HIV from 10 districts in Uganda. BMC Infect Dis. 2015;15:520. doi: 10.1186/s12879-015-1265-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Kim MH, Mazenga AC, Yu X, Ahmed S, Paul ME, Kazembe PN, et al. High self-reported non-adherence to antiretroviral therapy amongst adolescents living with HIV in Malawi: barriers and associated factors. J Int AIDS Soc. 2017;20:21437. doi: 10.7448/IAS.20.1.21437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Blank AE, Fletcher J, Verdecias N, Garcia I, Blackstock O, Cunningham C. Factors associated with retention and viral suppression among a cohort of HIV+ women of color. AIDS Patient Care STDs. 2015;29(Suppl 1):S27–35. doi: 10.1089/apc.2014.0272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Cluver LD, Hodes RJ, Toska E, Kidia KK, Orkin FM, Sherr L, et al. “HIV is like a tsotsi. ARVs are your guns”: associations between HIV-disclosure and adherence to antiretroviral treatment among adolescents in South Africa. AIDS. 2015;29(Suppl 1):S57–65. doi: 10.1097/QAD.0000000000000695. [DOI] [PubMed] [Google Scholar]
  • 44.Wind LA, Van Dalen J, Muijtjens AM, Rethans JJ. Assessing simulated patients in an educational setting: the MaSP (Maastricht Assessment of Simulated Patients) Med Educ. 2004;38:39–44. doi: 10.1111/j.1365-2923.2004.01686.x. [DOI] [PubMed] [Google Scholar]
  • 45.Rosen S, Fox MP. Retention in HIV care between testing and treatment in sub-Saharan Africa: a systematic review. PLoS Med. 2011;8:e1001056. doi: 10.1371/journal.pmed.1001056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Lamb MR, Fayorsey R, Nuwagaba-Biribonwoha H, Viola V, Mutabazi V, Alwar T, et al. High attrition before and after ART initiation among youth (15–24 years of age) enrolled in HIV care. AIDS. 2014;28:559–68. doi: 10.1097/QAD.0000000000000054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Hartung CAY, Brunette W, Lerer A, Tseng C, Borriello G. Open Data Kit: tools to build information services for developing regions, ICTD “10”_Proceedings of the 4th ACM/IEEE International Conference on Information and Communication Technologies and Development. 2010. Article 8
  • 48.Hemming K, Taljaard M, Forbes A. Analysis of cluster randomised stepped wedge trials with repeated cross-sectional samples. Trials. 2017;18:101. doi: 10.1186/s13063-017-1833-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, et al. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358. doi: 10.1186/s13063-015-0838-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
  • 51.Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in Health and Medicine. New York: Oxford University Press; 1996.
  • 52.Kenya_Ministry_of_Health. National guidelines for provision of adolescent Youth-Friendly Services (YFS) in Kenya. Nairobi, Kenya: Kenya_Ministry_of_Health; 2005.
  • 53.Kenya_Ministry_of_Health. National adolescent sexual and reproductive health policy. Nairobi, Kenya: Kenya_Ministry_of_Health; 2015.
  • 54.Stover J, Rosen J, Kasedde S, Idele P, McClure C. The impact and cost of the HIV/AIDS investment framework for adolescents. J Acquir Immune Defic Syndr. 2014;66(Suppl 2):S170–5. doi: 10.1097/QAI.0000000000000174. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Additional file 1: (658.5KB, eps)

Example case script. (EPS 658 kb)

Additional file 2: (171.6KB, pdf)

Example Consent Form, HCW. (PDF 171 kb)

Additional file 3: (122KB, doc)

SPIRIT Checklist. (DOC 122 kb)

Data Availability Statement

Not applicable at this time


Articles from Trials are provided here courtesy of BMC

RESOURCES