Abstract
Objectives:
To determine whether continuous quality improvement (CQI) improves quality of HIV testing services for adolescents and young adults (AYA).
Design:
CQI was introduced at two HIV testing settings: Youth Centre and Voluntary Counseling and Testing (VCT) Center, at a national referral hospital in Nairobi, Kenya.
Methods:
Primary outcomes were AYA satisfaction with HIV testing services, intent to return, and accurate HIV prevention and transmission knowledge. Healthcare worker (HCW) satisfaction assessed staff morale. T tests and interrupted time series analysis using Prais–Winsten regression and generalized estimating equations accounting for temporal trends and autocorrelation were conducted.
Results:
There were 172 AYA (Youth Centre = 109, VCT = 63) during 6 baseline weeks and 702 (Youth Centre = 454, VCT = 248) during 24 intervention weeks. CQI was associated with an immediate increase in the proportion of AYA with accurate knowledge of HIV transmission at Youth Centre: 18 vs. 63% [adjusted risk difference (aRD) 0.42,95% confidence interval (CI) 0.21 to 0.63], and a trend at VCT: 38 vs. 72% (aRD 0.30, 95% CI −0.04 to 0.63). CQI was associated with an increase in the proportion of AYA with accurate HIV prevention knowledge in VCT: 46 vs. 61% (aRD 0.39, 95% CI 0.02–0.76), but not Youth Centre (P = 0.759). In VCT, CQI showed a trend towards increased intent to retest (4.0 vs. 4.3; aRD 0.78, 95% CI −0.11 to 1.67), but not at Youth Centre (P = 0.19). CQI was not associated with changes in AYA satisfaction, which was high during baseline and intervention at both clinics (P = 0.384, P = 0.755). HCW satisfaction remained high during intervention and baseline (P = 0.746).
Conclusion:
CQI improved AYA knowledge and did not negatively impact HCW satisfaction. Quality improvement interventions may be useful to improve adolescent-friendly service delivery.
Keywords: adolescent, Africa, HIV testing, HIV-1, prevention of sexual transmission, quality improvement
Introduction
Adolescents have the highest HIV incidence of any age group and are the only age group in which HIV-related mortality increased between 2005 and 2013 [1]. While substantial progress has been made in HIV testing and treatment for adult populations, there has been less programmatic focus on adolescents (aged 10–19 years) and young adults (aged 20–24 years) [2,3]. Eighty-three per cent of all adolescents living with HIV reside in sub-Saharan Africa (SSA), and yet just 9–13% of adolescent boys and girls in the region have tested for HIV in the past year [4–6].
Addressing adolescents and young adults’ (AYA) unique HIV needs requires effectively promoting HIV prevention behaviors, and also detecting and managing HIV infection early by building routine HIV testing habits. Healthcare workers (HCWs), however, report feeling uncomfortable providing services to AYA. Major concerns highlighted by HCW include lack of confidence in counseling and interpersonal communication [7]; AYA depression and mental health; concern about doing the wrong thing; and challenges handling disclosure, AYA autonomy, and consent [8].
Whereas some studies have focused on uptake of HIV testing among AYA populations [9], fewer have focused on improving the perceived quality of such services. Quality services, defined as being well tolerated, effective, patient-centered, timely, efficient, and equitable [10], are an essential component of the World Health Organization's (WHO) adolescent-friendly strategy [11,12]. Quality interventions not only result in improved clinical practice [13], but have also demonstrated results in patient outcomes, including HIV prevention behaviors [14].
Continuous quality improvement (CQI) is a method of process optimization that includes testing a series of small, iterative changes for impact on a system. It is a flexible process that is particularly useful in healthcare settings because it can address unique system challenges and test locally informed solutions. Employing CQI and directly measuring and reviewing impact on adolescents’ perceptions of service quality may provide an acceptable and low-risk method for service delivery improvement. CQI has a large body of evidence, supporting its effectiveness in creating sustainable improvements in healthcare delivery in both resource-rich and resource-limited countries [13,15–19].
This study – Developing Adolescent Strategies for HIV Testing (DASH) – focused on improving the quality of AYA HIV testing services (HTS) at a large referral hospital in Kenya by conducting a CQI intervention at youth HTS sites. Targeted quality outcomes included satisfaction with the HTS experience, intent to return for HTS within a year, and accurate knowledge of HIV prevention and transmission.
Methods
Study design
This implementation science study used time-series and pre/post analytic approaches to quantitatively test a CQI intervention [20] for AYA HTS. The time-series data included daily, individual-level, cross-sectional AYA indicators. The study period was October 2015–June 2016.
Setting
Activities were conducted in the urban setting of Nairobi, Kenya. Two clinics within Kenyatta National Hospital (KNH) were included – the Voluntary Counseling and Testing (VCT) Center and the Youth Center clinics.
Participants
All HTS HCWs aged at least 18 years were invited to participate in CQI activities and pre/post surveys. We aimed to recruit every AYA 14–24 years old who had completed HTS within the study period to participate in anonymous surveys. AYA newly diagnosed with HIV were escorted directly to the HIV care clinic following diagnosis, and were thus excluded.
Ethical considerations
The study was approved by the Kenyatta National Hospital Ethics and Research Committee (KNH ERC #P281/05/2015) and the University of Washington Institutional Review Board (UW IRB #48627). All potential participants were offered referral to the study by clinic staff; study staff assessed eligibility and consented participants. AYA at least 18 years and mature minors (14–17 years) provided oral informed consent. Adolescents aged 14–17 years who were accompanied by a caregiver provided assent, and the caregiver provided consent. A waiver of parental permission was granted for this minimal risk, anonymous survey for unaccompanied AYA. HCWs provided written informed consent.
Continuous quality improvement intervention
The intervention involved CQI training, supervision, and support to improve four HCW-identified primary indicators related to quality of AYA services through plan-do-study-act (PDSA) cycles following the Model for Improvement [21]. CQI leadership teams were formed at each site and a key driver diagram was jointly developed to determine areas for iterative, clinic-based changes. In December 2015, a 4-day in-person training introduced the Model for Improvement and its application. HCWs reviewed each clinic's baseline information for the CQI outcomes and received coaching to plan their initial PDSA cycles. Table 1 lists tested change concepts. HCWs met weekly to discuss tested changes using run charts and to plan new tests of change. Staff participated in three subsequent webinar trainings, and supervisors participated in a webinar on constructive feedback.
Table 1.
Example change concepts tested.
| Indicator targeted | Letter | Change concept tested | Outcome |
| HIV prevention and transmission knowledge | A | Clarify with HCW team that open mouth kissing is not a route of HIV transmission | Retained |
| B | Emphasize methods of prevention and transmission during post-test session when information may be better absorbed than during pre-test session | Retained | |
| C | Poster on wall with modes of prevention and routes of transmission | Retained | |
| D | Try harder to remember to discuss modes of prevention and transmission | Not retained | |
| E | Administer pre and post counseling questionnaire about modes of prevention and transmission to AYA | Not retained | |
| F | Confirm and correct AYA knowledge by asking AYA to teach the information back to the HCW | Retained | |
| Satisfaction | G | Play ‘youthful’ music at clinic; ask AYA what kinds of music they like | Retained |
| H | Provide brochure about full range of services available at Youth Center, not only HTS | Retained | |
| I | Use all possible rooms in clinic to conduct HTS to reduce waiting time | Retained | |
| J | Ask AYA how satisfied they feel with services | Not retained | |
| K | Provide warm welcome at reception with smile | Retained | |
| L | Provide AYA-friendly reading material in reception area | Retained | |
| M | Measure waiting time at clinic | Not retained | |
| N | Coordination between counselors and reception to reduce waiting time for HTS | Retained | |
| FP and STI discussion | O | Introduce FP and STI screening into counseling | Retained |
| P | Incorporate FP and STI screening into standardized operating procedures (SOP) | Retained | |
| Q | Call referral clinic to ensure client linked to FP and STI evaluation services | Not retained | |
| R | 3-question pregnancy prevention form placed with other HTS form as visual aid and cue | Retained | |
| Trainings and overall changes | S | HCW training about CQI methodology | |
| T | Leadership training about how to interpret run charts and give feedback | ||
| U | Use of cue cards to prompt all adopted changes | ||
| Intent to retest | No change concepts tested |
AYA, adolescents and young adults; CQI, continuous quality improvement; FP, family planning; HCW, healthcare worker; HTS, HIV testing services; STI, sexually transmitted infection.
Data collection
Baseline data were collected between October and December 2015. To distinguish between baseline and intervention phases, no data were included during the second week of December. The intervention period began on the third week of December 2015. Primary outcomes, measured daily among each enrolled AYA throughout the baseline and intervention periods, were: HIV prevention knowledge, HIV transmission knowledge, satisfaction with the HTS experience, and intent to return for HTS within 1 year. HIV prevention and transmission knowledge were assessed using modified multiple-choice questions from the Demographic and Health Surveys (DHS) question guide [22]. Accurate knowledge was defined as selecting all correct options and not selecting any incorrect options. AYA satisfaction and intent to retest were measured using 5-point Likert scales throughout the baseline and intervention periods. To identify unintended negative consequences of the intervention, pre/post-HCW surveys, using 13 adapted items from the Safety Attitudes and Safety Climate Questionnaire (SAQ) short form [23], were conducted to assess HCW satisfaction as a balancing measure. Data were collected anonymously using hand-held tablets with audio computer-assisted self-interview (ACASI) and Open Data Kit (ODK) [24] for adolescent indicators and using paper forms later entered into ODK for HCW satisfaction.
Two secondary outcomes were added later in the intervention period: whether AYA reported that the HCW had discussed family planning and sexually transmitted infections (STIs) other than HIV during the HTS session. These two indicators were restricted to AYA aged at least 15 years to align with Kenyan practice guidelines. The period of October 2015 through the first week of March 2016 was considered the baseline period for these two indicators, and the third week of March was the beginning of the intervention period. All data collection ended by June 2016.
Statistical analysis
Descriptive characteristics were summarized using counts, means, proportions, medians, and interquartile ranges (IQRs). Three analytic approaches were used to assess the change associated with the CQI intervention.
First, ‘risk difference’ was estimated using interrupted time series analysis [25], which accounts for temporal trends and uses individual level data. Prais–Winsten regression (linear regression that models first order autocorrelation) was used to estimate the adjusted risk difference (aRD) associated with the CQI intervention for both continuous and binary outcomes. Secondly, to compare binary outcomes in terms of ‘relative change’ associated with CQI, generalized estimating equations with a log link, binomial family, and first-order autoregressive correlation structure were used to calculate adjusted relative risk (aRR). All models included a binary ‘step change’ term to model the immediate change associated with the intervention, a linear term to account for baseline natural temporal trends, and an interaction term to allow the intervention period temporal trend to differ from the baseline temporal trend, modeling waxing and waning of the intervention. The immediate impact of the intervention (‘step change’) is the primary focus throughout the manuscript.
Finally, CQI run chart interpretation rules were used, which define improvement in terms of shifts (6 consecutive points above or below the baseline median) and trends (5 consecutive points sequentially increasing or decreasing) [26,27]. We considered a change ‘sustained’ if it did not return to median baseline levels.
The HCW satisfaction scores were collected before and after intervention from the same HCW team; they were compared using unpaired t tests due to unlinked pre and post responses. All analyses were conducted using Stata 14 IC (StataCorp, College Station, Texas, USA). All tests were two-sided with alpha of 0.05.
Results
During the 6 weeks of baseline data collection, 172 AYA (109 from Youth Centre, 63 from VCT) were enrolled. During the 24 weeks of the CQI intervention, 702 AYA (454 from Youth Centre, 248 from VCT) were enrolled. Among all AYA who received HTS at participating clinics, 64% were referred to study staff and 89% of those referred were enrolled. The three most common reasons for nonreferral were client being in a hurry (58%), clinic staff forgetting to mention the study (17%), and clients not being interested in the study (14%). Among those referred but not enrolled, the two most common reasons were client being in a hurry (47%) and client not being interested in the study (34%).
Sociodemographic characteristics of adolescents and young adults and healthcare workers
The median age among AYA was 21 years (IQR 20–23). The majority (562/874, 64%) were enrolled in or had completed university education, and half were men. The demographic characteristics of AYA presenting at the two sites were heterogeneous in terms of age and educational status, with Youth Centre attracting a younger age demographic (37% aged 14–19 years at Youth Centre, 20% at VCT). There were 24 HCWs enrolled, representing job cadres from counselors to data clerks to drug officers, and they had a median 9 years of experience in their roles (IQR 5–11) (Table 2).
Table 2.
Sociodemographic characteristics of adolescents and young adults and healthcare workers enrolled in CQI intervention at Kenyatta National Hospital, Nairobi, Kenya.
| Adolescents and Young Adults | Healthcare workersa (N = 24) | |||
| VCT clinic (N = 311) | Youth Center (N = 563) | |||
| n (%) or median (IQR) | n (%) or median (IQR) | n (%) or median (IQR) | ||
| Age | 21 (20, 23) | 21 (18, 22) | Years in current roleb | 9 (5, 11) |
| 14–19 | 62 (20) | 207 (37) | Position | |
| 20–24 | 249 (80) | 356 (63) | Counselor | 19 (79) |
| Male | 156 (50) | 258 (46) | Data analyst | 2 (8) |
| Highest level of education completed | Nurse | 1 (4) | ||
| Primary | 4 (1) | 27 (5) | Program officer | 1 (4) |
| Secondary | 44 (14) | 212 (38) | Drug officer | 1 (4) |
| Polytechnic | 11 (4) | 14 (2) | ||
| University/college | 252 (81) | 310 (55) | ||
| Currently enrolled in school | 273 (88) | 504 (90) | ||
CQI, continuous quality improvement; IQR, interquartile range; VCT, Voluntary Counseling and Testing.
aNot stratified by clinic.
bN = 21.
HIV and sexual and reproductive health seeking history
Most AYA (74%) presented to care specifically for HTS; however, 28% of AYA who received HTS at Youth Centre had originally presented for basic counseling services only. A majority (66%) of AYA had received HTS before. Few AYA had ever presented to clinic for family planning services (9%) or non-HIV STI testing (22%), and half (50%) were aware of where they could receive STI screening services. The majority of AYA (71%) stated intent to retest within 1 year, and AYA reported being highly likely to return to the same clinic for repeat services; a third of AYA had visited this clinic before (Table 3).
Table 3.
Adolescent HIV, STI, and family planning history among adolescents and young adults seeking HIV testing at VCT clinics at Kenyatta National Hospital, Nairobi, Kenya.
| Total | VCT | Youth Center | ||||
| N | Mean, n (%); median (IQR) | n | Mean, n (%); median (IQR) | n | Mean, n (%); median (IQR) | |
| Reason for coming to VCT | 802 | 280 | 522 | |||
| HIV test | 591 (74) | 254 (91) | 337 (65) | |||
| Testing for non-HIV STIs | 8 (1) | 3 (1) | 5 (1) | |||
| General health information or counseling | 160 (20) | 14 (5) | 146 (28) | |||
| Family planning or pregnancy test | 4 (1) | 4 (1) | ||||
| Other | 39 (5) | 9 (3) | 30 (6) | |||
| First time visiting this clinic | 802 | 510 (64) | 280 | 186 (66) | 522 | 324 (62) |
| First HIV test | 802 | 273 (34) | 280 | 89 (32) | 522 | 184 (35) |
| Felt that healthcare worker gave sufficient information about HIV prevention | 874 | 788 (90) | 311 | 284 (91) | 563 | 504 (90) |
| Sexual and reproductive health access (age ≥15) | ||||||
| Ever gone to a clinic for FP | 854 | 78 (9) | 311 | 22 (7) | 543 | 56 (10) |
| Ever gone to a clinic for STI testing | 854 | 186 (22) | 311 | 67 (22) | 543 | 119 (22) |
| Know of any clinic offering STI testing | 854 | 426 (50) | 311 | 156 (50) | 543 | 270 (50) |
| HIV retesting | ||||||
| Timing for intended retest | 874 | 311 | 563 | |||
| Less than 6 months from today | 489 (56) | 181 (58) | 308 (55) | |||
| 7–12 months from today | 131 (15) | 50 (16) | 81 (14) | |||
| More than 1 year from today | 55 (6) | 21 (7) | 34 (6) | |||
| When I have a new partner | 75 (9) | 28 (9) | 47 (8) | |||
| I don’t know | 124 (14) | 31 (10) | 93 (17) | |||
| Likelihood of returning to this clinic for HIV retest (1–5) | 874 | 4.3 | 311 | 4.3 | 563 | 4.3 |
FP, family planning; IQR, interquartile range; STI, sexually transmitted infection; VCT, Voluntary Counseling and Testing.
HIV knowledge
At baseline, AYA had low HIV prevention and transmission knowledge (46 and 38% with full and accurate prevention and transmission knowledge, respectively, at VCT; 48 and 18% at Youth Centre) (Table 4). The most common HIV prevention misconceptions were that HIV could not be prevented through either abstinence or through having one faithful, HIV-negative partner; among those AYA with inaccurate knowledge, 67 and 54% failed to identify these prevention techniques, respectively. The most common HIV transmission misconception was that HIV could be transmitted through ‘open mouth kissing’; among those AYA with inaccurate knowledge, 94% answered this question inaccurately.
Table 4.
Impact of CQI intervention on adolescent and healthcare worker outcomes.
| Baseline | Intervention | ||||||||||||
| N | Mean, n (%) | N | Mean, n (%) | aRD | 95% CI | P value | aRR | 95% CI | P value | ||||
| VCT | |||||||||||||
| Change associated with CQI | −0.08 | −0.59 | 0.43 | 0.755 | |||||||||
| Satisfaction with testing visit (1–5) | 63 | 4.5 | 248 | 4.5 | Time trend during baseline (per week) | 0.06 | −0.07 | 0.19 | 0.380 | – | |||
| Change in time trend during intervention (per week) | −0.07 | −0.20 | 0.07 | 0.326 | |||||||||
| Change associated with CQI | 0.30 | −0.04 | 0.63 | 0.080 | 1.72 | 0.76 | 3.90 | 0.194 | |||||
| Correct knowledge of HIV transmission | 63 | 24 (38) | 248 | 178 (72) | Time trend during baseline (per week) | 0.01 | −0.07 | 0.10 | 0.773 | 1.00 | 0.97 | 1.04 | 0.787 |
| Change in time trend during intervention (per week) | −0.01 | −0.10 | 0.07 | 0.763 | 1.00 | 0.96 | 1.03 | 0.781 | |||||
| Change associated with CQI | 0.39 | 0.02 | 0.76 | 0.038 | 2.17 | 0.93 | 5.04 | 0.071 | |||||
| Correct knowledge of HIV prevention | 63 | 29 (46) | 248 | 151 (61) | Time trend during baseline (per week) | −0.04 | −0.14 | 0.06 | 0.411 | 0.99 | 0.96 | 1.02 | 0.396 |
| Change in time trend during intervention (per week) | 0.03 | −0.07 | 0.13 | 0.538 | 1.01 | 0.98 | 1.04 | 0.486 | |||||
| Change associated with CQI | 0.78 | −0.11 | 1.67 | 0.087 | |||||||||
| Intent to retest within 1 year (1–5) | 63 | 4.0 | 248 | 4.3 | Time trend during baseline (per week) | −0.12 | −0.36 | 0.11 | 0.299 | – | |||
| Change in time trend during intervention (per week) | 0.12 | −0.12 | 0.35 | 0.336 | |||||||||
| Change associated with CQI | −0.13 | −0.35 | 0.09 | 0.238 | 0.71 | 0.33 | 1.54 | 0.393 | |||||
| Provider discussed FP (adolescents age >15) | 160 | 29 (18) | 104 | 48 (46) | Time trend during baseline (per week) | 0.02 | 0.00 | 0.03 | 0.026 | 1.01 | 1.00 | 1.03 | 0.033 |
| Change in time trend during intervention (per week) | 0.03 | 0.00 | 0.06 | 0.048 | 1.00 | 0.98 | 1.01 | 0.838 | |||||
| Change associated with CQI | −0.06 | −0.29 | 0.17 | 0.595 | 0.86 | 0.50 | 1.46 | 0.568 | |||||
| Provider discussed STI (adolescents age >15) | 160 | 61 (38) | 104 | 57 (55) | Time trend during baseline (per week) | 0.01 | 0.00 | 0.03 | 0.158 | 1.01 | 1.00 | 1.01 | 0.131 |
| Change in time trend during intervention (per week) | 0.01 | −0.02 | 0.04 | 0.432 | 1.00 | 0.99 | 1.01 | 0.984 | |||||
| Youth Center | |||||||||||||
| Change associated with CQI | 0.12 | −0.16 | 0.40 | 0.384 | |||||||||
| Satisfaction with testing visit (1–5) | 109 | 4.5 | 454 | 4.5 | Time trend during baseline (per week) | −0.03 | −0.10 | 0.05 | 0.499 | – | |||
| Change in time trend during intervention (per week) | 0.02 | −0.05 | 0.09 | 0.606 | |||||||||
| Change associated with CQI | 0.42 | 0.21 | 0.63 | <0.001 | 2.27 | 1.05 | 4.94 | 0.038 | |||||
| Correct knowledge of HIV transmission | 109 | 20 (18) | 454 | 289 (63) | Time trend during baseline (per week) | 0.03 | −0.02 | 0.09 | 0.221 | 1.03 | 0.99 | 1.07 | 0.148 |
| Change in time trend during intervention (per week) | −0.04 | −0.10 | 0.01 | 0.147 | 0.97 | 0.93 | 1.01 | 0.127 | |||||
| Change associated with CQI | −0.03 | −0.26 | 0.19 | 0.759 | 0.95 | 0.59 | 1.53 | 0.826 | |||||
| Correct knowledge of HIV prevention | 109 | 52 (48) | 454 | 242 (53) | Time trend during baseline (per week) | 0.00 | −0.06 | 0.06 | 0.931 | 1.00 | 0.98 | 1.02 | 0.933 |
| Change in time trend during intervention (per week) | 0.01 | −0.05 | 0.07 | 0.731 | 1.00 | 0.98 | 1.02 | 0.767 | |||||
| Change associated with CQI | 0.39 | −0.19 | 0.96 | 0.185 | |||||||||
| Intent to retest within 1 year (1–5) | 109 | 4.0 | 454 | 4.2 | Time trend during baseline (per week) | −0.06 | −0.21 | 0.09 | 0.446 | – | |||
| Change in time trend during intervention (per week) | 0.06 | −0.10 | 0.21 | 0.460 | |||||||||
| Change associated with CQI | 0.11 | −0.06 | 0.28 | 0.188 | 1.95 | 0.99 | 3.83 | 0.053 | |||||
| Provider discussed FP (adolescents age >15) | 322 | 53 (16) | 158 | 77 (49) | Time trend during baseline (per week) | 0.00 | −0.01 | 0.01 | 0.669 | 1.00 | 0.99 | 1.01 | 0.803 |
| Change in time trend during intervention (per week) | 0.03 | 0.01 | 0.05 | 0.011 | 1.01 | 1.00 | 1.02 | 0.091 | |||||
| Change associated with CQI | 0.00 | −0.18 | 0.18 | 0.994 | 1.18 | 0.84 | 1.65 | 0.330 | |||||
| Provider discussed STI (adolescents age >15) | 322 | 145 (45) | 158 | 110 (70) | Time trend during baseline (per week) | 0.01 | −0.01 | 0.02 | 0.308 | 1.00 | 1.00 | 1.00 | 0.908 |
| Change in time trend during intervention (per week) | 0.03 | 0.00 | 0.05 | 0.025 | 1.00 | 1.00 | 1.01 | 0.128 | |||||
| Healthcare worker outcomes at VCT and Youth Center combineda | |||||||||||||
| Overall satisfaction with job environment (1–5) | 22 | 4.2 | 22 | 4.2 | Change associated with CQIa | −0.04 | −0.28 | 0.20 | 0.746 | – | – | – | – |
aRD, adjusted risk difference; aRR, adjusted relative risk; CQI, continuous quality improvement; FP, family planning; STI, sexually transmitted infection; VCT, Voluntary Counseling and Testing.
aUnpaired t test, no temporal trend adjustment made.
Bold values in the table indicate P-values <0.05 (statistically significant values). Bold italics values in the table indicate P-values >0.05 and <0.10 (trend towards significance).
Impact of continuous quality improvement on adolescents and young adults’ HIV knowledge
Controlling for temporal trends, CQI was associated with an immediate increase in accurate HIV prevention knowledge at VCT, but not Youth Centre, and increases in accurate HIV transmission knowledge at Youth Centre and a trend towards an increase at VCT. At VCT, accurate HIV prevention knowledge increased from 46% during baseline to 61% during the intervention [aRD 39 percentage points, 95% confidence interval (CI) 2 to 76, P = 0.038; aRR 2.17, 95% CI 0.93 to 5.04, P = 0.071]. At Youth Centre, the change in HIV prevention knowledge was not significant (P = 0.759). At Youth Centre, accurate HIV transmission knowledge increased from 18 to 63% (aRD 42 percentage points, 95% CI 21 to 63, P < 0.001; aRR 2.27, 95% CI 1.05 to 4.94, P = 0.038); at VCT, accurate transmission knowledge trended towards an increase from 38 to 72% (aRD 30 percentage points, 95% CI −4 to 63, P = 0.080; aRR 1.72, 95% CI 0.76 to 3.90, P = 0.194) (Table 4).
Satisfaction and intent to retest
At Youth Centre, intent to return for HTS within a year was high and did not change significantly between baseline and intervention (4.0 vs. 4.2, P = 0.185). At VCT, intent to retest trended towards an increase (4.0 vs. 4.3, P = 0.087). In both Youth Centre and VCT, satisfaction with testing was high during both baseline and intervention periods, and there was no change over time (Youth Centre: 4.5 vs. 4.5, P = 0.384; VCT: 4.5 vs. 4.5, P = 0.755) (Table 4).
Reported discussion of family planning and sexually transmitted infection services
The proportion of HCW that AYA reported having discussed family planning services increased at both Youth Centre and VCT (Youth Centre: from 16 to 49%; VCT: from 18 to 46%). Adjusted differences trended towards significance in Youth Centre in relative change, but not absolute change (aRD 11 percentage points, 95% CI −6 to 28, P = 0.188; aRR 1.95, 95% CI 0.99 to 3.83, P = 0.053), but not at VCT (P = 0.238). Proportions of HCW that AYA reported having discussed STI testing increased at both Youth Centre and VCT (Youth Centre: from 45 to 70%; VCT: from 38 to 55%), but adjusted differences were insignificant at both clinics (Table 4).
Changes in effect of intervention over time
The magnitude of the change associated with CQI grew significantly over time for three indicators: reported discussion of family planning services at both VCT and Youth Centre and reported discussion of STI at Youth Centre. For all other indicators, the change associated with CQI did not increase or decrease significantly over time (Table 4).
Run chart findings
The CQI run chart interpretation rules [26,27] had similar conclusions to interrupted time-series analyses, and also showed substantial variability in the indicators throughout the intervention period (Fig. 1a–d). There was a substantial and sustained shift in HIV transmission knowledge at Youth Centre with a 24-point shift sustained through the end of the intervention period (Fig. 1a). At VCT, there was an 8-point shift in HIV prevention knowledge, which was not sustained (Fig. 1b). At Youth Centre, there was a 6-point shift in HIV prevention knowledge midway through the intervention period that was not sustained (data not shown). There were no shifts or trends in either satisfaction (Fig. 1c) or intent to retest at either site (data not shown). There was a 12-point shift in adolescents who reported that HCW at Youth Centre discussed family planning that was sustained through the end of the intervention period (Fig. 1d), and there was a 9-point shift in adolescents who reported HCW at Youth Centre discussed STIs that was sustained through the end of the intervention (data not shown). Other run chart findings of significance, but not shown, include the following: a substantial and sustained 22-point shift in HIV transmission knowledge at VCT; a 7-point shift in reported HCW discussion of family planning at VCT that was sustained through the end of the intervention aside from one astronomical data point; a 5-point trend embedded in a 9-point shift in the reported HCW discussion of STIs at VCT, which was sustained to the end of the intervention.
Fig. 1.
Selected illustrative run charts.
Panel a: CQI initially improves HIV transmission knowledge indicator, but initial success is followed by some regression to lower levels. Renewed effort with a more reliable change concept results in substantial and sustained increase in indicator. Panel b: Baseline performance is erratic and unreliable; CQI process has early success in increasing performance in HIV prevention knowledge, but indicator returns to baseline levels over time. Panel c: CQI has no impact on adolescent satisfaction, which began and remained high throughout the baseline and intervention periods. Panel d: Discussion of family planning services is low during baseline; CQI process improves this secondary outcome. Letters A through U refer to change concepts and trainings described in Table 1.
Healthcare worker satisfaction with their job environment
Healthcare workers rated their overall satisfaction with their jobs as 4.2/5 at baseline and 4.2/5 during the intervention (P = 0.746). None of the 13 components of job satisfaction changed substantially or significantly (P > 0.1 for all comparisons).
Discussion
The DASH implementation science study found that CQI at youth HTS clinics in Kenya was associated with substantially improved AYA HIV prevention and transmission knowledge, and a trend towards improved intent to retest at one clinic. Additionally, AYA satisfaction with the HTS visit experience and intent to retest both began and remained high throughout the study period. CQI did not negatively impact HCW satisfaction.
As Joint United Nations Programme on HIV/AIDS (UNAIDS) and President's Emergency Plan for AIDS Relief (PEPFAR) prioritize reaching the 90-90-90 goals for AYA and reducing AYA HIV incidence [28], it is increasingly important to identify innovative approaches to improve the quality of AYA HIV services. Gaps in achieving AYA HIV goals are not due to absence of HIV testing and treatment commodities or providers, but rather in the quality of services offered [1]. However, improving service quality, as opposed to increasing service quantity, can be challenging; perceptions of quality are difficult to measure and have many underlying influencing factors. Historically, there have been few interventions that have resulted in robust improvements in AYA health service quality in resource-limited settings [29]. However, this study joins a growing, but mixed, body of literature that describes interventions – primarily large, national-level policy changes – that improve the quality of AYA services [30–33].
This is the first study to our knowledge to systematically evaluate CQI in an adolescent HTS setting aimed at improving quality of services. This study is unique in that it was able to observe changes in adolescent understanding of HIV prevention and transmission information, and also intent to retest – indicators that are closer to health impact than HCW-controlled process indicators. Additionally, CQI is a generalizable process that allows for locally informed changes to address unique issues at the clinic level. This intervention has potential for scalability to address AYA HIV prevention and management goals.
The study observed low levels of AYA HIV knowledge during baseline. AYA HIV knowledge gaps persist worldwide and may contribute to high HIV incidence. The gaps observed in the DASH study differed somewhat from the results from the Kenyan DHS surveys; condom knowledge, mother-to-child transmission knowledge, and mosquito transmission knowledge were higher among our study population than among age and sex-specific levels nationally, whereas abstinence and having one monogamous, uninfected partner knowledge were lower than national averages [34]. This may be due to slightly different wording of questions (e.g. ‘prevent HIV’ versus ‘reduce risk of HIV’) and high levels of education among our urban population of AYA. In settings where access to comprehensive sexual education is not widely available, the HTS visit may serve a critical role in providing accurate HIV knowledge for AYA. This study also observed low levels of historic adolescent engagement with family planning and STI services, even among a population of AYA who self-selected to present for HTS. HTS may be an important entry point for sexually active AYA to access comprehensive sexual and reproductive health services.
After large and sustained improvement had been observed in the original HIV knowledge indicators, family planning and STI indicators were added as secondary outcomes; it is common in CQI that once initial goals are achieved, new goals are addressed. However, conclusions about the impact of CQI on the family planning and STI indicators were mixed. Crude differences in proportions and traditional run chart interpretation rules all suggested a large intervention impact on these two indicators. However, analyses that modeled the temporal trends during the baseline and intervention periods did not suggest an immediate impact of the intervention, but rather noted that the intervention improved these indicators over time.
The success of the CQI intervention in this clinic setting may have been due to a well primed environment for change. The AYA focus of this intervention was directly informed by previous qualitative work with this clinical team in which HCW noted that they felt ill prepared to address the unique and challenging health needs of AYA seeking HTS [8]. Team members were directly involved in developing the CQI intervention and in determining the indicators targeted for change. In addition to a highly motivated clinical team, this intervention also benefitted from strong leadership buy-in and direct involvement in supervision of the CQI process. The main challenge faced during the intervention was maintaining morale and momentum for change; others have observed that routinely identifying errors to change service delivery is challenging when there is judgment attached to poor HCW performance [35]. Substantial effort was made to frame the tone of weekly data reviews with HCW to focus on achievements, rather than absence of change.
The study benefitted from a large sample of AYA seeking HTS in a public hospital, high levels of data completeness, and robust analytic methods. To limit social desirability and recall bias, AYA entered responses using ACASI and completed surveys immediately after the HTS session. There were limitations, however, in certain aspects of design, population, and analysis. There remained a high level of variability in several indicators, and there were AYA who were not approached and not referred to the study. It is possible that there was differential referral, though it would not be expected to vary over time and therefore would not be expected to affect the relative change observed. Our sample is limited in generalizability, as the majority were educated, HIV-negative, young adults attending a tertiary facility, who may not have the same experiences as younger or HIV-infected adolescents in other settings. A pre-post design is not as robust as a design that employs a concurrent control group; however, interrupted time-series analysis is a rigorous analytic method that accounts for both autocorrelation and temporal trends and generally results in more valid estimates than pre-post analysis [25]. Additionally, this analytic technique tended to have similar conclusions to CQI analytic methods for run chart interpretation, increasing the strength of study findings.
Continuous quality improvement offers a team-driven approach for improving quality of care. The staff ownership of the improvement process and adaptability of the intervention to address future quality outcomes were considered strengths. However, our intervention included dedicated staff to assist with data collection and analysis, which would be difficult in typical public health facilities in resource-limited settings. CQI has limitations in complexity and cost, and may be comparably effective to training interventions for targeted change such as knowledge improvement. Additional studies are needed to determine in which environments this intervention is most effective, what kinds of pre-CQI activities are needed to prime a site to be ready for change, and what the long-term sustainability of CQI is in resource-limited settings in the absence of additional resources to support the CQI process.
Conclusion
In summary, this CQI intervention was associated with substantial, significant, and sustained changes in AYA knowledge of HIV prevention and transmission. AYA satisfaction and intent to retest for HIV within a year began and remained high throughout the intervention period, and CQI did not have a detrimental effect on HCW satisfaction. CQI may be a valuable intervention for effecting locally-informed, flexible change in the quality of HIV service delivery for AYA in SSA.
Acknowledgements
Author contributions: A.D.W., C.M., S.B.M., P.M.M., D.C.W., D.B., I.N.N., G.O., G.J.S., J.A.S., and P.K.K. conceptualized the study design. A.D.W., C.M., S.B.M., P.M.M., D.C.W., D.B., J.N., I.N.N., G.O., J.A.S., and P.K.K. led development of the study materials. C.M., D.C.W., J.N., and A.D.W. oversaw data collection. A.D.W. conducted the analysis with input from S.B.M., J.N., G.J.S. J.A.S., and P.K.K. A.D.W. drafted the manuscript and all authors contributed to the final manuscript.
The authors thank the DASH study participants and their families, without whom this research would not be possible. The authors also thank the Kenyatta National Hospital staff at the VCT, Youth Center, and tents for their tremendous effort in the CQI process and in recruiting study participants. The authors thank the DASH administrative and data team for their dedication and support; and the Kenyan National AIDS and STI Control Programme (NASCOP) for valuable input during study design, conduct, and dissemination. Lastly, the authors thank members of the Kizazi Working Group (UW Global Center for Integrated Health of Women, Adolescents and Children (Global WACh)) and Kenya Research & Training Center (KRTC) for their support during the preparation of this article.
Research reported in this publication was supported by NIAID, NCI, NIMH, NIDA, NICHD, NHLBI, NIA, NIGMS, NIDDK of the National Institutes of Health under award number AI027757 and by a supplement to this award. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. Funding sources were not involved in the analyses or interpretation of data.
Conflicts of interest
The authors have no conflicts of interest.
References
- 1.(UNAIDS) Joint United Nations Programme on HIV/AIDS. Global Report: UNAIDS report on the global AIDS epidemic 2013. 2013. http://files.unaids.org/en/media/unaids/contentassets/documents/epidemiology/2013/gr2013/UNAIDS_Global_Report_2013_en.pdf. [Google Scholar]
- 2.Kasedde S, Kapogiannis BG, McClure C, Luo C. Executive summary: opportunities for action and impact to address HIV and AIDS in adolescents. J Acquir Immune Defic Syndr 2014; 66 Suppl 2:S139–S143. [DOI] [PubMed] [Google Scholar]
- 3.Idele P, Gillespie A, Porth T, Suzuki C, Mahy M, Kasedde S, et al. Epidemiology of HIV and AIDS among adolescents: current status, inequities, and data gaps. J Acquir Immune Defic Syndr 2014; 66 Suppl 2:S144–S153. [DOI] [PubMed] [Google Scholar]
- 4.(WHO) World Health Organization. ALL IN: #EndAdolescentAIDS. In; 2015. http://www.unaids.org/sites/default/files/media_asset/20150217_ALL_IN_brochure.pdf. [Google Scholar]
- 5.(WHO) World Health Organization. HIV and Adolescents: Guidance for HIV Testing and Counselling and Care for Adolescents Living With HIV. 2013. http://www.who.int/hiv/pub/guidelines/adolescents/en/. [Google Scholar]
- 6.(UNICEF) United Nations International Children's Emergency Fund. Turning the tide against AIDS will require more concentrated focus on adolescents and young people. https://data.unicef.org/topic/hivaids/adolescents-young-people/. [Google Scholar]
- 7.Godia PM, Olenja JM, Lavussa JA, Quinney D, Hofman JJ, van den Broek N. Sexual reproductive health service provision to young people in Kenya; health service providers’ experiences. BMC Health Serv Res 2013; 13:476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wagner A, O’Malley G, Firdawsi O, Mugo C, Njuguna I, Maleche-Obimbo E, Inwani I, et al. HIV testing for older children: a mixed-methods study examining challenges in decision to test, testing process, and coping posttesting. In: 21st International AIDS Conference. Durban, South Africa; 2016. [Google Scholar]
- 9.Govindasamy D, Ferrand RA, Wilmore SM, Ford N, Ahmed S, Afnan-Holmes H, et al. Uptake and yield of HIV testing and counselling among children and adolescents in sub-Saharan Africa: a systematic review. J Int AIDS Soc 2015; 18:20182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.(IOM) IoM. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001. [PubMed] [Google Scholar]
- 11.(WHO) World Health Organization. Adolescent friendly health services: An agenda for change. 2002. http://apps.who.int/iris/bitstream/10665/67923/1/WHO_FCH_CAH_02.14.pdf [Google Scholar]
- 12.(WHO) World Health Organization. Making health services adolescent friendly. 2012. http://www.who.int/maternal_child_adolescent/documents/adolescent_friendly_services/en/ [Google Scholar]
- 13.Leatherman S, Ferris TG, Berwick D, Omaswa F, Crisp N. The role of quality improvement in strengthening health systems in developing countries. Int J Qual Healthcare 2010; 22:237–243. [DOI] [PubMed] [Google Scholar]
- 14.Chopra M, Doherty T, Jackson D, Ashworth A. Preventing HIV transmission to children: quality of counselling of mothers in South Africa. Acta Paediatr 2005; 94:357–363. [DOI] [PubMed] [Google Scholar]
- 15.Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008; 336:1491–1494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kohler PK, Namate D, Barnhart S, Chimbwandira F, Tippet-Barr BA, Perdue T, et al. Classification and rates of adverse events in a Malawi male circumcision program: impact of quality improvement training. BMC Health Serv Res 2016; 16:61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014; 23:290–298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Twum-Danso NA, Dasoberi IN, Amenga-Etego IA, Adondiwo A, Kanyoke E, Boadu RO, et al. Using quality improvement methods to test and scale up a new national policy on early postnatal care in Ghana. Health Policy Plan 2014; 29:622–632. [DOI] [PubMed] [Google Scholar]
- 19.Rosen MA, Chima AM, Sampson JB, Jackson EV, Jr, Koka R, Marx MK, et al. Engaging staff to improve quality and safety in an austere medical environment: a case-control study in two Sierra Leonean hospitals. Int J Qual Healthcare 2015; 27:320–327. [DOI] [PubMed] [Google Scholar]
- 20.Langley GL, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A practical Approach to Enhancing Organizational Performance. 2nd ed.San Francisco: Jossey-Bass; 2009. [Google Scholar]
- 21.(IHI) Institute for Healthcare Improvement. Science of Improvement: How to Improve. In: Institute for Healthcare Improvement (IHI). http://www.ihi.org/resources/pages/howtoimprove/scienceofimprovementhowtoimprove.aspx. [Google Scholar]
- 22.USAID. Questionnaires: Household, Woman's and Man's: Demographic and Health Surveys Methodology. Calverton, Maryland: USAID; 2011. [Google Scholar]
- 23.Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006; 6:44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Carl Hartung YA, Brunette W, Lerer A, Tseng C, Borriello G. Open Data Kit: Tools to Build Information Services for Developing Regions. 1st ed: ICTD; 2010. [Google Scholar]
- 25.Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating healthcare quality improvements. Acad Pediatr 2013; 13:S38–S44. [DOI] [PubMed] [Google Scholar]
- 26.(IHI) Institute for Healthcare Improvement. Run Chart Tool. In: Institute for Healthcare Improvement (IHI). http://www.ihi.org/resources/pages/tools/runchart.aspx. [Google Scholar]
- 27.Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf 2011; 20:46–51. [DOI] [PubMed] [Google Scholar]
- 28.UNAIDS. 90-90-90: An Ambitious Treatment Target to Help End the AIDS Epidemic. Geneva, Switzerland; 2014. http://www.unaids.org/en/resources/documents/2017/90-90-90. [Google Scholar]
- 29.Tylee A, Haller DM, Graham T, Churchill R, Sanci LA. Youth-friendly primary-care services: how are we doing and what more needs to be done?. Lancet 2007; 369:1565–1573. [DOI] [PubMed] [Google Scholar]
- 30.Dickson KE, Ashton J, Smith JM. Does setting adolescent-friendly standards improve the quality of care in clinics? Evidence from South Africa. Int J Qual Healthcare 2007; 19:80–89. [DOI] [PubMed] [Google Scholar]
- 31.Mathews C, Guttmacher SJ, Flisher AJ, Mtshizana YY, Nelson T, McCarthy J, et al. The quality of HIV testing services for adolescents in Cape Town, South Africa: do adolescent-friendly services make a difference?. J Adolesc Health 2009; 44:188–190. [DOI] [PubMed] [Google Scholar]
- 32.Chandra-Mouli V, Chatterjee S, Bose K. Do efforts to standardize, assess and improve the quality of health service provision to adolescents by government-run health services in low and middle income countries, lead to improvements in service-quality and service-utilization by adolescents?. Reprod Health 2016; 13:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Denno DM, Hoopes AJ, Chandra-Mouli V. Effective strategies to provide adolescent sexual and reproductive health services and to increase demand and community support. J Adolesc Health 2015; 56:S22–41. [DOI] [PubMed] [Google Scholar]
- 34.Kenyan National Bureau of Statistics. 2014 Kenya Demographic and Health Survey. 2015. https://dhsprogram.com/pubs/pdf/fr308/fr308.pdf. [Google Scholar]
- 35.Lederman R, Dreyfus S, Matchan J, Knott JC, Milton SK. Electronic error-reporting systems: a case study into the impact on nurse reporting of medical errors. Nurs Outlook 2013; 61:417–426. e415. [DOI] [PubMed] [Google Scholar]