Cascade analysis: an adaptable implementation strategy across HIV and non-HIV delivery platforms

Abstract

Background

Cascades have been used to characterize sequential steps within a complex health system and are used in diverse disease areas and across prevention, testing, and treatment. Routine data has great potential to inform prioritization within a system, but is often inaccessible to frontline health care workers (HCW) who may have the greatest opportunity to innovate health system improvement.

Methods

The cascade analysis tool (CAT) is an Excel-based, simple simulation model with an optimization function. It identifies the step within a cascade that could most improve the system. The original CAT were developed for HIV treatment and the prevention of mother-to-child transmission of HIV (PMTCT).

Results

CAT has been adapted 7 times: to a mobile application for PMTCT; for hypertension screening and management and for mental health outpatient services in Mozambique; for pediatric & adolescent HIV testing and treatment, HIV testing in family planning, and cervical cancer screening and treatment in Kenya; and for naloxone distribution and opioid overdose reversal in the United States. The main domains of adaptation have been technical—estimating denominators and structuring steps to be binary sequential steps—as well as logistical—identifying acceptable approaches for data abstraction and aggregation, and not overburdening HCW.

Discussion

CAT allows for prompt feedback to HCW, increases HCW autonomy, and allows managers to allocate resources and time in an equitable manner. CAT is an effective, feasible, and acceptable implementation strategy to prioritize areas most requiring improvement within complex health systems, although adaptations are being currently evaluated.

INTRODUCTION

Data-driven optimization of health system efficiency is critical to sustainably improving service delivery. In low-resource health systems, data-generating structures are frequently built to report specific indicators “up”—to health authorities and donors. This often comes at the expense of frontline health care workers (HCWs) using routine data to guide contextually-appropriate strategies to improve service delivery [1]. Engaging HCW to use routine program data to inform changes can make this time-intensive activity worthwhile to HCW and improve data quality. Routine program data has great potential to inform data-driven optimization, though with potential quality limitations [2,3].

Cascade analysis is an implementation strategy that uses data to visualize and quantify system performance, inefficiencies, and bottlenecks. Cascades are characterized by a series of sequential, conditional steps in a process that yield a desired outcome, and explicitly model “handoffs” between and across complex, interdependent system components. For example, the prevention of mother-to-child HIV transmission (PMTCT) cascade involves HIV testing, linkage to care, antiretroviral therapy (ART) for HIV-positive mothers, antiretroviral prophylaxis (ARV) for infants, and repeated infant testing. Cascade analysis has informed numerous disease models and goal setting (e.g. UNAIDS 90–90-90 goals [4]), identified gaps in system performance (e.g. PMTCT [5,6] and HIV prevention cascades [7,8]), and helped prioritization of areas where improvement is most needed. Cascade analysis facilitates thinking across the entire care system to determine which steps are the most inefficient and where to focus improvement efforts to maximize impact.

Despite effectively identifying system inefficiencies, cascade analysis does not improve system performance unless paired with effective improvement strategies. The Systems Analysis and Improvement Approach (SAIA) is a package of implementation strategies that pairs a novel cascade analysis tool (CAT) with established implementation strategies for improvement—process mapping and continuous quality improvement (CQI)—to support frontline HCWs and managers who lead efforts to diagnose inefficiencies and guide iterative improvements in complex delivery systems [9]. In a three-country cluster randomized trial, SAIA reduced drop-offs in the PMTCT cascade [10]. CAT saved HCW time, enabled prompt feedback for self-assessment and motivation, increased program ownership, and guided managers to equitably allocate resources (including time).

In this paper we describe CAT adaptations for use within HIV and beyond. We define CAT and its components; identify processes, systems, and data sources well and poorly suited to CAT; describe seven adaptations of CAT; and identify gaps and opportunities in CAT adaptation and implementation for systems improvement.

METHODS

CAT structure and mathematical logic

CAT is intended to characterize present performance of a system and then aid in the relative prioritization of improvement opportunities. CAT visualizes cascades as a series of sequential, conditional, binary steps. It is a basic, Excel-based simulation model with an optimization function, using inputs from routine program data to identify the cascade step with the maximum opportunity for improvement. A cascade “step” may be either service receipt (e.g. HIV testing) or behavior (e.g. adherence).

We developed the first CAT in 2004 to model the HIV treatment cascade in Mozambique [11]. This CAT highlighted discrete cascade steps—(1) HIV testing, (2) ART clinic enrollment, (3) CD4 testing, (4) ART initiation, and (5) ART adherence—to prioritize time and resource allocation. CAT quantifies the number of people lost at each step (those eligible for but not completing each step) and the additional number of patients who would complete all five steps if an individual step achieved perfect performance while the performance of all other steps remained constant—what we call “cascade gain” [11].

This early CAT was adapted to create the “PCAT” (PMTCT Cascade Analysis Tool) [12,13] used in the original SAIA trial. The PCAT presented aggregated data from cross-sectional, month-long periods abstracted from existing registers showing mother-infant dyads. It did not follow specific woman-infant dyads through each step, but rather reflected dyads who completed each step relative to the number who were expected to, based on upstream steps during the same period. The PCAT is more complex than the initial CAT, as PMTCT includes two cascades—antenatal care (ANC) initiation through newborn ARV prophylaxis (Cascade 1), and enrollment of HIV-exposed infants in care through ART initiation for HIV-infected infants (Cascade 2) (Figure 1). Green cells represent data that is entered from PMTCT registers or demographic birth estimates, while white cells represent automatic calculations. For each cascade step, the PCAT shows the number (column A) and proportion (column B) completing that step, number not completing (column C), and the potential “cascade gain” if a given step were to perform optimally (column D).

Figure 1:

The prevention of mother-to-child transmission of HIV cascade analysis tool (CAT). Cascade 1 covers antenatal care (ANC) through postpartum prophylaxis (PPO). Cascade 2 covers at-risk child visits through antiretroviral therapy (ART) initiation for HIV-positive infants.

Cascade gain enumerates dyads that would successfully complete the full cascade if a given step performed perfectly, absent any changes to the remaining steps. In the example in Figure 1, if all mothers in ANC received HIV testing (improvement from 70% to 100%), without downstream improvements, we would expect another 127 HIV-exposed infants to successfully start HIV prophylaxis. The cascade gain column is used for relative prioritization when there are multiple opportunities for system improvement and insufficient resources to address them all simultaneously. For example, 30% gaps in infant linkage to care (step 5) and infant testing (step 6) could intuitively be seen as equally important; additionally, infant testing appears more proximal to the desired outcome than linkage. However, the cascade gain for infant linkage to care is greater, thus HCWs should prioritize this step for optimal systems improvement in settings with limited time and resources.

Although the calculations are relatively simple, cascade gain—a series of conditional probabilities multiplied by one another and applied to the absolute number of individuals who did not complete the step—is the most novel and challenging concept within CAT. Maintaining the example from Figure 1, 670 women missed HIV testing; if this step were fixed, those 670 women would have been tested, 214 (670•32%) would be expected to have been positive, 193 (670•32%•90%) would be expected to have linked to PMTCT, 127 (670•32%•90%•67%) would be expected to have brought their infants for HIV prophylaxis, reflecting the “cascade gain” column. This structure assumes counterfactual exchangeability, that is, we assume that if those dyads that failed a step had not failed, they would have had the same subsequent cascade probabilities as those who were observed to have completed the step. While subsequent CAT adaptations have moderately more complex structures, all have the same logic of sequential multiplied conditional probabilities.

Systems to which CAT is well and poorly suited

CAT is well suited to care models where 1) steps occur sequentially, 2) the system’s goal is to optimize (reach 100% completion) on at least one step, and 3) each step is binary (completed or not). There are, of course, conditions and exceptions:

1. While health systems are complex and not every patient follows the same flow, CAT is well suited to systems with a dominant service delivery flow pattern, even if minority flow patterns exist. For example, the PMTCT system has one dominant service delivery flow pattern making CAT suitable, while emergency room settings have multiple conditional, branching steps in patient management and highly individualized flow patterns challenging for CAT.

2. Some cascade steps are not intended to be optimized. For example, we would aspire to have 100% of pregnant women tested for HIV, but would not aspire to have 100% test HIV positive. CAT remains well suited to systems with these types of steps, but may be less well suited to a system in which management is less algorithmic and clinical decision-making is based on multiple, individualized factors (co-morbidity, clinical presentation, patient resources).

3. Some steps have a number of preferred outcomes based on test value ranges. A continuous variable may be transformed into discrete categorical options; with sufficiently few options, this system may remain well suited to CAT. For example, blood pressure (BP) readings are continuous but can be binned into ranges indicating a) ineligible for treatment, b) eligible for treatment, or c) refer for hypertensive emergency. These categories then can become multinomial or binary steps and remain appropriate for CAT. However, a patient encounter for common mental illnesses might reveal mild, moderate, or severe depression, potentially comorbid with anxiety and/or post-traumatic stress disorder. The absence of mutually exclusive categories makes this step poorly suited to CAT. Additionally, steps that assess quality or fidelity of delivery might be poorly suited to CAT as currently conceptualized.

CAT data sources

Ideal routine health information attributes include validity, timeliness, completeness, representativeness, and reliability [14]. Data sources for populating CAT should additionally be aggregated correctly, physically accessible, represent denominators of interest, and not add additional burden to HCWs. Unsurprisingly, it is rare to find a singular data source that meets all of these qualities. Adapting CAT to a new cascade requires determining which data sources are “good enough” and which services they represent. This process requires a methodical approach; we have employed data mapping procedures to identify possible data sources and note their strengths and limitations (Table 1). Data mapping procedures emerged from the related process mapping; instead of patient flow, they illustrate where data are collected, transported, recorded, and stored. Pre-determined rules for data quality have been useful in some CAT adaptations, often following guidance from the Global Fund that uses a 10% error rate as a threshold above which data quality improvement activities should be undertaken prior to introducing CAT. In some cases, adequate data systems do not exist and must be created. This may be more common in low-resource settings where health systems are moving from acute to chronic care service models.

Table 1:

Mapping steps for identifying and evaluating data sources to populate a cascade analysis tool (CAT).

General step	Specific activities suggestion	Special issues to consider
1. Identify all existing data sources	– Consider the use of census or other routine administrative data sources to inform denominators – Request information on official registers and best practices from health authorities – Perform physical walkthrough of clinic to:  • Confirm presence of data sources identified by health authorities  • Identify any additional sources (cards / files / booklets) being filled and utilized	– Consider informal tracking systems (e.g. tick marks on a side paper, waiting bay cards that are color coded to match services needed)
2. Consider logistics of accessing data source routinely	– For each source, determine:  • Person responsible for data entry and aggregation  • Storage  • Access if currently in use (after hours / weekends?)	– Data source may be stored in a way that prohibits accurate estimation of denominators (e.g. patient booklets that travel with patient, longitudinal registers that are batched by first visit in care)
3. Characterize the accuracy of each data source in informing numerator and denominator counts of cascade indicators	– For each source, check:  • Completeness of denominator data  • Consistency with other data sources  • Accuracy and completeness of numerator data  • Causes and impact of missingness  • Meaning of special marks / numbering systems  • How unusual scenarios are captured	– Numerator from upstream step should match denominator for next step; compare data sources to ensure the same number appears in both sources – Identification numbers should be unique to reflect a single patient; consider how dyads and repeat visits are tracked and linked, as well as accuracy of that system – Individuals may be systematically missing from denominator (e.g. arrive after service ends, referred to next level of care)
4. Compare any aggregated counts to their source data for accuracy and assess whether level of aggregation matches cascade population	– Perform field check or database comparison of a random sample – Consider decision rules for accuracy – Compare multiple levels of aggregation (e.g. within registry vs. facility-wide) for most accurate and convenient source	– Age bands or time periods for aggregation not matching population of interest – Aggregated summary counts for population that does not match cascade (e.g. combining known HIV positive and newly diagnosed HIV positive individuals)

RESULTS

CAT has been adapted to diverse cascades globally, to communicable and non-communicable diseases, and to single encounters and chronic care management. Evaluations are ongoing to assess the impact of combination interventions incorporating CAT on health outcomes and implementation outcomes, using qualitative and quantitative methods. A core set of investigators supported CAT adaptation, but scientific leadership was unique across the adaptations. This section outlines CAT adaptations (Table 2).

Table 2:

Cascade analysis tool (CAT) adaptations

	ART CAT	PCAT	mPCAT	PedCAT	FPCAT	MHCAT	CCS-CAT	HCAT	NCAT
Country	Mozambique	Mozambique, Kenya, Cote d’Ivoire	Mozambique, Kenya	Kenya	Kenya	Mozambique	Kenya	Mozambique	USA
Year of development	2004	2013	2018	2017–8	2018	2018	2019	2019	2019
Disease area(s)	HIV treatment	HIV prevention	HIV prevention	HIV testing, treatment, and suppression	HIV testing	Mental illness	Cervical cancer screening & treatment	Hypertension screening & management	Opioid overdose reversal
Population(s) of interest	Adults living with HIV	Pregnant women living with HIV & HIV-exposed infant dyad	Pregnant women living with HIV & HIV-exposed infant dyad	Children, adolescents, and young adults (0–24 years)	Family planning clinic attendees	Individuals diagnosed with a mental health problem in primary care	People living with HIV & general population, specifically family planning clinic attendees	People living with HIV/ Broader adult population in ambulatory care	People who inject drugs
Cohort vs cross-sectional approach to counts	Cohort	Cross-sectional	Cross-sectional	Cross-sectional	Cross-sectional	Cohort	Cohort	Cross-sectional	Cross-sectional
Time period reflected in cascade	12 month cohort	Minimum 1 month	Minimum 1 month	1 month	1 month	3 month moving cohorts	1 month	Minimum 1 month	Weekly for naloxone distribution cascade; 3-monthly for naloxone use cascade
Number of data sources to populate	3	4	4	8+	1	1	1	3 (still in development)	2
Type of data sources (paper/electronic)	Paper registers	Paper and electronic	Paper and electronic	Paper registers	Paper registers	Paper individual patient chart	Paper registers	Paper/electronic	Electronic
Available data aggregated to correct level to populate cascade	Yes	Yes	Yes	No	No	Yes	No	Yes	No
Number of sites where piloted or tested	1	Piloted in 6, trialed in 18	Piloted in 3, scaling in 36	6	12	4	Pilot pending	1 piloted (trialing soon in 8 sites)	1 current pilot, 3 planned
Steps of cascade	1) test for HIV, (2) enroll for HIV care, (3) undergo CD4 testing, (4) start ART (if eligible), (5) adhere to ART	1) 1st antenatal visit 2) tested for HIV (or known positive) 3) start ART 4) prophylaxis for HIV-exposed newborns 5) attend at-risk visits 6) PCR testing done within 8 weeks 7) start ART within 12 months	1) 1st antenatal visit 2) tested for HIV (or known positive) 3) start ART 4) prophylaxis for HIV-exposed newborns 5) attend at-risk visits 6) PCR testing done within 8 weeks 7) start ART within 12 months	1) Complete HIV testing 2) Link to HIV care 3) Initiate antiretroviral therapy 4) Receive viral load monitoring 5) Suppress viral load	1) FP clients 2) Counseled for HIV testing 3) Tested for HIV	1) New diagnoses of mental health problem 2) Medication prescribed 3) Follow-up date provided 4) Return to last follow-up visit 5) Return to last follow-up visit on time (+/− 5 days) 6) Adherent based on pill counts 7) Show improved function	In development	1) Outpatient visits 2) Blood pressure measurement 3) Eligible for hypertension medication 4) Medication prescribed 5) Prescription filled 6) Hypertension controlled at next visit	Naloxone Distribution: 1) Present for syringe access services 2) Trained in naloxone administration 3) Possess naloxone 4) Immediate access to naloxone Naloxone Use: 1) Overdoses observed 2) Community-based reversal attempted 3) Reversal successful

ART CAT: Antiretroviral therapy cascade analysis tool; PCAT: Prevention of mother-to-child transmission of HIV cascade analysis tool; PedCAT: Pediatric and adolescent cascade analysis tool; FPCAT: Family planning cascade analysis tool; MHCAT: Mental health cascade analysis tool; CCS-CAT: Cervical cancer cascade analysis tool; HCAT: Hypertension cascade analysis tool; NCAT: Naloxone cascade analysis tool

Mobile application

A mobile phone PCAT application was developed (mobile PCAT (“mPCAT”) (NIH R21AI124399))[15] to address the computer access and literacy barriers of the Excel-based PCAT. Mobile phones are widely used in Mozambique and Kenya, and mPCAT’s development was driven by nurses’ desire to independently use data to gauge service flow and identify bottlenecks. During usability testing and feasibility pilots in Kenya and Mozambique, several adaptations were made, including simplification of data entry, harmonization of labels with national registry norms, and ensuring easy sharing of results via WhatsApp or email [15]. Currently, the mPCAT is being used in SAIA-SCALE (R01MH113435)[16], which is scaling-up SAIA across one Mozambican province.

Pediatric & adolescent HIV

The second CAT adaptation was to the pediatric and adolescent HIV cascade in Kenya (F32HD088204; R34AI129900), the “PedCAT”. The PedCAT includes 5 steps: 1) HIV testing uptake, 2) linkage to care, 3) ART initiation, 4) viral load (VL) monitoring, and 5) VL suppression. Data sources aggregated to reflect children (0–9 years), adolescents (10–19 years), and young adults (20–24 years) distinctly were lacking. Three data sources were considered: a) monthly aggregated, clinic-level count data from the District Health Information Service; b) abstracted individual-level, cross-sectional facility paper registers, and c) individual-level, longitudinal or cross-sectional electronic medical records. No single data source was sufficient to cover the 5 cascade steps. Abstraction of paper registers was selected as the only feasible approach, and a minimum of 8 registers are required to fill the PedCAT appropriately.

Family planning & HIV testing integration

The third CAT adaptation was to model the integration of HIV testing into family planning (FP) services in Kenya (K24HD088229), the “FPCAT”. National guidelines recommend HIV testing in FP; in 2008, HIV testing questions were added to the FP clinic register, which contains the required inputs for the FPCAT; 1) HIV counseling, and 2) HIV testing.

Hypertension & HIV care integration

The fourth CAT adaptation focused on integrating hypertension services into HIV care in Mozambique (R01HL142412), the “HCAT”. In many resource-limited settings, chronic care models for non-communicable diseases (NCD) do not exist at scale, while the HIV chronic care platform is broadly implemented. Integrating hypertension services into the HIV platform represented an opportunity to standardize and scale NCD services. The HCAT took a cross-sectional approach and included the following steps: 1) BP screening, 2) diagnosis, 3) linkage to care, 4) treatment initiation, 5) ongoing BP monitoring, and 6) controlled hypertension.

Mental health

The fifth CAT adaptation focused on optimizing task-shared outpatient mental health care for epilepsy and schizophrenia in Mozambique (R21MH113691), the “MHCAT” [17,18]. In contrast to previous sequential cascades, chronic outpatient mental health care involves a complex series of repeated, cyclical treatment and intervention protocols, and steps may differ by patient symptoms, diagnoses, and within patients over time. A challenge was determining which cascade steps were most important for monitoring outpatient cascade performance, while being identical for all patients within each diagnosis. The resultant MHCAT cascade steps are high-level and lose much of the granularity of detailed outpatient mental health treatment protocols. This has the benefit of identifying improvement opportunities that may be common to multiple systems, but the disadvantage of losing the opportunity to identify protocol-specific improvement opportunities.

Opioid overdose reversal

The sixth CAT adaptation focused on optimizing integration of naloxone—an opioid antagonist that reverses opioid overdose—distribution within syringe access programs for people who inject drugs in California, USA (R21DA046703), the “NCAT”. Like the PCAT, the NCAT has two cascades: naloxone distribution and community-based overdose reversal. Cascade 1 consists of five steps: 1) present for syringe access services, 2) trained to administer naloxone, 3) possess naloxone, and 4) immediate access to naloxone. Cascade 2 contains three steps: 1) overdoses observed, 2) community-based reversal attempted, and 3) successful reversals. This adaptation is the only application of CAT in a resource-rich country and in a community-based, as opposed to health facility-based, setting. The absence of robust data systems, a stigmatized health service, challenges with loss to follow-up, and limited resources available for service delivery are similarities between this adaptation and others in resource-limited settings outside of the US. However, unique opportunities of this adaptation include having flexibility to modify data sources and markedly change service delivery organization without liaising with centralized bodies, such as ministries of health.

Cervical cancer screening

The seventh adaptation of CAT considers integration of cervical cancer screening into FP clinics in Kenya, the “CCS-CAT”, and is under development. Key challenges in this adaptation include properly identifying FP clients who have been screened within the recommended timeframe by age and HIV status and therefore do not require screening [19]; and properly categorizing women who receive screening at subsequent, rather than first, visits.

Challenges encountered during CAT adaptation

Challenges and design choices broadly fell into 8 categories (Table 3): 1) time periods for data aggregation and analysis, 2) handling low patient volumes, 3) use of cross-sectional versus cohort-based populations, 4) integrating with electronic health records or mobile platforms, 5) analyzing a full catchment area versus patients presenting for care, 6) estimating denominators, 7) managing appropriate exit and re-entry of groups, and 8) expanding CAT operators. Examples of each are described below.

Table 3:

Challenges and solutions in adapting cascade analysis tool (CAT)

Major modification domains	mPCAT	PedCAT	FPCAT	MHCAT	CCS-CAT	HCAT	NCAT
1) Time periods for data aggregation and analysis				Problem: Small counts make monthly cascades difficult because of “zero” cells; highly variable monthly counts lead to implausible proportions using cross-sectional approach Solution: Cascade covers 3 month long period to increase overall counts, cascade uses cohort-based population to retain relevant denominators and plausible proportions			Problem: Clients receiving naloxone have unstructured visit schedule that could result in repeat visits within a month to syringe services; however, naloxone use is infrequent Solution: Naloxone distribution cascade uses one week long period; naloxone use cascade uses 3 month long period
2) Use of cross-sectional vs. cohort-based populations
3) Handling low patient volumes							Problem: Small counts in naloxone use cascade make monthly cascades difficult because of “zero” cells Solution: Naloxone use covers 3 month long period to increase overall counts
4) Analyzing a full catchment area vs. patients presenting for care		Problem: Catchment area population undefined Solution: Cascade begins with population presenting to health center					Problem: Catchment area population undefined Solution: Cascade begins with population presenting to syringe service programs
5) Estimating denominators		Problem: Estimating denominator for viral load monitoring not feasible directly Solution: Utilized pseudo-denominators from simple mathematical formulas	Problem: Clients returning for repeat visits may have previously received service Solution: Include only new clients to avoid repeat testing challenges
6) Managing appropriate exit and re-entry of groups from cascade		Problem: Heterogeneity between sites in assessing HIV testing eligibility Solution: Included as optional step in cascade to accommodate sites with and without step			Problem: Not all clients are eligible for HIV testing, which depends on risk profile and recentness of testing Solution: Included appropriate exit for individuals who did not require testing at time point		Problem: Clients have two streams of services within a similar visit structure (either requiring provision of naloxone or not) and all contribute to final step of cascade of appropriate possession Solution: Cascade includes appropriate exit and re-entry steps in programming to reflect holistic end step in cascade
7) Integrating with electronic health records or mobile platforms	Problem: Access to computers with Excel limited potential users of tool; results challenging to share in intuitive format Solution: Migrated from Excel-based to mobile phone-based platform, with data visualization graphics that can be sent via WhatsApp and other messaging services			Problem: No existing data tools to link patient records over time; high data collection needs to populate cohorts Solution: Created new mental health modules in electronic medical record; integrated cascade with electronic medical record to auto-populate
8) Expanding CAT operators to include new HCW cadres							Problem: Clients served outside of a health facility setting by community-based organizations Solution: Cascade filled and operated by community-based organizations
9) Additional unique challenges		Problem: Age bands to define children and adolescents not reflected in aggregated data sources Solution: Abstract data from paper registers to enable custom age bands; this solution has downside of taking time and resources		Problem: Cascade steps may differ based on mental health diagnoses and for patients over time Solution: In process of resolving challenge		Problem: No existing data tools for hypertension; medical management may differ based on co-morbidities Solution: Created new hypertension registers for government; medical management steps left flexible to accommodate changing guidelines and comorbidities	Problem: Existing data sources exist but are time intensive and bias against service or representative denominator Solution: Introduce new streamlined data tool to populate cascade and separate from intensive questionnaire

The NCAT required modification of time periods (1) as, unlike in facility-based chronic care where visits typically occur at fixed intervals, syringe access visits may occur multiple times per month depending on client needs. To address this, NCAT Cascade 1 was aggregated weekly. In contrast, in NCAT Cascade 2 and the MHCAT, the time period was extended to three months. Using monthly data for individual MHCAT diagnoses and for overdose reversals caused “zero” cells due to small numbers (2); three-month data overcame this challenge, but is less sensitive to monthly CQI cycles.

The MHCAT was the only adaptation to employ a cohort-based approach (3). This adaptation required creating a unique form to track patient visits over time. An open-source electronic medical record software (OpenMRS) was used to automatically populate the MHCAT (4). Notably, the model of using individual-based cohort data creates a large data collection, review, and entry burden.

The PCAT is able to estimate the number of expected pregnancies in each clinic’s catchment area and the estimated number not presenting for ANC, making it well suited to evaluate both demand generation and supply side interventions (5). However, several others lack this step. For example, for the NCAT and MHCAT, epidemiologic data on population prevalence of targeted conditions is lacking; for the PedCAT, estimates of pediatric and adolescent HIV prevalence are not available below the county level. These CAT were limited to those presenting for care, making them unsuitable to evaluate demand generation interventions.

Sources of denominator data, even within the population presenting for care, can be challenging (6). For the HCAT and MHCAT, no patient tracking forms were in use and new registers had to be developed and piloted. For the PedCAT, estimating the number of children and adolescents requiring a VL sample was complex. Abstraction from patient files to determine when individual patients’ routine VLs were expected according to Kenyan guidelines was unfeasible; a mathematically calculated pseudo-denominator was chosen to overcome this challenge and extend the PedCAT through to a clinically relevant health outcome.

In the FPCAT and PedCAT, testing schedules for different Kenyan populations required determining appropriate exits from the cascade for individuals not requiring testing (7). In the FPCAT, exit was required for known HIV-positive clients or those with a recent HIV test (3-monthly for key populations and annually for general population). However, this population characterization was not routinely documented in the FP register, requiring modification of data systems. In contrast, the NCAT Cascade 1 has two different streams of services, depending on whether syringe access clients received an initial training; clients who have are not exited but have separate cascade steps. This creates two streams that run in parallel and intersect for the final cascade step.

The NCAT was the first to reflect community-based delivery and data collection (8). While data sources existed to partially populate the NCAT, data collection systems were lengthy and inhibited service provision. Shortened tools are under development.

DISCUSSION

Through the first HIV care CAT, the original SAIA PCAT, and 7 subsequent adaptations, we have learned lessons on CAT structure, mathematics, acceptability, feasibility, and usability. We have encountered technical challenges (e.g. absence of appropriate data sources or complex care patterns not easily dichotomized), and created technical innovations (e.g. mobile-based collection or EMR integration).

Additional opportunities remain for CAT adaptation, such as incorporation of non-binary outcomes, integration into e-platforms, and expansion of CAT operators. Quality of health services is a new frontier [20–22], but is often multi-dimensional and on a continuous scale, and not easily summarized by the binary CAT steps. Additional work is needed to incorporate quality and other non-binary steps into CAT, while maintaining the tool’s attractive simplicity. As CAT is applied to chronic care models, repeat visits within individuals needs to be modeled, as well as complex case management patterns to address co-morbidities, side effects, treatment switches, mortality, and migration. As electronic health information systems become more common, innovation is needed to integrate CAT into routine eHealth platforms (e.g. DHIS II or OpenMRS). As CAT is scaled up, as in SAIA-SCALE, or decentralized to community-based organizations, as in SAIA-Naloxone, questions remain about who should populate the tool—HCWs, community workers, or research staff. Additionally, further research is needed to identify settings in which CAT is most and least useful. For example, in the original SAIA trial, HCWs in Cote d’Ivoire (where HIV prevalence is lower, and zero counts were common) found the tool to be less useful than Mozambican and Kenyan HCW. Contextual factors beyond facility volume may impact acceptability and utility of CAT. Finally, data collection is underway to determine whether CAT is time saving for HCW and whether stakeholders have interest in expanding and sustaining CAT use beyond research settings.

There are important epidemiologic questions about using CAT to evaluate impact in a research context, balancing specificity and usability. While CAT is intended as an implementation tool, it could be used to populate charts for CQI evaluation, or CAT step performance could be tested using classical statistical hypothesis testing. However, there may be a tension between acceptability and specificity if CAT is to be used for both priority setting and impact evaluation. The target population to assess impact might be more homogeneous, exclude patients known to have been exposed to the intervention for longer periods of time, or otherwise restrict the patient population. Using CAT to evaluate impact is further complicated by the multiple steps, and therefore multiple potential outcomes; initial work has investigated the discriminating power of comparing high and low performance of individual versus composite indicators in the PMTCT cascade [23], an analysis that should be replicated for other cascades. Additionally, CAT includes both absolute numbers and proportions; improvements in upstream steps of CAT could result in a false sense of worsening downstream effects if only considered on the relative proportion scale rather than the absolute scale.

The use of CAT in the SAIA trial and subsequent adaptations suggest that CAT is an effective implementation strategy, although evaluations of adaptations to determine impact on health outcomes and implementation outcomes are ongoing. While CAT was developed within the SAIA package, and the examples demonstrate its role within SAIA, CAT may be used independently or in conjunction with other implementation strategies. CAT has high acceptability and feasibility across resource-limited and high-income settings, and with various HCW cadres. CAT integrates well with CQI and process mapping, two evidence-based implementation strategies [24], and may be a low cost addition to enhance their effectiveness. Further research is ongoing to examine other implementation outcomes associated with use of CAT, such as penetration and sustainability.

CONCLUSION

CAT is intended to overcome a well-described barrier between routine data availability and its use for data-informed systems optimization [25–27]. It is a powerful implementation strategy that synergizes to enhance the effectiveness of well-established implementation strategies, such as CQI and process mapping, by allowing frontline HCWs to view their systems holistically and support prioritization. CAT is adaptable to a wide range of health services, populations, and settings; innovations in technology have made CAT more acceptable and usable to widespread audiences. Future methodologic challenges include simplifying data collection, reflecting more complex care systems, and scaling this approach to routine program implementation.