PROMISE (Program Refinements to Optimize Model Impact and Scalability based on Evidence): a cluster-randomised, stepped-wedge trial assessing effectiveness of the revised versus original Ryan White Part A HIV Care Coordination Programme for patients with barriers to treatment in the USA

Abstract

Introduction

Growing evidence supports combining social, behavioural and biomedical strategies to strengthen the HIV care continuum. However, combination interventions can be resource-intensive and challenging to scale up. Research is needed to identify intervention components and delivery models that maximise uptake, engagement and effectiveness. In New York City (NYC), a multicomponent Ryan White Part A-funded medical case management intervention called the Care Coordination Programme (CCP) was launched at 28 agencies in 2009 in order to address barriers to care and treatment. Effectiveness estimates based on >7000 clients enrolled by April 2013 and their controls indicated modest CCP benefits over ‘usual care’ for short-term and long-term viral suppression, with substantial room for improvement.

Methods and analysis

Integrating evaluation findings and CCP service-provider and community-stakeholder input on modifications, the NYC Health Department packaged a Care Coordination Redesign (CCR) in a 2017 request for proposals. Following competitive re-solicitation, 17 of the original CCP-implementing agencies secured contracts. These agencies were randomised within matched pairs to immediate or delayed CCR implementation. Data from three 9-month periods (pre-implementation, partial implementation and full implementation) will be examined to compare CCR versus CCP effects on timely viral suppression (TVS, within 4 months of enrolment) among individuals with unsuppressed HIV viral load newly enrolling in the CCR/CCP. Based on current enrolment (n=933) and the pre-implementation outcome probability (TVS=0.54), the detectable effect size with 80% power is an OR of 2.75 (relative risk: 1.41).

Ethics and dissemination

This study was approved by the NYC Department of Health and Mental Hygiene Institutional Review Board (IRB, Protocol 18–009) and the City University of New York Integrated IRB (Protocol 018–0057) with a waiver of informed consent. Findings will be disseminated via publications, conferences, stakeholder meetings, and Advisory Board meetings with implementing agency representatives.

Trial registration number

Registered with ClinicalTrials.gov under identifier: NCT03628287, V.2, 25 September 2019; pre-results.

Strengths and limitations of this study.

• The PROMISE (Program Refinements to Optimize Model Impact and Scalability based on Evidence) trial, conducted in real-world service settings, leverages secondary analyses of programmatic and surveillance data to assess the effectiveness of a revised (Care Coordination Redesign (CCR)) versus original HIV care coordination programme to improve viral suppression; limitations include a lack of blinding and lack of control over participation in other services that may affect the outcome.

• To meet stakeholder expectations for rapid completion of the CCR rollout, the study applies a stepped-wedge design with a 9-month gap between steps, prompting use of a 4-month outcome (which limits the evaluation to short-term effectiveness) and a 5-month lead-in time for enrolment accumulation (which limits ability to detect and measure actual CCR effects with precision).

• Randomisation is performed at the agency level to minimise crossover between the intervention conditions and avert the logistical and ethical dilemmas that client-level randomisation would create for service providers; while the possibility remains that an agency assigned to delayed implementation might partially adopt some CCR elements (eg, client self-management assessment and counselling) prior to their official switch to CCR, this risk is minimised via distinct reimbursement structures for the two programme models and study arm-based staggering of provider trainings on programme revisions.

• Compared with randomisation at the level of the client, cluster randomisation generally yields lower statistical power; however, agency randomisation within matched pairs offers advantages akin to those of stratified random assignment: increasing statistical power in a situation where the number of units of randomisation is small, by maximising equivalency between the intervention and control groups on key observable variables, thus helping to isolate intervention effects.

• In addition, nuisance parameters are removed through the conditional analytical approach, which accounts and allows for the unavoidably imperfect matching of agencies and arbitrary variation of period effects across agency pairs.

Introduction

Successful HIV treatment at the individual level requires consistent adherence to antiretroviral therapy (ART) resulting in a sustained suppression of HIV-1 viral load (VL) in plasma to levels below the detection limit of HIV RNA tests used by healthcare providers. At the population level, viral suppression (VS) is the key to the dual goals of improving health and survival among people with HIV (PWH) and preventing HIV transmission. A growing consensus among researchers supports the blending of evidence-based social, behavioural and biomedical strategies (ie, ‘combination interventions’) to address barriers to ART use and adherence.^1–6 However, combination interventions can be resource-intensive to deliver and challenging to translate to new and diverse organisational settings.⁷ Research is needed to systematically inform the selection of intervention components and service delivery approaches that can maximise programme uptake, engagement and effectiveness at scale in a range of practice environments.

In New York City (NYC), a multicomponent Ryan White Part A-funded medical case management intervention known as the HIV Care Coordination Programme (CCP) was launched in late 2009 to meet the needs of individuals with suboptimal HIV care outcomes or new HIV diagnoses. By late 2013, an NYC Health Department-City University of New York (CUNY) research partnership had secured funding for the ‘CHORDS’ (Costs, Outcomes and Real-world Determinants of Success in HIV Care Coordination) study of CCP effectiveness (R01 MH101028). Early findings of increased care retention⁸ led the Centers for Disease Control and Prevention (CDC) to include the CCP in their Compendium of Evidence-based Interventions and Best Practices for HIV Prevention.⁹ Observational studies with a rigorously selected usual-care comparison group¹⁰ have shown positive CCP effects on short-term VS (relative risk (RR)=1.32, 95% CI: 1.23 to 1.42),¹¹ as well as durable VS (DVS) (RR: 1.16, 95% CI: 1.04 to 1.29),¹² among PWH without previous evidence of VS However, substantial room for improvement remains: over one-third of the clients drop out of the programme in the first year;^{8 12} and a minority of clients without previous evidence of VS achieve VS (43%)¹¹ or DVS (21%).¹² In addition, some of the original CCP design features have curbed client and provider engagement: a rigid system of programme tracks has impeded service intensity adjustment based on client need (ie, differentiated care^{13 14}); a complex reimbursement model has diverted staff time and attention to maintaining agency cash flow; and a requirement for weekly visits over a 3-month ‘induction period’ for new clients has reportedly deterred eligible PWH from enrolling and discouraged staff from suggesting the CCP for PWH perceived as unable to meet that commitment.¹⁵

After several years of CCP implementation, the NYC Health Department and the local Ryan White Part A community Planning Council outlined a set of programme modifications in response to the identified implementation barriers¹⁵ and the evolving epidemiological data and intervention literature, including findings from the CHORDS research collaboration.^{11 12} Programme revisions were integrated into the Health Department’s late-2017 request for proposals (RFP) initiating a competitive selection process for future NYC Care Coordination service delivery contracts. This RFP also outlined plans for agency-level randomisation to an early or delayed start of the revised model, as part of an experimental evaluation of its effectiveness.

This paper describes the experimental protocol for the study known as PROMISE (Program Refinements to Optimize Model Impact and Scalability based on Evidence), which continues the NYC Health Department-CUNY research partnership initiated with CHORDS. Our purpose is to inform practice-driven intervention research, particularly in the context of generating evidence for the optimisation of safety-net service delivery strategies. The overarching goal of PROMISE is to investigate the impact and implementation of empirically driven course corrections to an already effective intervention model. We will test the combined effect of intervention modifications in a cluster-randomised controlled trial applying a cross-sectional, stepped-wedge design to the rollout of the revised model in previously funded, re-awarded CCP provider agencies. Drawing on an implementation science framework and RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance)^{16 17} principles, we posit that the model revisions will reduce logistical and administrative barriers to service delivery and increase programme engagement (among staff and clients), reach, fidelity and effectiveness. Specifically, we hypothesise that a higher proportion of PWH with unsuppressed VL at enrolment in the Care Coordination Redesign (CCR) model will achieve timely VS, as compared with PWH with unsuppressed VL at enrolment in the original CCP during the same period.

Methods and analysis

Participants, intervention and outcomes

Study setting

Of the original 28 agencies funded to deliver the CCP since December 2009, 17 secured funding under the 2017 to 2018 re-solicitation, including a range of organisation types (hospitals, community health centres and community-based organisations without medical care onsite). Together, the 17 re-awarded agencies covered all NYC boroughs/counties and a collective caseload of more than 2000 active CCP clients as of the end of March 2018 (agency median: 96, IQR 82 to 181). Clients in the original CCP have been predominantly Black or Latino/Latina (91%), male (64%) and aged 45 and older (51%), reflecting the demographic composition of the larger NYC Ryan White Part A HIV-positive client population.^{10 18} Study sites are listed at: https://clinicaltrials.gov/ct2/show/NCT03628287. Agencies assigned to delayed CCR implementation received contract extensions to continue service delivery under the original CCP, while awaiting their transition to new/CCR contracts.

Eligibility criteria (for clients and sites)

PWH eligible for the trial analysis of CCR intervention effects include those newly enrolled in the CCP or CCR and having unsuppressed VL (HIV RNA ≥200 copies/mL) as of their latest test in the year prior to enrolment or having no VL test result reported to surveillance in the year prior to enrolment (presumed out of care).¹⁰ To allow 4 months of VL outcome observation per enrollee prior to the start of the next phase of CCR rollout, the trial-eligible enrolment window for each 9-month implementation period is restricted to the first 5 months. The trial excludes newly awarded (CCP-naïve) agencies and includes only the 17 re-awarded agencies, which could be assigned to continue CCP delivery uninterrupted or begin CCR delivery in the initial implementation phase.

Intervention (CCR) and control (CCP) conditions

The CCP has been described previously.^{8 18 19} The control condition is the site-level continuation of CCP model delivery, while the intervention condition is a site-level change to deliver the CCR. Sites’ ability to begin CCR delivery hinges on reimbursement for CCR services and approaches, and on receipt of Health Department trainings related to programmatic changes. Study assignments have been maintained through the scheduling of agencies for CCR trainings and contract starts (tied to the change in reimbursement).

Specific programme revisions²⁰ (added, changed or removed components) and their anticipated benefits are displayed in table 1. Major additions include a training and set of tools for baseline and quarterly assessment and counselling around client HIV self-management capacity;²¹ allowance of video chat as a delivery mode for certain services, such as directly observed therapy, and optional iART^{22 23} (‘immediate’ ART: ensuring that the client has a filled prescription within 4 days of either enrolment or diagnosis). Changes include the replacement of per-member-per-day reimbursement with a fee-for-service reimbursement model that accounts for resource demands, such as staff travel to and from clients’ homes, and offers higher rates for meeting performance standards (eg, same-day prescription fulfilment). Other changes include greater emphasis and guidance on identifying and recruiting individuals with documented clinical need (eg, unsuppressed VL). Some CCP requirements (ie, induction period, enrolment tracks) were selected for removal in favour of flexibility for client-centred/differentiated care.^{13 14}

Table 1.

Revisions expected to improve uptake, fidelity, engagement, effectiveness and reach/impact

	Added components			Changed		Removed
	Self-management assessment	Use of video chat tools (optional)	iART (optional)	Eligibility criteria	Payment structure	Rigid programme tracks
Uptake (provider)						X
Fidelity (provider)		X			X	X
Engagement	X	X				X
Effectiveness	X	X	X		X	X
Reach/impact	X	X	X	X	X	X

iART, immediate antiretroviral therapy.

Individual clients may switch from one of the 17 sites to another, potentially changing their intervention condition. However, based on intent to treat, clients count toward the trial only in their first enrolment during the study period. Clients in the trial may access other interventions within CCP/CCR settings or other agencies where they receive services. The NYC HIV-related services landscape includes many provider agencies and funding sources beyond those that can be tracked by any single entity; it is not feasible to prohibit simultaneous receipt of other interventions that may affect the study outcome.

Outcome measurement

To assess the clinical benefit of the programmatic revisions distinguishing the CCR from the CCP, we will analyse client-level, surveillance-based laboratory test data.¹⁰ The outcome, timely VS (TVS), is defined as VL <200 copies/mL on the last VL test reported to the NYC HIV surveillance registry in the 4 months following CCP/CCR enrolment (TVS=1). We have chosen to dichotomise VL data for statistical analysis using the cut-off value of 200 copies/mL, in accordance with the CDC definition of VS.²⁴ Consistent with our prior CCP work,8 10 11 18 those without any VL measure during follow-up will be considered not to have achieved VS (TVS=0), given their lack of documented clinical monitoring since their last unsuppressed VL. The 4-month follow-up period aligns with US Department of Health and Human Services HIV guidelines, which reinforce the standard practice of VL monitoring every 3 to 4 months, or more often when adherence difficulties are apparent.²⁵ For PWH starting ART or a modified ART regimen, the guidelines recommend VL monitoring every 4 to 8 weeks until VS is reached, and state that ‘individuals who are adherent to [ART] and do not harbor resistance mutations to the component drugs can generally achieve viral suppression 8 to 24 weeks after ART initiation.’²⁵ Recent publications also support the applications of shorter-term measures of VS; researchers at NYC’s Health Department have proposed adding a 3-month VS indicator for tracking national progress on the HIV care continuum,²⁶ and a San Francisco study of a vulnerable population of newly diagnosed patients referred for rapid ART initiation found that the median time from start of ART to VS was 41 days.²⁷ Our TVS measure takes into account both the timing of routine VL monitoring in as-yet-unsuppressed PWH and current expectations for VS achievement in a context of effective ART and universal/immediate treatment policies.^{25 28 29}

Timeline

Figure 1 illustrates the three 9-month periods used in the stepped-wedge design: Period 0, with CCP at all 17 agencies and no CCR; Period 1, representing CCR implementation only at sites randomised to an early start (and thus encompassing the months of simultaneous operation of the CCP and the CCR); and Period 2, representing CCR implementation at all 17 sites.

Figure 1.

Stepped-wedge design with three implementation periods. CCP, Care Coordination Programme.

Recruitment

Beyond standard contract startup deliverables based on early programme enrolment milestones, no specific incentives have been used to encourage recruitment. Analyses will include all eligible enrolments in CCP/CCR services at any of the 17 study sites.

Assignment of interventions

Randomisation

Though the unit of analysis for TVS is the individual, the unit of randomisation is the Care Coordination provider agency (ie, cluster). Cluster randomisation serves to minimise crossover between intervention conditions and avert the logistical and ethical dilemmas posed by client-level randomisation.^30–32 Characteristics and study arm assignments of the 17 agencies are shown in table 2. Agencies were matched and randomised within pairs (including one case in which two smaller agencies were matched to a larger one). Matching accounted for characteristics plausibly related to the TVS outcome: agency type, primary location/borough and programme size (measured via a combination of CCP caseload at the time of re-award and award amount). While randomisation could not feasibly be stratified by each of these variables, the lead analyst suggested pairs maximising similarity on these variables. Pairings were finalised with input from other team members knowledgeable about the programmes/agencies involved. The lead analyst used a random number generator in Excel to determine agency assignments within pairs, and assignments were communicated as contract conditions in the notifications of awards.

Table 2.

Agency characteristics, pairings and study arm assignments

Site ID	Award increased >20% from prior year?	Typical (prior) caseload	Borough within NYC	Type of site	Pair	Phase (study arm)
21	Yes	84	Bronx	CBO	1	1
1	No	101	Bronx	CBO	1	2
20	Yes	109	Brooklyn	Public Hospital	2	1
14	No	151	Brooklyn	Public Hospital	2	2
28	Yes	87	Brooklyn	Private Hospital	3	1
24	No	96	Brooklyn	Community Health Centre	3	2
25	No	62	Manhattan	Community Health Centre	4	1
9	No	78	Manhattan	Community Health Centre	4	2
23	No	228	Manhattan	Private Hospital	5	1
18	No	220	Manhattan	Private Hospital	5	2
13	Yes	82	Bronx	Public Hospital	6	1
11	Yes	82	Queens	Public Hospital	6	2
5	No	202	Bronx	Private Hospital	7	1
4	No	181	Manhattan	Private Hospital	7	2
8	Yes	77	Staten Island	CBO	8	1
16	No	63	Brooklyn	Community Health Centre	8	1
2	No	184	Manhattan	Community Health Centre	8	2

CBO, community-based organisation; NYC, New York City.

Blinding

Blinding was not feasible for this study. Assignments were transparent to implementing agencies, study team members and interested stakeholders, since contracts are publicly available information.

Data collection, management and analysis

Data collection

As with prior studies of CCP effectiveness,8 10–12 18 the outcome measure for clients in both study arms will be derived from the NYC HIV surveillance registry (‘the Registry’), a population-based data source of electronically reported longitudinal laboratory (VL, CD4) records on all diagnosed NYC PWH.^{33 34} Use of the Registry allows near 100% ascertainment of VS for PWH in NYC HIV medical care, regardless of specific NYC medical provider, and for periods extending before and after programme enrolment or discontinuation.

Each client’s CCP/CCR enrolment agency and start date are determined from a database of contractually required Ryan White Part A provider reporting to the Health Department, the Electronic System for HIV/AIDS Reporting and Evaluation (eSHARE). These programme reporting-based measures are available (non-missing) for all CCP/CCR clients and all implementing agencies. Programme data collection forms are located on the NYC Health Department website (https://www1.nyc.gov/site/doh/health/health-topics/aids-hiv-care-coord-tools.page).

Data management and quality assurance

All data for the trial are entered as part of established, legally or contractually required reporting, and are protected according to CDC physical and electronic data security and confidentiality policies.³⁵ Health Department staff clean and freeze surveillance data sets on a quarterly basis, and conduct matches of programme to surveillance data semi-annually, with human review of each near-match by two independent analysts and a separate ‘tie-breaker’ when the analysts’ determinations differ. Details on the deterministic matching algorithm have been previously described.³⁶ Through the match, participants are assigned a unique record number used in merging surveillance and programmatic data for analytical data sets, which are stripped of all personal identifiers prior to analysis and stored on the most secured drives on the Health Department network. eSHARE data quality is checked by Health Department analysts at the time of each monthly extraction. For purposes of payment, provider agencies also review draft extracts and fill in any missing enrolment and services data monthly.

Statistical analysis for the matched-pairs stepped-wedge trial

Analysis overview and rationale.

We will apply an innovative, fully conditional analysis that, in addition to allowing for arbitrary period effects, allows for arbitrary within-pair site differences. The analysis plan is based on the exact, conditional distribution theory of non-central multiple hypergeometric distributions and their convolutions,³⁷ which will enable us to estimate and test the effect of the revised intervention as a single parameter defined below. The conventional statistical analysis proposed for cross-sectional stepped-wedge designs (ie, with independent samples of clients enrolled at each step)³⁸ assumes a mixed model with random cluster effects and fixed period effects, but this is not appropriate for our matched-pairs stepped-wedge trial. For one, the matching of pairs is under the investigators’ control and so should be conditioned on. Second, the generalised linear mixed model has limitations, such as a gratuitous and unverifiable assumption of normal distribution for random effects and poor variance estimation performance in small samples (of clusters), even with robust variance estimation, such that jackknifing must be used. However, the following exact analysis avoids those problems by conditioning out the nuisance parameters.

Analysis approach and assumptions.

As shown in table 3, for each pair of sites, we will produce two 2×3 tables (one table per site in pair), cross-classifying the number of TVS and non-TVS outcomes in Period 0 (with original CCP but no CCR implementation), Period 1 (with CCR only at sites assigned to an early start) and Period 2 (with CCR at all sites). For identification purposes, we refer to ‘Site 1’ within a matched pair as the site randomised to switch in Period 1 (early start) and ‘Site 2’ as the site randomised to switch in Period 2 (delayed start). We begin by assuming the following logistic regression model for the three binomial outcomes: the logit of the probability of TVS for a given site, period and intervention (CCR versus original CCP) equals an intercept representing an arbitrary, pair-specific log odds on TVS for Site 2 in the pair, plus an arbitrary log OR (LOR) for Site 1 versus Site 2 in the pair (allowing for imperfectly matched sites), plus two arbitrary pair-specific LORs for Period 1 and Period 2 effects relative to Period 0, plus one structural LOR of interest, the global intervention effect (non-existent in Period 0, applicable to Site 1 in Period 1, and applicable to both sites in Period 2). The exponent of this last parameter is the target of statistical inference, namely, the OR for TVS versus non-TVS comparing the CCR to the CCP. A key assumption is that any site effects apply in each period and any period effects apply to each site, independent of the intervention effect (ie, that there are no site-by-intervention or period-by-intervention interactions). This assumption will be tested and the model elaborated if needed. Note that under the key assumption, the constant site and period effects are allowed to vary arbitrarily from one matched pair to the next.

Table 3.

Illustration of 2x3 tables cross-classifying TVS and non-TVS outcomes by period.

Site 1 in pair i (adopts CCR in Period 1)		Period 0	Period 1	Period 2	Total
	TVS	Xi10	Xi11	Xi12	Xi1+
	No TVS	Ni10– Xi10	Ni11– Xi11	Ni12– Xi12	Ni1+– Xi1+
	Total	Ni10	Ni11	Ni12	Ni1+
Site 2 in pair i (adopts CCR in Period 2)	TVS	Xi20	Xi21	Xi22	Xi2+
	No TVS	Ni20– Xi20	Ni21– Xi21	Ni22– Xi22	Ni2+– Xi2+
	Total	Ni20	Ni21	Ni22	Ni2+
Pair i totals	TVS	Xi+0	Xi+1	Xi+2	Xi++
	No TVS	Ni+0– Xi+0	Ni+1– Xi+1	Ni+2– Xi+2	Ni++– Xi++
	Total	Ni+0	Ni+1	Ni+2	Ni++

Light grey cells represent the two 2×3 tables in site pair i. Dark grey cells represent the margins upon which the analysis will condition, whereas white cells represent the margins calculated by summing or subtracting other fixed margins.

Estimating the CCR intervention effect.

Next, by conditioning on the marginal totals within each site (numbers of eligible clients enrolled in each period and total numbers of TVS and non-TVS outcomes for each site), the joint distribution of the numbers of TVS outcomes for Site 1 by period becomes a non-central multiple hypergeometric distribution with only three parameters (the period LORs and the intervention LOR); that is, the conditional distribution does not depend on the nuisance site parameters. By further conditioning on the sum of TVS outcomes across the two sites in each period, the fully conditional joint distribution depends on only one parameter, the intervention effect; that is, the fully conditional distribution depends neither on the nuisance site effects nor on the nuisance period effects. In fact, the sufficient statistic for the intervention LOR in the fully conditional likelihood function is simply the number of TVS outcomes from Site 1 in Period 1. It is then straightforward to calculate the marginal distribution of this outcome as a function of the intervention effect. Therefore, we will calculate that distribution for each of the 8 matched pairs (including the case of two programmes jointly matched to a third) and convolute those distributions to obtain the sampling distribution of the sum of sufficient statistics. Once we obtain the fully conditional sampling distribution of the sufficient statistic as described above, we will report the conditional maximum likelihood estimate of the intervention LOR with an exact, test-based 95% CI. The test of the null hypothesis at the two-tailed 0.05 significance level will be based on the exact two-tailed p value (using the point probability definition),³⁷ and will form the primary outcome analysis. In sensitivity analyses, we will also report the Wald, Score and Likelihood Ratio test results, which should be close to each other, given client numbers per site per period and the level of TVS from baseline CCP data.

Sample size

For the planning of the study, we used April 2012 to June 2014 CCP data to provide a set of marginal totals and proportions of TVS in 9 matched pairs of sites. We then prepared a simulation study with 10 000 replications to estimate the power of the primary test of intervention effect. For any given simulation replication, each site within the 9 matched pairs was randomly assigned to switch to the CCR at Period 1 or Period 2 (with independent randomisations per replication). For the site and period effects, we used the actual past Period 0 CCP data for the two sites to provide the within-pair site effect and the (randomly selected) second site’s TVS proportions in Period 1 and Period 2 to provide the period effects. We then applied a given intervention effect to Site 1 in Period 1 and to both sites in Period 2, for a set of plausible TVS proportions. For each such replication, we recorded the results of the exact conditional analysis described above. The pre-randomisation power of the primary test was estimated as the proportion of exact two-tailed p values ≤0.05.

The resulting detectable effect size (80% power with exact Type I error rate ≤0.05 two-tailed) was an OR of ~2.15. Since ORs overstate risk ratios (RRs) when the outcome proportion is common, to aid in interpretation table 4 indicates what detectable revised-CCP TVS proportions and RRs would correspond to various base proportions. The final two columns indicate what the power would have been for various other intervention-effect ORs. In summary, the planned study had a detectable OR of 2.15, corresponding to RRs between 1.37 and 1.53. Power estimates ranged between ~73% and 85% for true ORs between 2.00 and 2.25, respectively.

Table 4.

Power calculations for the Care Coordination Redesign effect on TVS (as originally planned)

Reference P [TVS]	Detectable P [TVS]	Risk ratio at Detectable P [TVS] for True OR=2.15	True OR	Power (%)
0.50	0.683	1.37	2.25	84.8
0.45	0.638	1.42	2.20	83.4
0.40	0.589	1.47	2.15	80.4
0.35	0.537	1.53	2.10	78.1
nb: Average P[TVS] among all sites in all			2.05	75.6
periods=0.437. Monte Carlo standard			2.00	72.8
error for power values is less than 0.5%.

Table 5 provides the corresponding detectable effect size and power values given current actual, post-randomisation numbers of eligible enrollees for Period 0 (n=172), Period 1 (n=385) and Period 2 (n=376) and assumed TVS proportions based on the actual proportion for Period 0. The table reflects lower enrolments than planned and the randomisation of 8 rather than 9 matched pairs. Because the randomisation of sites within pairs has already been set, the simulations for table 5 condition on this fact; that is, early-implementing and late-implementing sites are considered fixed as randomised. The post-randomisation power is somewhat greater than the pre-randomisation power, and this partially offsets the decrease in power due to lower-than-expected enrolments and a higher-than-anticipated base proportion of TVS. The detectable effect size (80% power with exact Type I error rate ≤0.05 two-tailed) is currently an OR of 2.75, corresponding to RRs between 1.34 and 1.54. Power estimates now range between ~77% and 84% for true ORs between 2.65 and 2.90, respectively.

Table 5.

Power calculations for the Care Coordination Redesign effect on TVS (as currently estimated)

Reference P [TVS]	Detectable P [TVS]	Risk ratio at Detectable P [TVS] for True OR=2.75	True OR	Power (%)
0.60	0.805	1.34	2.90	84.1
0.55	0.771	1.40	2.85	82.7
0.50	0.733	1.47	2.80	81.8
0.45	0.692	1.54	2.75	80.4
nb: Average P[TVS] among all sites in base period = 0.541. Monte Carlo standard error for power values is less than 0.5%.			2.70	78.6
			2.65	77.0

Patient and public involvement

Starting in 2016, the local HIV Planning Council (comprising practitioners, patients, advocates and researchers), as well as Care Coordination service providers not on the Planning Council, co-developed the revised intervention model with Health Department representatives. The plan for this trial was communicated to the Planning Council co-chairs, who provided letters of support in September 2017 after receiving the proposal aims. During a September 2017 public ‘town hall’ meeting to discuss the redesign and re-solicitation, Health Department staff described the intent to use phased implementation with random assignments for a side-by-side comparison of the two models. The approach was further outlined in the December 2017 RFP, which incorporated community feedback. The community was not involved in the methods for agency matching and randomisation, due to timing and potential conflicts of interest. Once awarded, the study team engaged 6 implementing service agencies as study partners and began Advisory Board meetings with those partners, who will advise on dissemination of findings and have contributed to instrument design and recruitment planning for the relevant parts of the larger study. There is no recruitment or primary data collection specifically for the trial.

Monitoring

The Health Department will oversee data monitoring and protocol compliance. Participation in the trial involves receipt of one model or another similar model of support services, rather than any medical device or treatment. As the intervention is delivered by contracted agency staff, and the analysis is based on secondary data sources, study investigators have no direct contact with human subjects for the purpose of the trial. To enrol in NYC HIV Care Coordination services, participants sign a consent form that covers the uses of data applicable to this trial. Given the determination of minimal risk and the routine nature of secondary analyses of merged programmatic-surveillance data as part of Health Department evaluation activities, the study team will not convene a Data Safety and Monitoring Board or conduct audits.

Ethics and dissemination

This trial was approved by the NYC Department of Health and Mental Hygiene Institutional Review Board (IRB) under protocol number: 18–009, and the CUNY Integrated IRB under protocol number: 018–0057, and is registered with ClinicalTrials.gov. The trial was granted a waiver of informed consent in accordance with the pre-2018 requirements set forth in 45 CFR (Code of Federal Regulations) 46.116(d), based on its reliance on secondary data analysis. Any changes to the trial eligibility criteria, outcome measures or analysis plans would be mutually agreed on between the CUNY and Health Department Principal Investigators, vetted with the PROMISE Advisory Board, and submitted to the IRB as protocol modifications. We anticipate no changes that would affect CCR or trial enrolment at this stage.

The full IRB protocol and statistical code will be made available by the investigators on request. Due to legal restrictions (New York Public Health Law Article 21, Title III) and the confidential nature of HIV surveillance data, the study team cannot release a de-identified individual-level public-use data set. NYC Health Department staff retain sole custody of the merged study data sets and are available to assist external researchers with any inquiries. Requests can be sent via email to hivreport@health.nyc.gov.

Results will be reported in accordance with the Consolidated Standards of Reporting Trials extension to cluster-randomised trials.³⁹ We will disseminate results of this study through scientific conference presentations, peer-reviewed publications and meetings with key stakeholders. Locally, results will be communicated annually at Health Department-convened meetings with CCR provider agencies, semi-annually in PROMISE Advisory Board meetings and approximately semi-annually at public HIV Planning Council meetings. The investigators have also been sharing this work with NIH (National Institutes of Health) leadership,⁴⁰ the Research Synthesis and Translation Team of the CDC Prevention Research Branch⁴¹ and interventionists and researchers in other jurisdictions.

Conclusion

Phasing in intervention implementation within a one-year period and using random agency assignment (within matched pairs) to early or delayed implementation offers a means of rigorously evaluating a set of changes to a major public-services programme, while ensuring fair and uninterrupted access to programme benefits in the eligible population. Intentionally staggered starts can also offer advantages in terms of managing the practical demands (eg, trainings, technical assistance and administrative work) of a large-scale programme rollout. Through robust health department-university partnerships that include joint planning of research in advance of key policy or practice initiatives, locally important research questions can be answered without substantially slowing the pace of desired change, and with methods that support knowledge generation and generalisability. In this case, NYC’s experience with implementing course corrections to a complex evidence-based HIV care intervention will yield findings that can valuably inform multiple jurisdictions’ efforts to advance progress along the HIV care continuum and ultimately end the epidemic.