Cost-effectiveness of Check It: A Novel Community-Based Chlamydia Screening and Expedited Treatment Program for Young Black Men

Abstract

Background

We assessed the cost-effectiveness of the Check It program, a novel community-based chlamydia screening and expedited partner treatment program for young Black men conducted in New Orleans since 2017.

Methods

We implemented a probabilistic cost-effectiveness model using a synthetic cohort of 16 181 men and 13 419 women intended to simulate the size of the Black, sexually active population in New Orleans ages 15–24 years.

Results

The Check It program cost $196 838 (95% confidence interval [CI]: $117 320–$287 555) to implement, saved 10.2 quality-adjusted life-years (QALYs; 95% CI: 7.7–12.7 QALYs), and saved $140 950 (95% CI: −$197 018 to −$105 620) in medical costs per year. The program cost $5468 (95% CI: cost saving, $16 717) per QALY gained. All iterations of the probabilistic model returned cost-effectiveness ratios less than $50 000 per QALY gained.

Conclusions

The Check It program (a bundled seek, test, and treat chlamydia prevention program for young Black men) is cost-effective under base case assumptions. Communities where Chlamydia trachomatis rates have not declined could consider implementing a similar program.

A synthetic cohort study of a novel community-based chlamydia screening program with expedited partner treatment for young Black men in New Orleans found the program cost-effective.

Chlamydia trachomatis (Ct) infected approximately 1.8 million people in the United States in 2018 [1]. Prior efforts at disease control have largely focused on screening women as they bear the brunt of serious disease sequelae, including pelvic inflammatory disease, infertility, and ectopic pregnancy. Despite decades of screening young women, prevalence rates have remained flat [2], suggesting that men may be serving as a reservoir of infection for women. While screening women has been shown to be cost-effective in nearly all scenarios and cost-saving in circumstances of even modest chlamydia prevalence rates [3, 4], less is known about the cost-effectiveness of screening men. Prior work has shown that screening high-risk men is likely cost-effective even without expedited partner treatment [5]. Other studies have shown that chlamydia screening interventions among women that include partner-notification strategies are also generally cost-effective [6, 7], yet no cost studies have examined partner treatment strategies among men who have sex with women. Further, previous cost-effectiveness analyses of screening strategies have not focused on health disparities, and most are not evaluations of screenings conducted in the community [8].

While the health risks posed to men are generally less dire than for women, men still account for 40% of chlamydia infections reported to the Centers for Disease Control and Prevention [9]. Prior modeling studies have speculated that increasing partner treatment rates in screening regimes targeting mainly females may be more cost-effective than increasing screening rates of males [10], but uptake of partner treatment programs remains low [11] and many barriers exist [12, 13]. One prior study has examined the cost-effectiveness of screening men and notifying partners [5], but no studies have examined the cost-effectiveness of screening men and providing expedited treatment for partners of Ct-infected men who have sex with women.

The goal of the Check It program was to determine if Ct screening of young Black or African-American men would reduce the rates of infection in women. Check It is a “seek, test, and treat” program based in nonclinical, community venues that began in May 2017. Check It enrolls Black men ages 15–24 years who have sex with women and spend most of their time in Orleans Parish (county) [14]. Check It focuses on Black men as Ct rates are 7.5 times higher than those among White men [15]. The program has been evaluated using agent-based modeling and it was found that each man screened prevented approximately 0.062 cases in men and 0.204 cases of chlamydia in women [16]. A study using Medicaid claims found the program resulted in a 1.69% decline in claims related to chlamydia positivity in the first year of its operation [17].

Men are recruited through active and passive methods including staffing events at community venues, referral from community partners, flyers/posters, social media, website, and peer referral. Venues include community-based locations such as historically Black colleges and universities (HBCUs), job training centers, and Black-owned businesses including barbershops, restaurants, and car washes. After providing consent, participants completed a research survey and provided a first-catch urine sample that was tested for chlamydia and gonorrhea with nucleic acid amplification tests. A modest monetary incentive ($25; ie, voucher for a haircut or gift card) was provided for completing these steps. When a participant tests positive for Ct or Neisseria gonorrhoeae, the program provides antibiotics, free of charge, for the young man and his sexual partner(s) either through partnering community pharmacies or via mail. The aim of this study was to determine if the Check It program was cost-effective.

METHODS

We modeled the impacts of the Check It program using a probabilistic cost-effectiveness model on a synthetic cohort of 16 181 men and 13 419 women intended to represent the size of the 15–24-year-old Black, sexually active population in New Orleans. We used a spreadsheet-based Monte Carlo simulation program [18] to capture uncertainties around the impacts of key model parameters (eg, baseline chlamydia incidence rate, impact of the Check It program, disease treatment costs, and staff wages) on program costs and quality-adjusted life-years (QALYs) gained. We used the societal perspective and included lost health, medical costs, and program cost.

The previous mathematical modeling study indicated that Check It program impacts increased as the program continued until reaching a plateau after 5 years of operation [19]. For this cost-effectiveness study we modeled a single year of the Check It program after it had reached this steady state. Health consequences of cases prevented during this single year of operation of the Check It program were tracked over a lifetime. As is standard in cost-effectiveness studies, long-term health and economic consequences of chlamydia-related disease endpoints were discounted at 3% annually [20]. All costs were converted to 2018 dollars using the medical care component of the Consumer Price Index [21].

We used previously published estimates of the incidence of chlamydia-related disease endpoints (Table 1), the costs of treating those symptoms (Table 1), and QALYs (Table 2). We used published estimates of the annual number of chlamydia cases for men and women from modeling based on National Health and Nutrition Examination Survey data [22].

Table 1.

Chlamydia Incidence Rates and Costs

Event	Base Incidence Rate per 100 000 Population	25th percentile	75th percentile	Reference		Reference
Chlamydia (Ct)a
Women	8219	7781	8699	Kreisel 2021 [22]		…
Men	4122	3646	4648	Kreisel 2021		…
	Incidence Rate per 100 Cases of Ctb	Reference	Base Cost in 2018 US$	5%	95%
Treatment for symptomatic acute infection
Women	30.00	Stratton 2000 [26]	$62.27	$62.27	$173.65	Gift 2008 [5]
Men	75.00	Stratton 2000	$62.27	$62.27	$173.65	Gift 2008
Epididymitis (men)
Inpatient	0.20	Stratton 2000	$1525.05	$1525.05	$1629.46	Gift 2006 [27]; Gift 2008
Outpatient	1.80	Stratton 2000	$464.63	$441.40	$487.86	Stratton 2000
Pelvic inflammatory disease (PID)
Outpatient	14.00	Stratton 2000	$432.52	$410.89	$454.14	Stratton 2000
Inpatient, surgical	0.80	Stratton 2000	$9299.11	$8834.15	$9764.06	Stratton 2000
Inpatient, nonsurgical	1.70	Stratton 2000	$6559.84	$6231.84	$6887.83	Stratton 2000
Outpatient after inpatient	2.50	Stratton 2000	$144.17	$136.96	$151.38	Stratton 2000
Chronic pelvic pain	3.00	Stratton 2000	$6127.60	$929.27	$13 660.26	Stratton 2000
Ectopic pregnancy
Outpatient	0.70	Stratton 2000	$2263.70	$185.85	$7396.99	Stratton 2000
Inpatient	0.70	Stratton 2000	$8270.50	$929.27	$14 589.53	Stratton 2000
Infertility	3.300	Stratton 2000	$3417.85	$2601.95	$10 593.67	Stratton 2000
Cervical cancer	0.021	Van Dyne 2018 [28]	$48 416.29	$38 308.77	$65 636.52	Chesson 2012 [29]; Kim & Goldie 2008 [30]
Local	0.063	Kim & Goldie 2008	$38 259.52	$16 771.08	$38 993.33	Goldie 2004 [31]
Regional cancer	0.034	Kim & Goldie 2008	$40 947.79	$20 848.80	$51 684.90	Goldie 2004
Distant cancer	0.004	Kim & Goldie 2008	$65 584.69	$22 511.87	$14 4895.10	Goldie 2004
Cervical intraepithelial neoplasia (CIN)b	0.270	Insinga 2004 [32]	$2659.57	$1968.19	$3350.95	Henk 2010 [33]
Other genital cancersb
Anal cancerc	0.054	Van Dyne 2018	$45 171.90	$21 837.25	$87 598.55	Chesson 2012; Hu & Goldie 2008 [34]; Goldie 2000 [35]
Oropharyngeal cancer c	0.056	Van Dyne 2018	$53 906.80	$25 081.64	$76 367.97	Chesson 2012; Hu & Goldie 2008; Lang 2004 [36]
Penile cancer (men)	0.009	Van Dyne 2018	$24 707.28	$12 228.86	$48 416.29	Chesson 2012; Hu & Goldie 2008; Stratton 2000
Vaginal cancer	0.008	Van Dyne 2018	$33 816.54	$25 331.21	$42 551.43	Chesson 2012; Hu & Goldie 2008; Fetters 2003 [37]
Vulvar cancer	0.043	Van Dyne 2018	$29 449.09	$19 341.56	$39 556.61	Chesson 2012; Hu & Goldie 2008; Beller 2006 [38]
Onset recurrent respiratory papillomatosis
AORRPb	0.002	Fortes 2017 [39]	$72 998.79	$35 039.42	$18 8336.9	Chesson 2012; Hu & Goldie 2008
JORRPb	0.004	Kim & Goldie 2008; Armstrong 2000 [40]; Derkay 1995 [41]	$187 176.39	$89 844.67	$48 2915.1	Chesson 2012; Hu & Goldie 2008
Genital warts	42.542	Insinga 2003 [42]	$1010.75	$511.62	$1160.494	Chesson 2012; Hu & Goldie 2008; Insinga 2003; Chesson 2004 [43]

Abbreviations: AORRP, adult-onset recurrent respiratory papillomatosis; Ct, Chlamydia trachomatis; JORRP, juvenile-onset recurrent respiratory papillomatosis.

^aBase Ct incidence rates are from published sources; minimum and maximums are from the National Health and Nutrition Examination Survey.

^bThe incidence rates are all age-reported for Ct cases in ages 15–24 years old, unless otherwise noted below. The incidence rate for CIN is the rate for all ages. The incidence rates for all other genital cancers (ie, anal, oropharyngeal, penile, vaginal, and vulvar) represent age-reported incidence rates for all persons <40 years old. JORRP incidence rate reflects age-reported incidence rate for adolescents <15 years old. AORRP incidence rate reflects the age-reported incidence rate for persons >14 years old.

^cAge-reported incidence rates for anal cancer and oropharyngeal cancer are each combined weighted averages of men and women incidence rates for each cancer. All costs are reported in 2018 US dollars and were adjusted using the medical care component of the Consumer Price Index for All Consumers [21]. 95% confidence intervals included.

Table 2.

Quality-Adjusted Life-Year Decrements

Event	QALY Decrement	Reference
Treatment for symptomatic acute infection
Women	0.010	[5]
Men	0.006	[5]
Epididymitis (men)
Inpatienta	0.005	[5, 26]
Outpatient	0.010	[5]
Pelvic inflammatory disease (PID)
Outpatient	0.009	[5]
Inpatient, surgical	0.007	[5]
Inpatient, nonsurgical	0.006	[5]
Outpatient after inpatient	0.005	[26]
Chronic pelvic painb	1.599	[5]
Ectopic pregnancyb
Outpatient	0.025	[5]
Inpatient	0.026	[5]
Infertilityb	1.036	[5]
Cervical cancerc	0.363	[30]
Local	0.240	[30]
Regional cancer	1.650	[30]
Distant cancer	0.520	[30]
Cervical intraepithelial neoplasia (CIN)	0.640	Melnikow 2010 [44]
Other genital cancers
Anal cancer	0.320	[30]
Oropharyngeal cancer	0.320	[30]
Penile cancer (men)	0.320	[30]
Vaginal cancer	0.320	[30]
Vulvar cancer	0.320	[30]
Onset recurrent respiratory papillomatosis
AORRP	0.310	[30]
JORRP	0.310	[30]
Genital warts	0.022	[30]

Abbreviations: AORRP, adult-onset recurrent respiratory papillomatosis; JORRP, juvenile-onset recurrent respiratory papillomatosis; QALY, quality-adjusted life-year.

^aTo calculate epididymitis inpatient QALY decrement, the health utilities from Stratton et al [26] were applied to the health utilities in Gift et al [5].

^bQALY decrement for chronic pelvic pain, ectopic pregnancy (inpatient and outpatient), and infertility were each discounted to include a delay in onset of 2 years, 5 years, and 10 years, respectively (Gift et al [5]). This adjustment also included a discount of the probability of dying between age 19 (the midpoint of our sample) and age 21 of 0.14%, age 19 and age 24 of 0.41%, and age 19 and age 28 of 0.97% for each value respectively drawn from the National Vital Statistics [45].

^cThe overall cervical cancer QALY decrement reflects a weighted average of stage-specific utility weights and distribution of disease according to stage. Discounted annual rate of 3%. Beta distribution.

We first determined the annual number of chlamydia cases separately for men and women and then determined the number of each sequelae likely to result from those cases in the absence of the Check It program. To model the Check It program we used impact estimates of the Check It program from mathematical modeling of 95.59 (95% confidence interval [CI]: 65.56–125.61) fewer cases for women and 50.97 (95% CI: 31.09–70.84) fewer cases for men in a single year [19].

To calculate program costs for a single year of operation, we reviewed staff wages and supply costs from 2019. As the program also contained a research component, we used time diaries and a time-motion study to determine what percentage of staff time was spent on program activities compared with research activities. Time-motion measurements were conducted at 4 study sites every 6 months, and time diaries were collected annually [23]. The administrator spent an average of 54.4% of a typical 40-hour week on program activities. While community health workers did spend some time administering research study surveys, much of their time on site was spent waiting for new patients to enroll. Since community health worker time on site was not a binding constraint, we assigned 100% of their hourly labor costs to program activities. Wages and distributions were drawn from Occupational Employment Statistics published by the Bureau of Labor Statistics. We used code 21-1094 “Community Health Workers” to represent our on-the-ground program staff, who had a national mean hourly wage of $20.90 (80% CI: $12.50–$31.60) and code 11-0000 “Management Occupations” after restricting to North American Industry Classification System (NAICS) code 999300 “Local Government” to represent the program administrator with a national mean hourly wage of $44.63 (80% CI: $11.17–$75.66).

As part of developing the Check It program, a graphics design firm developed logos, palettes, and images for printed and web promotional materials. We include the full amount of this cost ($6395), even though this cost is a one-time fixed cost. While we only model a single year of the Check It program here, we include this as a conservative estimate of cost-effectiveness. This effectively assumes a screening program that only lasts a single year with respect to the graphics development costs, even though a screening program that ran for several years would be correct to amortize this cost over the life of the program.

Our base case (aka reference case) estimates of the cost-effectiveness of the Check It program do not include the cost of the participation incentive as other locales implementing the Check It protocol would not need to administer a lengthy research survey in addition to programmatic activities. However, it is possible that our participation rates from young Black men were higher due to this incentive, so we show one-way sensitivity analysis that includes the cost of this participant incentive.

Cost inputs were drawn from log-normal distributions, and all other parameters were drawn from normal distributions. To obtain a broad distribution of input parameters during multivariate sensitivity testing, we performed Latin Hypercube sampling. We continued simulation until there was a 95% chance that the mean estimate of each of the parameters listed in the output table was within 5% of its true value [24]. We produced 95% CIs on key outputs, a tornado diagram showing which inputs had the largest impact on the cost-effectiveness ratio, as well as a scatterplot of simulation iterations in a cost-effectiveness plane.

RESULTS

Under base case parameter assumptions, the Check It program cost $196 838 (95% CI: $117 320–$287 555) annually to implement, saved 10.2 QALYs annually (95% CI: 7.7, 12.7 QALYs), and saved $140 950 (95% CI: −$197 018 to −$105 620) in medical costs per year, which is reflected in a cost-effectiveness ratio of $5468 (95% CI: cost saving, $16 717) per QALY gained (Table 3).

Table 3.

Cost-effectiveness of the Check It Program

	No Check It	Check It	Change	95% CI
Chlamydia cases (total)	1956.0	1809.5	−146.6	−176.5 to −116.3
Women	907.4	811.8	...	...
Men	1048.7	997.7	...	...
QALYs (total)	−105.2	−95.0	10.2	7.7–12.7
Women	-90.0	-80.5	...	...
Men	−15.2	−14.4	...	...
Medical cost	$1 650 840	$1 509 890	−$140 950	−$197 018 to −$105 620
Program cost (total)	$0	$196 838	$196 838	$117 320–$287 555
Staff wages	...	$172 728	...	...
Campaign design	...	$6395	...	...
Recurring marketing	...	$5471	...	...
Labs	...	$7423	...	...
Treatment	...	$1208	...	...
Travel	...	$3614	...	...
Cost per QALY	...	...	$5468	Cost savings, $16 717

The table shows the cases of chlamydia (Ct), QALYs lost, Ct-related medical cost, and Check It program cost in New Orleans with and without the Check It program.Abbreviations: CI, confidence interval; Ct, Chlamydia trachomatis; QALY, quality-adjusted life-year.

For multivariate sensitivity testing, the convergence conditions specified in the Methods section were achieved after 1800 model iterations. Program costs were smaller than medical costs saved (eg, the program was cost-saving) in 18% of runs and costs in 95% of runs were less than $16 700 per QALY (Figure 1). Figure 2 plots the outcomes of each of these runs in a cost versus QALY plane. All model iterations indicated costs less than $50 000 per QALY.

Figure 1.

Check It program acceptability curve. The figure indicates the percentage of model iterations where the Check It program was more cost-effective than the indicated threshold. Abbreviation: QALY, quality-adjusted life-year.

Figure 2.

Cost per QALY plane. Each dot indicates the net costs and QALY change in a model iteration. Points to the right of the $50 000/QALY line cost less than $50 000 per QALY. Abbreviation: QALY, quality-adjusted life-year.

The tornado diagram (Figure 3) is organized to display the average cost-effectiveness ratio among simulation iterations when the indicated parameter is drawn from the bottom (top) 10% of values. The diagram indicates that the cost-effectiveness ratio was most sensitive to the value used for staff wages, as that made up the bulk of program costs. When the staff wages were drawn from the bottom 10% of the distribution of staff wages, the program saved $1981.72 per QALY gained; and when staff wages were drawn from the top 10% of the distribution for staff wages, the program cost $13 306.34 per QALY gained. The next most important parameter in the simulation was the number of cases of chlamydia prevented among females followed by the administrator’s hourly wage.

Figure 3.

Tornado diagram of the most influential model inputs on cost per QALY of the Check It program. Model inputs are ordered by importance of impact on cost-effectiveness of Check It program. A large bar indicates large possible impact. “Baseline” indicates average cost-effectiveness across all iterations in multivariate sensitivity analysis (rather than base case). Abbreviations: Admin, administrator’s; Ct, Chlamydia trachomatis; QALY, quality-adjusted life-year; Tx, treatment.

When we conducted one-way sensitivity analysis and included the cost of incentives for study participants as a program cost, rather than excluding it as a research cost, the cost-effectiveness ratio increased (ie, became less cost-effective) to $6897 (95% CI: cost saving, $18 556) per QALY (Table 4).

Table 4.

Cost-effectiveness of the Check It Program Including Research Study Incentive

	No Check It	Check It	Change	95% CI
Chlamydia cases (total)	1956.0	1809.5	−146.6	−176.9 to −116.2
Women	907.4	811.8
Men	1048.7	997.7
QALYs (total)	-105.2	-95.0	10.2	7.7–12.7
Women	-90.0	-80.5
Men	-15.2	-14.4
Medical cost	$1 650 840	$1 509 890	−$140 950	− $196 749 to −$105 723
Program cost (total)	$0	$211 438	$211 438	$130 910–$303 147
Staff wages		$172 728
Campaign design		$6395
Recurring marketing		$5471
Labs		$7423
Treatment		$1208
Travel		$3614
Incentive		$14 600
Cost per QALY			$6897	Cost savings, $18 556

See notes for Table 3. Results include a $25 participation incentive for study participants. Abbreviations: CI, confidence interval; QALY, quality-adjusted life-year.

Discussion

The Check It chlamydia prevention program for young Black men cost $5468 per QALY gained in the base case. One standard threshold of cost-effectiveness is $50 000 per QALY [25]. Programs that cost less than $50 000 per QALY are deemed cost-effective (eg, they give good value for the money spent). In our Check It analysis, since 100% of our model simulation runs were below this standard threshold, Check It was cost-effective by this standard benchmark (Figure 2). This figure also compares favorably with prior work that found that screening men in the community setting without expedited partner treatment cost $13 189 per QALY (in 2018 dollars) [5]. Neither of the strategies targeting men for screening are as efficient as those that target women, as strategies for screening women have been found to be cost-saving [3]. However, screening men may be viewed as a cost-effective complement to screening women, rather than a substitute.

The key limitation of this study is that the impact of the Check It program is based on mathematical modeling. While this limitation applies to all population-based screening approaches, the true impact of the program may be higher (lower) indicating that the cost-effectiveness ratio in this paper is biased down (up). In future work we intend to estimate chlamydia diagnosis rates in the parish (county) where Check It was conducted and compare those rates to similar parishes (counties) in Louisiana. A second limitation is that the program was conducted in New Orleans, which has a particular density of young Black men. Areas attempting to implement the Check It protocol with lower (higher) densities may find that it takes more (less) staff time to reach the same number, which, in turn, would drive up (down) the cost-effectiveness ratio.

The Check It program returned approximately 75% of program costs in saved medical cost. The program cost only $5468 per QALY gained, which was cost-effective using a standard threshold of $50 000 per QALY gained. The Check It program decreased chlamydia burden among Black men and women, who have a higher disease burden than White men and women, thus decreasing health disparities. Localities that already have robust screening programs targeting females may consider implementing male screening with expedited partner treatment in community settings as a complementary program.

Notes

Financial support. This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the National Institute of Allergy and Infectious Diseases (NIAID) (grant number R01-HD086794) (payments made to Tulane University).

Potential conflicts of interest. Z. Q. reports travel reimbursement (personal payments) for the following: Second Sectional Meeting of the Society for Industrial and Applied Mathematics TX/LA Section in Dallas, Texas (1–3 November 2019); International Conferences on Mathematical Modeling and Analysis of Populations in Biological Systems VII: Seventh International Conference on Mathematical Modeling and Analysis of Populations in Biological Systems (12–14 October 2019); and Population Health Sciences Seminar at Georgia State University (24 October 2019). P. J. K. reports support from the Gates Foundation (via subcontract to 2 University of Washington studies) and receiving travel support from the American Sexually Transmitted Disease Association to attend the The International Society for Sexually Transmitted Diseases Research meeting.All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.