Real-world comparative effectiveness of mRNA-1273 and BNT162b2 vaccines among immunocompromised adults identified in administrative claims data in the United States

Abstract

Introduction

Head-to-head studies comparing COVID-19 mRNA vaccine effectiveness in immunocompromised individuals, who are vulnerable to severe disease are lacking, as large sample sizes are required to make meaningful inferences.

Methods

This observational comparative effectiveness study was conducted in closed administrative claims data from the US HealthVerity database (December 11, 2020-January 10, 2022, before omicron). A 2-dose mRNA-1273 versus BNT162b2 regimen was assessed for preventing medically-attended breakthrough COVID-19 diagnosis and hospitalizations among immunocompromised adults. Inverse probability of treatment weighting was applied to balance baseline characteristics between vaccine groups. Incidence rates from patient-level data and hazard ratios (HRs) using weighted Cox proportional hazards models were calculated.

Results

Overall, 57,898 and 66,981 individuals received a 2-dose regimen of mRNA-1273 or BNT161b2, respectively. Among the weighted population, mean age was 51 years, 53 % were female, and baseline immunodeficiencies included prior blood transplant (8%–9%), prior organ transplant (7%), active cancer (12%–13%), primary immunodeficiency (5–6%), HIV (20%–21%), and immunosuppressive therapy use (60%–61%). Rates per 1,000 person-years (PYs; 95% confidence intervals [CI]s) of breakthrough medically-attended COVID-19 were 25.82 (23.83–27.97) with mRNA-1273 and 30.98 (28.93, 33.18) with BNT162b2 (HR, 0.83; 95% CI, 0.75–0.93). When requiring evidence of an antigen or polymerase chain reaction test before COVID-19 diagnosis, the HR for medically-attended COVID-19 was 0.78 (0.67–0.92). Breakthrough COVID-19 hospitalization rates per 1,000 PYs (95% CI) were 3.66 (2.96–4.51) for mRNA-1273 and 4.68 (3.91–5.59) for BNT162b2 (HR, 0.78; 0.59–1.03). Utilizing open and closed claims for outcome capture only, or both cohort entry/outcome capture, produced HRs (95% CIs) for COVID-19 hospitalization of 0.72 (0.57–0.92) and 0.66 (0.58–0.76), respectively.

Conclusions

Among immunocompromised adults, a 2-dose mRNA-1273 regimen was more effective in preventing medically-attended COVID-19 in any setting (inpatient and outpatient) than 2-dose BNT162b2. Results were similar for COVID-19 hospitalization, although statistical power was limited when using closed claims only.

Study registration

NCT05366322.

1. Introduction

As of April 26, 2022, in the United States, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused more than 80.2 million confirmed coronavirus 2019 (COVID-19) cases and more than 983,000 deaths [1], with actual cases estimated to be 5 to 50 times greater than those reported and incidence fluctuating continuously [1], [2]. Comprising ∼3% of the adult US population, immunocompromised individuals are especially vulnerable to COVID-19, with greater risk of serious or prolonged illness [3], [4].

Three COVID-19 vaccines are available in the United States: mRNA-1273 (Spikevax®; Moderna, Inc., Cambridge, MA), BNT162b2 (COMIRNATY®; BioNTech, Mainz Germany, and Pfizer Inc, New York, NY), and Ad26.COV2.S (Janssen Biotech, Inc, Horsham, PA). At the time of this study, mRNA-1273 had been authorized by the US Food and Drug administration (FDA) as a 2-dose series (100-μg mRNA per dose) in individuals aged ≥ 18 years [5], [6]; BNT162b2 had also been authorized by the FDA as a 2-dose series (30-μg mRNA per dose) in individuals aged ≥ 16 years and authorized under Emergency Use Authorization (EUA) as a 2-dose series in individuals aged 5 to 15 years [7], [8], [9]; and Ad26.COV2.S had been authorized under EUA as a single-dose (5 × 10¹⁰ virus particles per dose) in individuals aged ≥ 18 years [10]. Booster doses had also been authorized under EUA, administered ≥ 5 months after the primary series as a half dose for mRNA-1273 (50 μg) and a full dose for BNT162b2 (30 μg) and ≥ 2 months after the primary dose for Ad26.COV2.S [5], [7], [10], [11]. While all 3 vaccines were approved for use in the United States, the Advisory Committee on Immunization Practice (ACIP) and the Centers for Disease Control and Prevention (CDC) advise that the use of an mRNA vaccine is recommended over Ad26.COV2.S due to greater vaccine effectiveness (VE) and lower risk of serious adverse events (ie, thrombosis with thrombocytopenia syndrome [TTS]) [12], [13].

Although high vaccine efficacy was reported in phase 3 trials of mRNA-1273 and BNT162b2 in the general population, these trials excluded immunocompromised individuals, including those with underlying immunocompromising conditions and those prescribed immune-modifying therapies (with the exception of individuals with stable HIV and very low viral load) [14], [15], [16]. Subsequent real-world data in the vaccinated immunocompromised population showed attenuated VE, with a higher risk of infection, hospitalization, death, persistent infection and shedding, viral evolution, reduced antibody and neutralization titers, and infection of household contacts [17]. Given these increased risks, a third and fourth dose of mRNA-based vaccines had been recommended for this vulnerable group [17], [18], [19]. An additional primary dose of mRNA-1273 (100 μg; individuals aged ≥ 18 years) or BNT162b2 (30 μg; individuals aged ≥ 12 years) was authorized for administration ≥ 1 month after completion of the primary series in moderately to severely immunocompromised individuals [5], [7]. For moderately to severe immunocompromised individuals, the CDC recommended 1 mRNA-based booster be given ≥3 months after an mRNA-based primary vaccination series and ≥2 months after an Ad26.COV2.S primary dose [20].

Head-to-head studies comparing the VE of the COVID-19 vaccines in immunocompromised individuals have not been conducted to date; however, numerous studies have investigated antibody-mediated immunogenicity in response to vaccination and reported a more robust immune response to mRNA-1273 compared to BNT162b2 in this vulnerable cohort [3], [17], [21], [22], [23], [24]. It is hypothesized that these differences in immunogenicity between mRNA vaccines may translate into differences in VE against COVID-19 infections and hospitalizations in immunocompromised individuals. One observational study of immunocompromised adults vaccinated with 2 doses of an mRNA COVID-19 vaccine reported VE against COVID-19 hospitalizations to be 81% (95% CI, 76%–85%; n = 4,337) with mRNA-1273, and 71% (95% CI, 65%–76%; n = 6,227) with BNT162b2; however, direct comparison of vaccines in the immunocompromised population cannot be made, as the test-negative study design was not intended for head-to-head statistical comparisons [25].

In this study, we compared the real-world VE specifically of 2-dose mRNA-based vaccines, mRNA-1273 and BNT1262b2, against medically-attended breakthrough COVID-19 diagnosis and COVID-19 hospitalizations in immunocompromised adults (NCT05366322).

2. Methods

2.1. Objectives

The primary objective was to compare the real-world VE of 2 doses of mRNA-1273 versus 2 doses of BNT162b2 against medically-attended breakthrough COVID-19 diagnosis among immunocompromised adults. The secondary objective was to compare the real-world VE of 2 doses of mRNA-1273 versus 2 doses of BNT162b2 vaccine against breakthrough COVID-19 hospitalizations among immunocompromised adults.

2.2. Study design and data source

This observational comparative VE cohort study utilized medical and pharmacy claims data from December 11, 2020, through January 10, 2022 (the available time period in the data when the delta variant dominated COVID-19 infection) aggregated by HealthVerity. HealthVerity data was drawn from a variety of US sources, including 2 closed claims databases (Private Source 17 and Private Source 20) and numerous open claims data sources, with data elements including provider-submitted claims, adjudicated insurance claims, and pharmacy billing manager claims submissions. These sources include individuals insured under commercial, Medicare (limited available claims data) or Medicaid plans, or providers participating in several large medical and pharmacy insurance claims submission systems (eg, medical/pharmacy claims clearinghouses). The primary analysis was limited to closed medical claims only and data were truncated 3 months (to October 12, 2021) from the most recently available data at the time of the analysis to allow for claims adjudication (Table S1).

The HealthVerity database includes demographic variables such as age, sex, and 3-digit zip level; race and ethnicity are not available. Hospitalizations are included in the data at a summary level. Open claims are sourced directly through medical clearinghouses and pharmacy benefit managers; there is no associated enrollment file, resulting in incomplete capture of healthcare system interactions. Closed claims represent claims accepted by and paid by health insurance companies and generally lagged 3 to 6 months to allow for full adjudication.

To create linkages across data feeds and ensure de-identified, longitudinal, de-duplicated patient data, all data partners use the HealthVerity technology within their system to create a unique, secure, encrypted, and non-identifiable patient token that is usable across datasets. Use of data and the precise granularity available is controlled by Health Insurance Portability and Accountability Act (HIPAA) requirements or application of public health exemption. No protected health information or personal identifying information leaves the data owner’s possession, and all research data were certified HIPAA compliant by expert determination.

The study follows the Guidelines for Good Epidemiologic Practice practices laid out in 2005 US FDA Good Pharmacy Practice and the 2008 International Society of Pharmacoepidemiology Good Pharmacy Practice [26], [27]. This study was exempt from review by the New England Institutional Review Board (#1–9757—1). All participant data were de-identified, and all participant-level and provider-level data within the database contained synthetic identifiers to protect the privacy of individuals and data contributors.

2.3. Vaccine groups

Vaccinations were captured in the database via manufacturer-specific current procedural terminology (CPT) and national drug code (NDC; Table S2). Drugs dispensed by a pharmacy were captured, while over-the-counter medications and those provided during an inpatient stay were not. A 2-dose regimen was defined as 2 doses of mRNA-1273 (exposure group) or BNT1262b2 (referent group). An exploratory analysis of the distribution of days between administration of 2 doses of vaccine found the fifth percentile for mRNA-1273 or BNT162b2 to be 25 days and 18 days, respectively. Given these results and the aim of inclusivity with respect to dose 2 in this high-risk population, a minimum of 14 days between doses was required between receiving 2-dose mRNA-1273 or BNT162b2 vaccines The index date was defined as the date of vaccine regimen completion (ie, date of second dose of mRNA-1273 or BNT1262b2) (Fig. 1 ).

Fig. 1.

Study design schema. Immunocompromised adults who had completed a 2-dose homologous vaccine regimen were identified via central procedural terminology (CPT) and nation drug codes (NDC) from December 11, 2020, through January 10, 2022.

2.4. Population and follow-up

Individuals aged ≥ 18 years who received 2 doses of mRNA-1273 or BNT162b2 and were continuously enrolled in a medical and pharmacy plan for 365 days before the index date were included in the analysis (Fig. 1).

Immunocompromised individuals were identified using an adapted claims-based algorithm aligning with the CDC definition of immunocompromised individuals eligible for COVID-19 vaccine booster doses (Table S2) [28]. Immunocompromised individuals were identified as those meeting at least one of the following criteria: evidence of blood or stem cell transplant 2 years before the index date; history of organ transplant and taking immunosuppressive therapy 60 days before the index date; evidence of active cancer treatment 180 days before the index date with an active cancer diagnosis in the year before treatment; any history of primary immunodeficiency disorder; or any history of HIV.

Exclusion criteria included evidence of COVID-19 infection before the index date or in the 13 days following the index date, receipt of a heterologous COVID-19 vaccine in the 365 days before or in the 13 days following the index date, receipt of additional dose of homologous COVID-19 vaccine in the 13 days following the index date, or missing or unknown sex on the index date. Additional exclusion details are presented in the Supplementary text.

The baseline period was defined as the 365 days before the index date. Participants were followed from 14 days following completion of a 2-dose vaccine regimen until an outcome of interest, receipt of a third COVID-19 vaccine dose (heterologous or homologous), disenrollment from a medical/pharmacy plan, or October 12, 2021, whichever occurred first.

2.5. Study outcomes

The primary outcome of medically attended breakthrough COVID-19 diagnosis was defined as a claim with the ICD-10 diagnostic code for COVID-19 (U07.1) in any setting, including inpatient, outpatient, emergency department, and urgent care. The secondary outcome of breakthrough hospitalization for COVID-19 was defined as a hospital stay with the ICD-10 diagnostic code for COVID-19 (U07.1) listed as the primary diagnosis or within 21 days prior to hospital admission.

A phased approach was used, including an exploratory phase to inform key design decisions (see details in study protocol: NCT05366322) and a diagnostic phase to ensure baseline balance and positivity were achieved and to assess the existence of non-differential censoring. Once the pre-specified diagnostic criteria were met, the association between vaccine group and the study outcomes was assessed in the inferential phase during which the outcome models were implemented. Further details are presented in the Supplementary text.

2.6. Statistical analysis

The distributions of baseline variables within each vaccine cohort were described as number and percentage for categorical variables and as mean (standard deviation), median (interquartile range), and range (minimum, maximum) for continuous variables. Baseline variables hypothesized to be confounding variables were included in a propensity score (PS) model; age, sex, payer type, state of residence, and calendar time were assessed at the index date. Healthcare resource utilization, the Charlson comorbidity score, frailty score, and number of unique immunosuppressive therapies used were assessed in the 365 days before the index date. Individual comorbid conditions were assessed using all prior claims available. The algorithms used to identify immunocompromised adults are presented in Table S3, and a full list of baseline covariates of interest are presented in Table S4. Inverse probability of treatment weights (IPTW) was calculated as 1/PS for participants in the mRNA-1273 group (exposed) and as 1/(1-PS) for participants in the BNT162b2 group (referent). Exposure and referent groups were considered balanced if the absolute standardized differences (ASDs) for all baseline covariates used to generate the PS were < 0.10 [29], [30], [31].

Incidence rates per 1,000 PYs with corresponding 95% confidence intervals (CIs) were estimated for each treatment group. Kaplan–Meier plots with 95% CIs and Schoenfeld residuals were generated to assess comparative VE over time. Weighted Cox proportional hazard models were executed to estimate hazard ratios (HRs) and corresponding 95% CIs. Subgroup analyses were implemented for age categories, calendar quarter of vaccine receipt, COVID-19 transmission level at index date, and immunocompromised subgroup.

Analyses were conducted using the Aetion Evidence Platform®, software for real-world data analysis, which has been scientifically validated for observational cohort studies using large healthcare databases [32]. Transformations of the raw data are preserved for full reproducibility and audit trails are available, including a quality check of the data ingestion process.

2.7. Sensitivity analyses

Several per-protocol sensitivity analyses were conducted within this study (Table S1). These included evaluating the impact of the adjudication lag in closed claims, where the primary analysis was implemented through January 10, 2022, without truncation. Another sensitivity analysis used an alternative definition of medically-attended breakthrough COVID-19 diagnosis that required evidence of a rapid antigen test on the same day or 3 days before the COVID-19 diagnosis code or a polymerase chain reaction test 3 to 14 days before the COVID-19 diagnosis code.

In addition, analyses were conducted utilizing both closed and open claims for cohort entry and outcomes. As open claims are not associated with an enrollment file, individuals without an enrollment file and identified via open claims only were required to have at least one open medical and open pharmacy claim in the 365 days before the index date. During follow-up, individuals without an enrollment file or those who disenrolled from their medical/pharmacy plan continued to be followed-up if there was evidence of a medical or pharmacy claim every 60 days. Open claims with a diagnosis code for COVID-19 were incorporated into the primary and secondary outcomes.

A post hoc sensitivity analysis was performed, which required individuals to have an enrollment file on the index date, with continuous enrollment in the 365 days before and continuous enrollment during follow-up, while allowing for capture of the outcomes of interest utilizing both closed and open claims.

3. Results

A total of 124,879 immunocompromised adults identified through HealthVerity database records from December 11, 2020, through October 12, 2021, were included in the primary analysis; 57,898 completed a 2-dose vaccination with mRNA-1273 and 66,981 completed a 2-dose vaccination with BNT162b2 (Fig. 2 ). Prior to weighting, compared to the BNT162b2 group, individuals in the mRNA-1273 group were slightly older (mean ages of 50 and 52 years, respectively) and there were slightly less women in the population (53% and 52%, respectively; Table S5) before weighting. The distribution of individuals in California, New York, and Pennsylvania was higher for the mRNA-1273 group, while BNT162b2 was more represented in in Illinois, New Jersey, and Texas. Those in the mRNA-1273 vaccine group had a slightly higher frequency of clinical comorbidities (Table S5).

Fig. 2.

Participant attrition diagram. Participant attrition flow diagram using claims from the HealthVerity database. Numbers represent the patient size before inverse probability of treatment weighting.

Inverse probability of treatment weighting resulted in a pseudopopulation of 124,588 mRNA-1273 individuals and 124,313 BNT162b2 individuals (Table S7). Following weighting, exposure and referent groups were considered balanced, as ASDs for all baseline covariates were < 0.10. The mean age of the participants was 51 years, 53% were female, and participants were primarily insured by commercial insurance (62%) or Medicaid (32%) (Table 1 ). Texas (16%), Illinois (13%), California (9%), New Jersey (8%), and Michigan (6%) were the top 5 represented states (Table S5). Clinical history was balanced between the vaccine groups with respect to a history of arrythmia (25%), prior cardiovascular disease (64%), hypertension (52%), prior heart failure (7%), chronic lung disease (23%), obesity (32%), diabetes (22%), and irritable bowel syndrome (47%). Within this immunocompromised population, 8–9% had received a prior blood transplant, 7% had a prior organ transplant, 12–13% had a cancer diagnosis with active treatment, 5–6% had a primary immunodeficiency, 20–21% had HIV, and 60–61% used an immunosuppressive therapy 60 days before the index date (Table 1).

Table 1.

Pseudopopulation baseline characteristics of immunocompromised adults with a 2-dose regimen of mRNA-1273 or BNT162b2.^a

	mRNA-1273	BNT162b2	Weighted absolute standardized difference
Populations
Individuals, n	57,000	66,757	–
Pseudopopulation (sum of weights)	124587.74	124312.52	–
General characteristics
Mean age, n (SD)	50.9 (13.44)	50.8 (13.47)	0.009
Female sex, n (%)	65,568 (52.6)	65,440 (52.6)	0.000
Primary payer type (%)
Commercial	77,545 (62.2)	77,445 (62.3)	0.001
Medicare	4,920 (4.0)	4,831 (3.9)	0.003
Medicaid	40,095 (32.2)	40,014 (32.2)	0.000
Missing	2,028 (1.6)	2,022 (1.6)	0.000
Comorbidities, n (%)
Arrhythmia	31,395 (25.2)	31,300 (25.2)	0.001
Blood transplantb	10,128 (8.1)	10,697 (8.6)	0.017
Cancer diagnosis with active treatmentc	15,405 (12.4)	15,525 (12.5)	0.004
Cardiovascular disease	79,186 (63.6)	78,953 (63.5)	0.001
Chronic lung disease	28,370 (22.8)	28,300 (22.8)	0.000
Diabetes (type I or II)	27,524 (22.1)	27,401 (22.0)	0.001
HIV diagnosisd	25,492 (20.5)	24,377 (19.6)	0.021
Hypertension	64,728 (51.9)	64,516 (51.9)	0.001
Irritable bowel syndrome	58,056 (46.6)	57,866 (46.6)	0.001
Obesitye	40,296 (32.3)	40,181 (32.3)	0.000
Organ transplantd	8,997 (7.2)	9,177 (7.4)	0.006
Primary immunodeficiencyd	6,972.5 (5.6 %)	6,874.2 (5.5 %)	0.003
Prior heart failure	8,823 (7.1)	8,773 (7.1)	0.001
Use of immunosuppressive therapyf	74,842 (60.1)	75,223 (60.5)	0.009
Charlson comorbidity index score (%)
Score=0	35,495.1 (28.5)	35,466.9 (28.5)	0.001
Score=1	24,114.0 (19.4)	24,056.9 (19.4)	0.000
Score ≥ 2	64,978.7 (52.2)	64,788.7 (52.1)	0.001
Frailty score (%)
Robust (0–0.149)	89,675.0 (72.0)	89,518.4 (72.0)	0.001
Prefrail (0.15–0.249)	31,495.3 (25.3)	31,390.5 (25.3)	0.001
Mild frailty (0.25–0.349)	2,994.0 (2.4)	2,981.6 (2.4)	0.000
Moderate to severe frailty (≥0.35)	423.5 (0.3)	422.0 (0.3)	0.000
Mean no. of events,g n (SD)
Hospitalization events	1.57 (7.68)	1.56 (7.03)	0.001
Outpatient events	34.79 (53.65)	34.75 (52.81)	0.001
Mean unique immunotherapies,g n (weighted SD)	0.92 (0.99)	0.92 (0.98)	0.000

SD, standard deviation.

^aInverse probability weighted pseudopopulation; ^bIn 730 days before cohort entry date; ^c In the year before initiation of vaccination; ^dAny history; ^bBody mass index ≥ 30 kg/m²; ^f60 days before cohort entry date; ^gOver 365-days before the baseline period.

Results from the primary analysis showed that the overall rate of medically-attended COVID-19 was lower among individuals who received 2 doses of mRNA-1273 (25.82 per 1,000 PYs; 95% CI, 23.83–27.97) compared those who received 2 doses of BNT162b2 (30.98 per 1,000 PYs; 95% CI, 28.93–33.18; HR, 0.83; 95% CI, 0.75–0.93) (Table 2 ; Fig. 3 A). In subgroup analyses, results were similar across categories of transmission level, calendar quarter of completion of the 2-dose regimen, and age, although the small numbers of individuals in the subgroups of those who received second doses administered in Q3 2021 and individuals ≥65 years, limited interpretation of these results (Table S8). Among immunocompromised subgroups, the rate of medically-attended COVID-19 was lower with mRNA-1273 than BNT162b2 vaccination for individuals with active cancer (HR, 0.69; 95% CI, 0.50–0.95). The rate of medically-attended COVID-19 was highest for individuals who had undergone blood/stem cell transplant (44.1 and 45.8 per 1,000 PYs for mRNA-1273 and BNT162b2, respectively) or solid organ transplant (56.8 and 58.9 per 1,000 PYs for mRNA-1273 and BNT162b2, respectively), with no difference observed between mRNA vaccines administered.

Table 2.

Incidence and VE of mRNA-1273 and BNT162b2 on medically-attended breakthrough COVID-19 diagnosis (primary outcome) and against breakthrough COVID-19 hospitalizations (secondary) among immunocompromised individuals.

	No. of events		Rate (95 % CI) (per 1000 PYs)		HR mRNA-1273 vs BNT162b2		Follow-up time; days [IQR]		Median days to event [IQR]
	mRNA-1273	BNT162b2	mRNA-1273	BNT162b2	HR (95 % CI)	P-value	mRNA-1273	BNT162b2	mRNA-1273	BNT162b2
Medically-attended COVID-19	617	840	25.82 (23.83–27.97)	30.98 (28.93–33.18)	0.83 (0.75–0.93)	0.001	159 [130–182]	158 [123–181]	114.00 [77.00–148.00]	111.00 [72.00–139.00]
COVID-19 hospitalization	90	125	3.66 (2.96–4.51)	4.68 (3.91–5.59)	0.78 (0.59–1.03)	0.082	160 [130–182]	159 [124–181]	101.00 [63.00–133.00]	110.50 [62.25–141.25]

CI, confidence interval; HR, hazard ratio; IQR, interquartile range; PY, person-yea.

Fig. 3.

Comparative VE over time. Kaplan–Meier plots with 95% confidence intervals and Schoenfeld residuals over time. A) Medically-attended COVID-19; B) COVID-19 hospitalization.

The rate of breakthrough COVID-19 hospitalization was 3.66 per 1,000 PYs (95% CI, 2.96–4.51) among individuals who received 2 doses of mRNA-1273 and 4.68 per 1,000 PYs (95% CI, 3.91–5.59) among those who received 2 doses of BNT162b2, with a corresponding HR of 0.78 (95% CI, 0.59–1.03) (Table 2; Fig. 3B). The number of hospitalizations resulted in low precision around the effect estimate and a lack of statistical power, limiting the interpretation of subgroup analyses (Table S9).

Sensitivity analyses for medically-attended COVID-19 demonstrated consistent results when not applying data truncation (HR, 0.83; 95% CI, 0.75–0.93) and when utilizing a 1:1 PS matching approach (HR, 0.84; 95% CI, 0.75–0.94) (Fig. 4 ; Table S10). When a COVID-19 test was required prior to a diagnosis code for COVID-19, the HR was 0.78 (95% CI, 0.67–0.92) (Fig. 4; Table S10). By requiring a continuous closed claim enrollment file for 365 days before the index date and capturing events from closed claims only, the primary analysis was restricted both in the number of individuals included in the analytic cohort and in the number of events captured. Utilizing both open and closed claims during the follow-up period while enforcing the same approaches for cohort inclusion and follow up as the primary analysis, the number of medically-attended COVID-19 events increased from 1457 to 1734 with a resulting HR of 0.82 (95% CI, 0.74–0.90) (Fig. 4; Table S10). Utilizing open claims for both cohort entry and outcomes increased the cohort size from 124,457 to 388,965 immunocompromised individuals and increased the number of outcomes to 5,060 (HR, 0.74; 95% CI, 0.70–0.79) (Fig. 4; Table S10).

Fig. 4.

Sensitivity analyses for medically-attended COVID-19. Forest plot of hazard ratios of mRNA-1273 versus BNT-1626b in primary and sensitivity analyses of medically-attended COVID-19 data.

For the outcome of COVID-19 hospitalization, sensitivity analyses were consistent with the primary analysis (Fig. 5 ; Table S11). When incorporating open claims for outcomes and utilizing the same approaches for cohort inclusion and follow-up as the primary analysis, the number of COVID-19 hospitalizations increased from 215 to 282, with a corresponding HR of 0.72 (95% CI, 0.57–0.92) (Fig. 5; Table S11). When utilizing open claims for both cohort entry and outcomes, 874 COVID-19 hospitalizations were captured with a HR of 0.66 (95% CI, 0.58–0.76) (Fig. 5; Table S11).

Fig. 5.

Sensitivity analyses for COVID-19 hospitalization. Forest plot of hazard ratios of mRNA-1273 versus BNT-1626b in primary and sensitivity analyses of COVID-19 hospitalization data.

4. Discussion

Given the risk of severe COVID-19 outcomes in immunocompromised individuals, it is imperative to understand the real-world VE of COVID-19 vaccines; however, no studies have assessed the comparative VE of mRNA-based COVID-19 vaccines in this population. In the observational study reported herein, among a population of immunocompromised adults, 2 doses of mRNA-1273 were more effective in preventing breakthrough medically-attended COVID-19 infection compared to 2 doses of BNT162b2 (HR, 0.83; 95% CI, 0.75–0.93). The data also suggest that 2 doses of mRNA-1273 may be more effective compared to BNT162b2 in preventing breakthrough COVID-19 hospitalization (HR, 0.78; 95% CI, 0.59–1.03), although the small number of events resulted in low precision around the effect estimate. Our finding that the mRNA-1273 vaccine may be more effective in preventing medically-attended COVID-19 outcomes is consistent with real-world head-to-head studies in the general population and in adults with chronic comorbidities that were conducted during a time period when the delta and alpha variants dominated) [32], [33], [34], [35].

These head-to-head comparative effective results should be interpreted within the context of previously established real-world effectiveness of both the mRNA-1273 and BNT162b2 vaccines among immunocompromised individuals compared to unvaccinated or partially vaccinated individuals, with the estimated vaccine effectiveness ranging from 71% for preventing infection [36] to 77% for preventing COVID-19 hospitalization.[25] Furthermore, the increased effectiveness of mRNA-1273 is not evident until 100 days post-completion of a 2-dose regimen. Also at the time of the design of this study, while a third dose of mRNA-based vaccines gained authorization for IC individuals (recognized as a booster at the time), the sample size of the IC population who had received a third dose within claims records may not have been sufficiently large enough for to provide meaningful interpretation, thus limiting the current analysis to 2-dose regimens. Future work may consider the effectiveness of additional vaccine doses in this high-risk population.

Within immunocompromised subgroups, the comparative VE of a 2-dose regimen of mRNA-1273 compared with BNT162b2 against breakthrough medically-attended COVID-19 appeared to be greater among immunocompromised individuals with active cancer and among those with a primary immunodeficiency. Among individuals who had received blood/stem cell transplant, solid organ transplant, or those on active immunosuppressive therapy, the HRs were attenuated. Similar patterns were observed for COVID-19 hospitalization, although the small number of events and resulting lack of precision warrant caution in their interpretation.

Results of sensitivity analyses (incorporating open and closed claims, PS matching, foregoing application of the 3-month data truncation, and an alternative definition of medically-attended COVID-19 requiring evidence of a recent polymerase chain reaction or antigen test) were consistent with the primary analyses.

A strength of this head-to-head study is the utilization of a large claims database, which provided a sufficient sample size to detect differences between vaccine groups of less common outcomes (ie, breakthrough COVID-19 diagnoses) among specific subpopulations such as immunocompromised individuals. Furthermore, incorporation of open and closed medical and pharmacy claims in sensitivity analyses provides insight into the implications of open claims. This resulted in real-time capture of outcomes and an almost threefold increase in cohort size. For medically-attended breakthrough COVID-19 diagnoses, the gains in speed and precision paired with consistent effect estimates with the primary analysis (utilizing closed claims only) provides support for future incorporation of open claims for outcome ascertainment in comparative VE studies, especially when paired with a required enrollment file among eligible individuals.

The results of this study should be interpreted in light of several limitations. First, there remains the potential for residual confounding by unmeasured variables. Second, identifying the primary outcome of medically-attended COVID-19 identified via the ICD-10 U-code for COVID-19 diagnosis in the inpatient or outpatient setting has not been previously validated specifically within the HealthVerity database. However, a recent study in the US Veterans Affairs database estimated the positive predictive value (PPV) of the ICD-10 code U07.1 in all settings (inpatient, outpatient, and emergency department/urgent care) to be 84.2 % [37], which supports the robustness of this outcome and also cautiously noting the high PPV dependency associated with high disease prevalence within a pandemic setting. Furthermore, the analysis of an alternative definition of medically-attended COVID-19 requiring the presence of an antigen or polymerase chain reaction test produced an effect estimate slightly further from the null, thereby confirming the hypothesis that misclassification of the primary outcome would be non-differential between vaccine groups, with the resulting bias to be towards the null [38]. The difference in the HR between the base-case U-code definition and the alternative definition requiring testing (0.05) is in line with our bias analysis based on the PPV estimate (see supplementary text and Fig. S1). Additionally, in a real-world study of Ad26.COV2.S using the HealthVerity data, there was little difference in the VE when limiting the outcome to the COVID-19 diagnostic U-code in any setting or requiring a nucleic acid amplification lab confirmation of COVID-19 [28]. Regarding hospitalization for COVID-19, a similar algorithm as utilized in the current study has been previously leveraged in two published/ongoing cohort studies utilizing the HealthVerity claims database [28]. Recent validation studies of the ICD-9 U-code for COVID-19 in an inpatient setting also demonstrated a positive predictive value of at least 90 %, again noting that PPV is dependent on disease prevalence [39], [40].

A limitation of the HealthVerity claims data is the high amount of missingness of the place of service field (61.9% in both vaccinated groups across Private Source 17 and Private Source 20), which may also lead to misclassification of the outcomes. To mitigate this limitation, we identified suspected inpatient stays through a combination of claim types, billing codes, and place of service (where available) likely indicating an inpatient event. This study is also limited to individuals in the United States with continuous medical and pharmacy insurance coverage and does not include the uninsured, who may have additional co-morbidities or exposure factors placing them at higher risk of breakthrough COVID-19 infection and hospitalization. Also as demonstrated by the subgroup analyses, Medicare coverage is limited in the HealthVerity database and in other US commercial claims databases limiting inferences about comparative VE in older individuals aged ≥ 65 years. Finally, due to the absence of variant specific laboratory data in this study, the potential effect modification by the circulating COVID-19 variants cannot be assessed, also noting the truncated data cut-off date ofOctober 12, 2021, prior to the dominant circulation of the omicron variant. The majority of patients included in the analytic cohort received a second dose of vaccine between March and May 2021, when the alpha variant was predominant in the US, with 15–20 % receiving a second dose during June-September 2021, when the delta variant became prominent [41]. Despite this limitation, the outcome model included a state-based level of transmission, which incorporated calendar month of the start of follow-up for each patient, and the PS model included month and year of index date, both of which may, in part, control for the dominant circulating variant over the study period. Furthermore, subgroup analyses by calendar quarter provide insight into the impact of the circulating variants at that time. Further to this limitation, the lack of clinical data available in administrative data limits the ability to identify severely immunocompromised individuals, such as those with significantly low CD4 accounts among the HIV + population. Lastly, again given the lack of clinical data, the identification of immunocompromised patients in the claims database and the resulting study population may not represent all immunocompromised patients at the time of index.

Importantly, even with a large database such as HealthVerity, identifying a rare event such as breakthrough COVID-19 hospitalizations utilizing only closed claims may have limited the precision of effect estimates. Although use of fully adjudicated closed claims is typically the gold standard for database studies, open claims provide an ability to estimate the VE of vaccines and therapies with less lag (typically days to weeks for open claims versus 3–6 months for closed claims), which is instrumental in the setting of a global pandemic such as COVID-19 with rapid mass vaccination uptake. The consistency of the results from this study and the gain in precision suggest the ability to incorporate real-time open claims into comparative VE analyses. However, using open claims without a corresponding enrollment file may introduce bias, as baseline covariates could be under-captured, resulting in slightly different populations compared to a closed claims analysis. Therefore, our recommendation for future analyses is to use open claims in the follow-up phase for outcome ascertainment only.

In conclusion, results from this observational comparative VE database study provide evidence that among immunocompromised adults, 2 doses of mRNA-1273 is more effective in preventing breakthrough medically-attended COVID-19 infection compared to 2 doses of BNT162b2. Results also suggest that mRNA-1273 may have greater VE in preventing serious outcomes in comparison to BNT162b2 in this high-risk patient population.

Ethics committee approval statement

This study was exempt from approval by the New England Institutional Review Board.

Funding

This work was supported by Moderna, Inc.

Role of funding source

Authors employed by Moderna, Inc. were involved in the study design, data collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: LSC, DBE, DM, JM, CV, XC, and NV are employees of Moderna, Inc. and hold stock/stock options in the company. CAT was an employee of and a shareholder in Moderna, Inc. at the time of these analyses; CAT is currently an employee of AstraZeneca. KEM, BK, DAP, AG, and NMG are employees of Aetion, Inc., which has been contracted by Moderna, Inc., for the conduct of the present study. KEM is a stock option holder of Aetion, Inc. NMG is a stock option holder of Aetion, Inc and owns stock in Pfizer, Inc.

.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Data availability

Data will be made available on request.