Abstract
Background
We conducted a prospective cohort study at Kaiser Permanente Southern California to study the vaccine effectiveness (VE) of mRNA-1273 over time and during the emergence of the Delta variant.
Methods
The cohort for this planned interim analysis consisted of individuals aged ≥18 years receiving 2 doses of mRNA-1273 through June 2021, matched 1:1 to randomly selected unvaccinated individuals by age, sex, and race/ethnicity, with follow-up through September 2021. Outcomes were SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death. Cox proportional hazards models were used to estimate adjusted hazard ratios (aHR) with 95% confidence intervals (CIs) comparing outcomes in the vaccinated and unvaccinated groups. Adjusted VE (%) was calculated as (1-aHR)x100. HRs and VEs were also estimated for SARS-CoV-2 infection by age, sex, race/ethnicity, and during the Delta period (June-September 2021). VE against SARS-CoV-2 infection and COVID-19 hospitalization was estimated at 0-<2, 2-<4, 4-<6, and 6-<8 months post-vaccination.
Results
927,004 recipients of 2 doses of mRNA-1273 were matched to 927,004 unvaccinated individuals. VE (95% CI) was 82.8% (82.2–83.3%) against SARS-CoV-2 infection, 96.1% (95.5–96.6%) against COVID-19 hospitalization, and 97.2% (94.8–98.4%) against COVID-19 hospital death. VE against SARS-CoV-2 infection was similar by age, sex, and race/ethnicity, and was 86.5% (84.8–88.0%) during the Delta period. VE against SARS-CoV-2 infection decreased from 88.0% at 0-<2 months to 75.5% at 6-<8 months.
Conclusions
These interim results provide continued evidence for protection of 2 doses of mRNA-1273 against SARS-CoV-2 infection over 8 months post-vaccination and during the Delta period, and against COVID-19 hospitalization and hospital death.
Introduction
Since coronavirus disease 2019 (COVID-19) vaccine rollout, multiple observational studies have demonstrated that mRNA-1273 (Moderna Inc, Cambridge, USA) is highly effective against symptomatic COVID-19, with reports of vaccine effectiveness (VE) as high as 100% [1–4]. mRNA-1273 has also been shown to have high VE against COVID-19 hospitalization and death [1, 3].
However, the emergence of several SARS-CoV-2 variants led to concerns of potentially lower VE of COVID-19 vaccines against variants [5, 6]. The higher transmissibility of the Delta variant led to a surge in infections, hospitalizations, and deaths in the United States (US) since becoming the dominant circulating strain in July 2021 [7, 8]. Even though the majority of cases have been in unvaccinated individuals, breakthrough cases among vaccinated individuals have also been reported [9–12]. We previously evaluated the VE of mRNA-1273 against SARS-CoV-2 variants, and reported an overall VE of 86.7% against Delta infection through July 2021 [13]. Another test-negative observational study in Ontario, Canada, from December 2020 to August 2021, reported a 95% VE against symptomatic infection with Delta after two doses of mRNA-1273 with similar VE against hospitalization and death [14]. Furthermore, several studies conducted in the US post Delta emergence reported high VE of mRNA-1273 against COVID-19 hospitalization ranging from 91.6% to 95.0% [15–17].
As the pandemic continues, the durability of protection of COVID-19 vaccines, for which data are limited [18], and strategies for booster implementation have become critical public health questions. We previously reported that waning of VE was most pronounced for the Delta variant, with VE against Delta infection declining from 94.1% at 14–60 days after vaccination to 80.0% at 151–180 days after vaccination [13]. A test-negative study conducted in the United Kingdom compared VE of mRNA-1273 against Delta symptomatic disease and hospitalization over 14 weeks, showing a decline in VE from 95.6% at 1 week after vaccination to 85.6% at 14 weeks after vaccination [19].
Few published studies are available on the durability of protection of mRNA-1273 several months after vaccination during the Delta period among the general US population. In this study, we evaluated the durability (with up to 8 months of follow-up) and effectiveness of 2 doses of mRNA-1273 in preventing SARS-CoV-2 infection, stratified by age, sex, and race/ethnicity, and during the Delta period in the US. We also evaluated VE and durability against COVID-19 hospitalization.
Methods
Study setting
Kaiser Permanente Southern California (KPSC) is an integrated health care system providing care to a diverse 4.6 million members [20]. KPSC’s electronic health record (EHR) is comprehensive, capturing details across several areas such as inpatient, outpatient, emergency, and virtual care; claims also capture any care received outside of the KPSC system. The study was reviewed and approved by the KPSC Institutional Review Board (IRB #12758). A waiver of informed consent was obtained as this is an observational study of the authorized and recommended Moderna COVID-19 vaccine administered during routine clinical care. To facilitate the conduct of this study, a waiver was obtained for written HIPAA authorization for research involving use of the EHR.
Study objectives
This is the second planned interim analysis (IA) of a 5-year cohort study at KPSC to evaluate the VE of mRNA-1273 in preventing SARS-CoV-2 infection and severe COVID-19 disease. Detailed methods and results of the first IA with follow-up through June 30, 2021 were reported elsewhere [1]. For this analysis, the primary objectives were to evaluate the VE of 2 doses of mRNA-1273 in preventing SARS-CoV-2 infection and severe COVID-19 disease. Secondary objectives were to evaluate the VE of 2 doses of mRNA-1273 in preventing SARS-CoV-2 infection by age, sex, and race/ethnicity groups. We also evaluated the waning of VE over time to assess the durability of mRNA-1273 against SARS-CoV-2 infection and COVID-19 hospitalization. To specifically assess VE against Delta in the absence of waning, we also evaluated VE against Delta infection in individuals newly vaccinated in June 2021 and followed through September 30, 2021. The study protocol is presented in the S1 Appendix.
Study population
For the study cohort, eligible individuals were ≥18 years old, and were KPSC members for at least 1 year prior to the index date (allowing a 31-day membership gap) and for ≥14 days after the index date. Individuals who received 2 doses of mRNA-1273 ≥24 days apart (4-day grace period allowed prior to the recommended 28-day interval) from December 18, 2020 through June 30, 2021 made up the vaccinated group. The index date was defined as the date of receipt of the second dose of mRNA-1273 for vaccinated individuals; their matched unvaccinated counterparts were assigned the same index date. Individuals who received a COVID-19 vaccine other than mRNA-1273 on or before the index date, or who had no healthcare utilization or no vaccination from the 2 years prior to the index date through the index date were excluded. Individuals who died, had a COVID-19 outcome, or received any COVID-19 vaccine <14 days after the index date were also excluded.
The unvaccinated group was composed of those who did not receive any COVID-19 vaccine as of the index date. Unvaccinated comparators were randomly selected and matched 1:1 to the vaccinated individuals by age group (18–44 years, 45–64 years, 65–74 years, and ≥75 years), sex, and race/ethnicity (Non-Hispanic White, Non-Hispanic Black, Hispanic, Non-Hispanic Asian, and Other/Unknown). Index dates were also balanced through matching, since matched vaccinated and unvaccinated individuals shared the same index date.
Exposure and outcomes
Information on mRNA-1273 exposure was ascertained from KPSC’s EHR. The EHR included vaccines administered within KPSC as well as COVID-19 vaccines administered outside of the KPSC health system. These were imported daily into the KPSC EHR from the California Immunization Registry (CAIR), CalVax (Cal Poly Pomona mass vaccination site), Care Everywhere (Epic EHR feature which allows health care systems to exchange medical information), claims (e.g., pharmacies), and member self-report with documentation. All providers of COVID-19 vaccines were required by law to provide COVID-19 vaccine administration data daily to CAIR [21].
The first primary outcome was SARS-CoV-2 infection defined as a positive molecular test or a COVID-19 diagnosis code for both symptomatic and asymptomatic infections. Individuals with COVID-19 who had a COVID-19 diagnosis code or a SARS-CoV-2 positive molecular test in the 90 days prior were not considered incident cases. The second primary outcome was severe COVID-19 disease, which included COVID-19 hospitalization (hospitalization with a SARS-CoV-2 positive test or a COVID-19 diagnosis code, or a hospitalization ≤7 days after a SARS-CoV-2 positive test) and COVID-19 hospital death (death occurring during COVID-19 hospitalization). COVID-19 hospitalization was confirmed 1) by at least one documented SpO2 <90% during hospital stay, or 2) if SpO2 ≥90%, by manual chart review done by a physician investigator [BKA] and trained chart abstractors to verify the presence of severe COVID-19 symptoms.
Individuals were followed for COVID-19 outcomes ≥14 days after the index date through September 30, 2021 (end of follow-up) or until occurrence of a censoring event (termination of KPSC membership allowing for a 31-day gap, death, or receipt of a COVID-19 vaccine), resulting in up to 8 months of follow-up post-vaccination for this analysis. Unvaccinated individuals stopped contributing unvaccinated person-time if they received a first dose of mRNA-1273 during follow-up and started contributing vaccinated person-time when they received an eligible second dose of mRNA-1273.
Starting from June 1, 2021, Delta (B.1.617.2, AY.*) was the predominant strain among positive SARS-CoV-2 specimens successfully sequenced at KPSC [1]. To evaluate VE of mRNA-1273 specifically against SARS-CoV-2 infection during the Delta period, a subset of individuals was included in an additional Delta period analysis if they had an index date during June 1–30, 2021 and were followed up to September 30, 2021.
Other variables
Baseline characteristics were extracted from the EHR. Variables assessed at index date included age, sex, race/ethnicity, socioeconomic status (Medicaid coverage, neighborhood median household income), medical center area, pregnancy status, and KPSC physician/employee status. Variables assessed in the two years prior to index date included smoking and body mass index (BMI). Variables assessed in the year prior to index date included Charlson comorbidity score, autoimmune conditions, health care utilization (virtual, outpatient, emergency department [ED], inpatient encounters), preventive care (other vaccinations, screenings, well-visits), chronic diseases (kidney disease, heart disease, lung disease, liver disease, diabetes), and frailty index [22]. Other variables included history of SARS-CoV-2 infection and molecular test performed from March 1, 2020 to index date (irrespective of result), and immunocompromised status.
Statistical analyses
Baseline characteristics of the vaccinated and unvaccinated groups were described. Continuous variables were compared using two-sample t-test or Wilcoxon rank-sum test, as appropriate, and categorical variables were compared using χ2 test or Fisher’s exact test, as appropriate. Potential confounders were identified based on the literature. To assess the balance of covariates across groups, absolute standardized differences (ASD) were used. Covariates with ASD>0.1, as well as age, sex, race/ethnicity, month of index date were included in the multivariable models. The missing indicator method was used for covariates with missing data [23].
The number of incident events divided by person-years was used to calculate the incidence rates (IR) per 1,000 person-years of SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death for both vaccinated and unvaccinated groups. Kaplan-Meier curves were used to estimate the cumulative incidence of SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death for both groups; the differences between the curves were tested by log-rank test.
Cox proportional hazards regression models were used to estimate unadjusted and adjusted hazard ratios (HR) with 95% confidence intervals (CIs) comparing SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death in the vaccinated and unvaccinated groups overall. VE (%) was calculated as (1-HR) x 100. HRs and VEs were also estimated comparing SARS-CoV-2 infection in vaccinated and unvaccinated individuals by age, sex, race/ethnicity, and during the Delta period.
To evaluate durability of protection, VE against SARS-CoV-2 infection and COVID-19 hospitalization was estimated every two months post-vaccination (0-<2, 2-<4, 4-<6, and 6-<8 months) using time-varying Cox regression models. VEs by months post-vaccination against SARS-CoV-2 infection and COVID-19 hospitalization were also estimated and stratified by age groups (18–64 years old and ≥65 years old).
All analyses were conducted using SAS software version 9.4, Cary, USA.
Results
Baseline characteristics
The study cohort consisted of 927,004 recipients of 2 doses of mRNA-1273 vaccine (hereafter, ‘vaccinated’) and 927,004 matched unvaccinated individuals (Fig 1). Overall, the median age was 52 years (interquartile range [IQR] 36–65) with 74.3% aged 18–64 years; there were more females than males (54.5%); and 38.6% were Hispanic, 33.6% were non-Hispanic White, 12.8% were Asian, and 7.5% were Black (Table 1). The vaccinated and unvaccinated groups had similar characteristics (ASD≤0.1) of BMI, smoking, Charlson comorbidities index, frailty index, chronic diseases, immunocompromised status, autoimmune conditions, pregnancy status, history of SARS-CoV-2 infection and molecular test, ED visits, hospitalizations, Medicaid enrollment, and median household income. Compared to the unvaccinated individuals, those vaccinated had more outpatient and virtual visits, and more preventive care visits in the year prior to the index date (ASD>0.1). A higher proportion of vaccinated individuals were KPSC physicians/employees (ASD>0.1). There were some differences in the distribution of vaccinated and unvaccinated groups across the medical centers (ASD>0.1). The greatest proportion of index dates occurred in April 2021 (34.5%) followed by May 2021 (21.2%), coinciding with California’s vaccination phase when mRNA-1273 became available to all individuals ≥18 years old [24]. Vaccinated individuals received their second dose after a median 28 days (IQR 28–29) post-first dose, and 0.1% received a concomitant vaccine with mRNA-1273.
Fig 1. Flow chart for 2-dose mRNA-1273 analysis cohort.
Table 1. Baseline characteristics of 2-dose mRNA-1273 vaccinated and unvaccinated cohort.
| Vaccinated | Unvaccinated | Absolute | |
|---|---|---|---|
| N = 927004 | N = 927004 | Standardized Difference | |
| N (%) | N (%) | ||
| Age at index date, years, median (IQR) | 52 (37, 65) | 51 (36, 65) | 0.0452 |
| Age at index date, years | N/A | ||
| 18–44 | 339646 (36.6) | 339646 (36.6) | |
| 45–64 | 349556 (37.7) | 349556 (37.7) | |
| 65–74 | 154865 (16.7) | 154865 (16.7) | |
| ≥75 | 82937 (8.9) | 82937 (8.9) | |
| Sex | N/A | ||
| Female | 505654 (54.5) | 505654 (54.5) | |
| Male | 421350 (45.5) | 421350 (45.5) | |
| Race/Ethnicity | N/A | ||
| Non-Hispanic White | 311799 (33.6) | 311799 (33.6) | |
| Non-Hispanic Black | 69159 (7.5) | 69159 (7.5) | |
| Hispanic | 357415 (38.6) | 357415 (38.6) | |
| Non-Hispanic Asian | 118333 (12.8) | 118333 (12.8) | |
| Other/Unknown | 70298 (7.6) | 70298 (7.6) | |
| Body Mass Indexb | 0.0545 | ||
| <18.5 | 9848 (1.1) | 13021 (1.4) | |
| 18.5 - <25 | 205813 (22.2) | 215470 (23.2) | |
| 25 - <30 | 269826 (29.1) | 272903 (29.4) | |
| 30 - <35 | 177499 (19.1) | 178076 (19.2) | |
| 35 - <40 | 83982 (9.1) | 81874 (8.8) | |
| 40 - <45 | 36994 (4.0) | 32855 (3.5) | |
| ≥45 | 23525 (2.5) | 20579 (2.2) | |
| Unknown | 119517 (12.9) | 112226 (12.1) | |
| Smokingb | 0.0361 | ||
| No | 667670 (72.0) | 663128 (71.5) | |
| Yes | 162747 (17.6) | 173785 (18.7) | |
| Unknown | 96587 (10.4) | 90091 (9.7) | |
| Charlson comorbidity scorea | 0.0827 | ||
| 0 | 646513 (69.7) | 681024 (73.5) | |
| 1 | 130475 (14.1) | 115147 (12.4) | |
| ≥2 | 150016 (16.2) | 130833 (14.1) | |
| Frailty indexa | 0.0456 | ||
| Quartile 1 | 191548 (20.7) | 187533 (20.2) | |
| Quartile 2 | 266325 (28.7) | 281684 (30.4) | |
| Quartile 3 | 230017 (24.8) | 233376 (25.2) | |
| Quartile 4 | 239114 (25.8) | 224411 (24.2) | |
| Chronic diseasesa | |||
| Kidney disease | 51638 (5.6) | 45129 (4.9) | 0.0316 |
| Heart disease | 27855 (3.0) | 27594 (3.0) | 0.0017 |
| Lung disease | 71550 (7.7) | 65003 (7.0) | 0.0270 |
| Liver disease | 26146 (2.8) | 23954 (2.6) | 0.0146 |
| Diabetes | 139390 (15.0) | 116794 (12.6) | 0.0707 |
| Immunocompromised | 29784 (3.2) | 24102 (2.6) | 0.0365 |
| HIV/AIDS | 3930 (0.4) | 2074 (0.2) | |
| Leukemia, lymphoma, congenital and other immunodeficiencies, asplenia/hyposplenia | 10823 (1.2) | 9318 (1.0) | |
| Organ transplant | 2750 (0.3) | 1910 (0.2) | |
| Immunosuppressant medications | 17467 (1.9) | 14539 (1.6) | |
| Autoimmune conditionsa | 25882 (2.8) | 22327 (2.4) | 0.0241 |
| Rheumatoid arthritis | 11764 (1.3) | 10185 (1.1) | |
| Inflammatory bowel disease | 4384 (0.5) | 3694 (0.4) | |
| Psoriasis and psoriatic arthritis | 8755 (0.9) | 7500 (0.8) | |
| Multiple sclerosis | 1566 (0.2) | 1429 (0.2) | |
| Systemic lupus erythematosus | 2124 (0.2) | 1772 (0.2) | |
| Pregnant at index date | 5063 (0.5) | 10551 (1.1) | 0.0648 |
| 1st trimester | 1609 (0.2) | 2625 (0.3) | |
| 2nd trimester | 1863 (0.2) | 3988 (0.4) | |
| 3rd trimester | 1591 (0.2) | 3938 (0.4) | |
| History of SARS-CoV-2 infectionc | 92461 (10.0) | 113396 (12.2) | 0.0719 |
| History of SARS-CoV-2 molecular testc | 364575 (39.3) | 337279 (36.4) | 0.0607 |
| Number of outpatient and virtual visitsa,d | 0.2504 | ||
| 0 | 68573 (7.4) | 128820 (13.9) | |
| 1–4 | 327996 (35.4) | 357042 (38.5) | |
| 5–10 | 263631 (28.4) | 231974 (25.0) | |
| ≥11 | 266804 (28.8) | 209168 (22.6) | |
| Number of emergency department visitsa,d | 0.0564 | ||
| 0 | 806834 (87.0) | 789726 (85.2) | |
| 1 | 88007 (9.5) | 97545 (10.5) | |
| ≥2 | 32163 (3.5) | 39733 (4.3) | |
| Number of hospitalizationsa,d | 0.0449 | ||
| 0 | 887170 (95.7) | 878587 (94.8) | |
| 1 | 32205 (3.5) | 38153 (4.1) | |
| ≥2 | 7629 (0.8) | 10264 (1.1) | |
| Preventive carea | 691389 (74.6) | 568154 (61.3) | 0.2878 |
| Medicaid | 59896 (6.5) | 77950 (8.4) | 0.0743 |
| Neighborhood median household income | 0.0750 | ||
| < $40,000 | 42362 (4.6) | 47664 (5.1) | |
| $40,000-$59,999 | 182542 (19.7) | 194304 (21.0) | |
| $60,000-$79,999 | 225417 (24.3) | 235308 (25.4) | |
| $80,000+ | 476050 (51.4) | 447572 (48.3) | |
| Unknown | 633 (0.1) | 2156 (0.2) | |
| KPSC physician/employee | 31111 (3.4) | 15132 (1.6) | 0.1107 |
| Concomitant vaccinatione | 559 (0.1) | N/A | N/A |
| Time between first and second doses, days, median (IQR) | 28 (28–29) | N/A | N/A |
| Index date | N/A | ||
| January 2021 | 20561 (2.2) | 20561 (2.2) | |
| February 2021 | 156278 (16.9) | 156278 (16.9) | |
| March 2021 | 174115 (18.8) | 174115 (18.8) | |
| April 2021 | 320109 (34.5) | 320109 (34.5) | |
| May 2021 | 196973 (21.2) | 196973 (21.2) | |
| June 2021 | 58968 (6.4) | 58968 (6.4) |
aDefined in the one year prior to index date.
bDefined in the two years prior to index date.
cDefined based on all available medical records from March 1, 2020 to index date.
dIndicator of overall health care utilization / care-seeking.
eAmong subjects with concomitant vaccines: influenza vaccine (24.0%), Tdap (49.4%), shingles vaccine (4.5%), pneumococcal vaccines (PCV13/PPSV23) (9.3%), HPV vaccine (8.1%) and other vaccine (9.3%); 32.7% concomitant with 1st dose and 68.7% concomitant with 2nd dose.
Medical center area not shown. The absolute standardized difference of the distribution of the vaccinated and unvaccinated individuals across the 19 medical center areas is 0.2095.
N, sample size; IQR, interquartile range; N/A, not applicable (matching variable or pertains only to vaccinated individuals).
VE against SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death
There were 7,685 cases of SARS-CoV-2 infection among the vaccinated individuals and 16,809 cases among the unvaccinated individuals, with IRs per 1,000 person-years of 19.09 (95% CI: 18.67–19.52) and 94.02 (95% CI: 92.61–95.45), respectively (Table 2). IRs per 1,000 person-years for COVID-19 hospitalization and COVID-19 hospital death were 0.60 (95% CI: 0.53–0.69) and 0.03 (95% CI: 0.02–0.06), respectively, among vaccinated individuals and 13.07 (95% CI: 12.55–13.61) and 0.90 (95% CI: 0.77–1.05), respectively, among the unvaccinated individuals. The cumulative incidences of SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death were all significantly higher in unvaccinated individuals than vaccinated individuals (log-rank test p<0.001) (Fig 2A–2C). The adjusted VE (Table 2) was 82.8% (95% CI: 82.2–83.3%) against SARS-CoV-2 infection; 96.1% (95% CI: 95.5–96.6%) against COVID-19 hospitalization; and 97.2% (95% CI: 94.8–98.4%) against COVID-19 hospital death.
Table 2. Incidence rates, hazard ratios, and vaccine effectiveness of 2 doses of mRNA-1273 in preventing SARS-CoV-2 infection, and COVID-19 hospitalization and hospital death.
| Vaccinated (N = 927004) | Unvaccinated (N = 927004) | Hazard Ratio (95% CI) | VE % (95% CI) | |||||
|---|---|---|---|---|---|---|---|---|
| Outcomes | Number of cases | Incidence per 1000 person-years (95% CI) | Number of cases | Incidence per 1000 person-years (95% CI) | Unadjusted | Adjusteda | Unadjusted | Adjusteda |
| SARS-CoV-2 infection | 7685 | 19.09 (18.67–19.52) | 16809 | 94.02 (92.61–95.45) | 0.17 (0.16–0.17) | 0.17 (0.17–0.18) | 83.4 (82.9–83.9) | 82.8 (82.2–83.3) |
| COVID-19 hospitalization | 244 | 0.60 (0.53–0.69) | 2361 | 13.07 (12.55–13.61) | 0.04 (0.03–0.04) | 0.04 (0.03–0.05) | 96.4 (95.9–96.8) | 96.1 (95.5–96.6) |
| COVID-19 hospital death | 13 | 0.03 (0.02–0.06) | 163 | 0.90 (0.77–1.05) | 0.03 (0.02–0.05) | 0.03 (0.02–0.05) | 96.9 (94.5–98.2) | 97.2 (94.8–98.4) |
aAdjusted for covariates age groups (4 categories), sex, race/ethnicity, index date (in months), number of outpatient and virtual visits, preventive care, KPSC physician/employee status, medical center area.
N, sample size; CI, confidence interval; VE, vaccine effectiveness.
Fig 2. Cumulative incidence estimates by vaccination status in 2-dose mRNA-1273 cohort.
a. SARS-CoV-2 infection. b. COVID-19 hospitalization. c. COVID-19 hospital death.
VE against SARS-CoV-2 infection by subgroups
Among vaccinated and unvaccinated groups, the IR of SARS-CoV-2 infection was higher in those <65 years as compared to those ≥65 years (Table 3). IRs among vaccinated and unvaccinated females were higher than their male counterparts. IRs were higher among Hispanic individuals in the vaccinated group (23.03 [95% CI: 22.28–23.81]) and among non-Hispanic Black individuals in the unvaccinated group (109.90 [95% CI: 104.47–115.61]) when compared to the other racial/ethnic groups. Despite differences in IRs of SARS-CoV-2 infection across age, sex, and race/ethnicity groups, the adjusted VE against SARS-CoV-2 infection was similar by age, sex, and race/ethnicity, ranging from 77.9% to 87.0%. Among the subgroup of individuals who completed their 2-dose series in June 2021 and their unvaccinated matched counterparts, the IRs for SARS-CoV-2 infection during the Delta period were 20.38 (95% CI: 18.27–22.75) and 146.73 (95% CI: 140.40–153.35), respectively (Table 3). Adjusted VE against SARS-CoV-2 infection comparing individuals vaccinated during the Delta period and their unvaccinated counterparts was 86.5% (95% CI: 84.8–88.0%).
Table 3. Incidence rate, hazard ratio, and vaccine effectiveness of 2 doses of mRNA-1273 in preventing SARS-CoV-2 infection by age, sex, race/ethnicity, and Delta period subgroups.
| Vaccinated (N = 927004) | Unvaccinated (N = 927004) | Hazard Ratio (95% CI) | VE % (95% CI) | |||||
|---|---|---|---|---|---|---|---|---|
| Number of cases | Incidence per 1000 person-years (95% CI) | Number of cases | Incidence per 1000 person-years (95% CI) | Unadjusted | Adjusteda | Unadjusted | Adjusteda | |
| Age at index date, years | ||||||||
| 18–44 | 3196 | 23.05 (22.26–23.86) | 7145 | 109.90 (107.38–112.48) | 0.16 (0.15–0.17) | 0.17 (0.17–0.18) | 84.0 (83.3–84.7) | 82.6 (81.7–83.5) |
| 45–64 | 2811 | 19.15 (18.46–19.88) | 6912 | 104.79 (102.35–107.29) | 0.14 (0.13–0.15) | 0.15 (0.15–0.16) | 85.9 (85.2–86.5) | 84.6 (83.8–85.4) |
| 65–74 | 1098 | 14.64 (13.80–15.53) | 1877 | 62.57 (59.81–65.47) | 0.19 (0.18–0.21) | 0.18 (0.17–0.20) | 80.8 (79.3–82.2) | 81.6 (79.9–83.1) |
| ≥75 | 580 | 13.78 (12.71–14.95) | 875 | 49.13 (45.98–52.49) | 0.24 (0.22–0.27) | 0.22 (0.20–0.25) | 75.9 (73.2–78.3) | 77.9 (75.1–80.3) |
| Sex | ||||||||
| Female | 4459 | 19.97 (19.39–20.56) | 9116 | 96.18 (94.23–98.18) | 0.17 (0.16–0.17) | 0.18 (0.17–0.18) | 83.2 (82.6–83.8) | 82.4 (81.7–83.1) |
| Male | 3226 | 18.00 (17.39–18.64) | 7693 | 91.58 (89.56–93.65) | 0.16 (0.16–0.17) | 0.17 (0.16–0.18) | 83.7 (83.0–84.4) | 83.1 (82.3–83.9) |
| Race/Ethnicity | ||||||||
| Non-Hispanic White | 2601 | 18.88 (18.16–19.62) | 6061 | 94.48 (92.13–96.89) | 0.17 (0.16–0.18) | 0.18 (0.17–0.19) | 83.2 (82.4–84.0) | 82.1 (81.1–83.0) |
| Non-Hispanic Black | 523 | 17.41 (15.98–18.97) | 1497 | 109.90 (104.47–115.61) | 0.14 (0.12–0.15) | 0.13 (0.12–0.15) | 86.2 (84.7–87.5) | 87.0 (85.5–88.4) |
| Hispanic | 3488 | 23.03 (22.28–23.81) | 7177 | 103.15 (100.79–105.57) | 0.18 (0.17–0.19) | 0.18 (0.17–0.19) | 82.1 (81.3–82.9) | 81.8(80.9–82.6) |
| Non-Hispanic Asian | 612 | 11.37 (10.50–12.31) | 940 | 52.47 (49.22–55.93) | 0.16 (0.14–0.18) | 0.17 (0.15–0.19) | 83.9 (82.1–85.6) | 83.4 (81.2–85.4) |
| Delta periodb | 319 | 20.38 (18.27–22.75) | 1973 | 146.73 (140.40–153.35) | 0.14 (0.12–0.15) | 0.13 (0.12–0.15) | 86.3 (84.6–87.9) | 86.5 (84.8–88.0) |
aAdjusted for covariates age groups (in 4 categories, and not adjusted in the age subcohort analysis), sex (not adjusted in the sex subcohort analysis), race/ethnicity (not adjusted in the race/ethnicity subcohort analysis), index date (in months), number of outpatient and virtual visits, preventive care, KPSC physician/employee status, medical center area.
bIndividuals with index dates in June 2021 were followed up to September 30, 2021.
N, sample size; CI, confidence interval; VE, vaccine effectiveness.
Durability of protection
Adjusted VE against SARS-CoV-2 infection and COVID-19 hospitalization by months after vaccination both overall and by age groups are presented in S1 Table. Adjusted VE against SARS-CoV-2 infection decreased from 88.0% at 0-<2 months to 75.5% at 6-<8 months (Fig 3). Adjusted VE against COVID-19 hospitalization remained stable, from 95.9% in 0-<2 months to 94.5% in 6-<8 months. The durability of protection VE estimates by age groups for both SARS-CoV-2 infection and COVID-19 hospitalization were similar to the overall results.
Fig 3. Vaccine effectiveness of 2 doses of mRNA-1273 in preventing SARS-CoV-2 infection and COVID-19 hospitalization by months after vaccination.
Discussion
This interim analysis was conducted among a large cohort to assess the effectiveness and durability of 2 doses of mRNA-1273 under real-world conditions. The study results confirmed the high effectiveness of 2 doses of mRNA-1273, with VE against SARS-CoV-2 infection of 82.8%, VE against COVID-19 hospitalization of 96.1%, and VE against COVID-19 hospital death of 97.2%. Furthermore, VE against COVID-19 hospitalization remained robust and stable over 8 months of follow-up. Moderate waning was observed for VE against SARS-CoV-2 infection, with a VE of 75.5% at 6-<8 months post-vaccination. The individuals included in the cohort were vaccinated through June 2021 and then followed through September 2021, overlapping with the rapid spread of the Delta variant in the US. Additionally, for individuals who completed the two-dose series in June 2021, VE against SARS-CoV-2 infection during the Delta period remained high at 86.5% through the end of September 2021.
These results add to a body of work evaluating the VE of mRNA-1273 in the general population prior to the Omicron surge that occurred in December 2021. A case-control study across 21 hospitals in 18 US states assessing the VE of mRNA-1273 in preventing COVID-19 hospitalizations during March 11, 2021 to August 15, 2021 reported a VE of 93% (95% CI: 91%-95%), similar to the VE found in this study [17]. The findings of our study are also consistent with and extend those of our prior interim analysis [1] with follow-up through June 2021. VE against COVID-19 hospitalization and hospital death was higher than the VE against SARS-CoV-2 infection, corroborating previous evidence that mRNA vaccines offer greater protection against more severe disease [1, 25, 26].
We previously found mRNA-1273 to have a high VE against SARS-CoV-2 infection before the Delta variant spread in the US [1]. The Delta variant started emerging in the US in late May 2021 and quickly became the dominant variant during June through August 2021 [27]. A study in Qatar evaluated VE of mRNA-1273 against the Delta variant [28], and found a VE against SARS-CoV-2 infection of 73.1% (95% CI: 67.5%-77.8%). In the current study, VE against SARS-CoV-2 infection was evaluated for the Delta period in a subset of individuals newly vaccinated during June 2021 and followed up through September 2021; VE against Delta after three months of follow-up was 86.5%, similar to the VE observed in our prior interim analysis (87.4%) [1]. This finding is less subject to waning and suggests that mRNA-1273 generates protection against the Delta variant.
The current study also examined the durability of the 2-dose mRNA-1273 vaccine. While VE against COVID-19 hospitalization was stable during the follow-up, VE against SARS-CoV-2 infection steadily decreased from 88.0% to 75.5% across the 8 months of follow-up. The same VE waning effects were apparent across age categories (18–64 years and ≥65 years). These findings are consistent with the findings of our previous studies that used a test-negative design [13, 29], the most recent of which found VE against Delta infection decreased from 80.2% to 61.3% within 12 months of follow-up. Due to this steady decrease in VE, our findings support the current mRNA-1273 booster recommendations of receiving a booster shot at least 5 months after completing the primary series [30].
One strength of the current study is the matched cohort design which allows for generalizability to the general population eligible for mRNA-1273. Another strength is KPSC’s diverse member population. The study also utilized KPSC’s EHR and was able to gather comprehensive data on millions of members; the data included COVID-19 vaccine information, demographics, medical history, and health care utilization. However, residual confounding from unmeasured factors might have still been present. Some health seeking behaviors such as adherence to masking guidelines and occupation are not captured by the EHR; these may contribute to differences in COVID-19 risk and testing behaviors between vaccinated and unvaccinated individuals. Misclassification of SARS-CoV-2 infection from false positive or false negative test results or from inaccurate diagnosis codes from outside claims may have been another limitation. This non-differential misclassification could have underestimated the VE. In addition, individuals vaccinated during the Delta period were younger as vaccines had recently become available to the general population and may have had fewer COVID-19 risk factors, which could have led to overestimation of VE during the Delta period. Lastly, since follow-up for this study ended in September 2021, VE during the Omicron period was not assessed.
In conclusion, this second interim analysis of an ongoing cohort study found that VE of 2 doses of mRNA-1273 against SARS-CoV-2 infection declined moderately over the course of 8 months, but VE against COVID-19 hospitalization remained robust and stable over the same period. In addition, VE against SARS-CoV-2 infection in newly vaccinated individuals during the Delta period remained high. Continued long-term follow-up is needed to fully evaluate the real-world effectiveness of mRNA-1273 overall and in different subgroups of the population over time.
Supporting information
(PDF)
(PDF)
Acknowledgments
The authors would like to acknowledge the following Kaiser Permanente Southern California staff: Radha Bathala, Maria Navarro, Elsa Olvera, Brittany Brown, Jeannie Song, and Stephanie Reese Rillon for their contributions in manual chart review; Harpreet Takhar, Michael Aragones, Soon Kyu Choi, Jennifer Charter, Julie Stern, Joy Gelfond, and Lee Childs for their coordination in processing SARS-CoV-2 specimens; and Raul Calderon, Kourtney Kottmann, Ana Acevedo, Elmer Ayala, and Jonathan Arguello for their technical and laboratory support in processing SARS-CoV-2 specimens. The authors would like to acknowledge Helix OpCo, LLC, for their whole genome sequencing of SARS-CoV-2 specimens. The authors would also like to acknowledge the contributions by Moderna staff: Yamuna Paila, PhD, and Julie Vanas. Medical writing and editorial assistance were provided by Srividya Ramachandran, PhD, and Jared Mackenzie, PhD, of MEDiSTRAVA in accordance with Good Publication Practice (GPP3) guidelines, under the direction of the authors. The authors thank the patients of Kaiser Permanente for their partnership with us to improve their health. Their information, collected through our electronic health record systems, leads to findings that help us improve care for our members and can be shared with the larger community.
Data Availability
Individual-level data reported in this study involving human research participants are not publicly shared due to potentially identifying or sensitive patient information. Upon request, and subject to review, KPSC may provide the deidentified aggregate-level data that support the findings of this study. Anonymized data (deidentified data including participant data as applicable) that support the findings of this study may be made available from the investigative team in the following conditions: (1) agreement to collaborate with the study team on all publications, (2) provision of external funding for administrative and investigator time necessary for this collaboration, (3) demonstration that the external investigative team is qualified and has documented evidence of training for human subjects protections, and (4) agreement to abide by the terms outlined in data use agreements between institutions. Interested researchers should contact Ana Florea, PhD, MPH, the corresponding author, at ana.florea@kp.org or Kristi Reynolds, PhD, MPH, Director of Epidemiologic Research, Department of Research and Evaluation, Kaiser Permanente Southern California at Kristi.Reynolds@kp.org.
Funding Statement
This work and the medical writing and editorial assistance provided by MEDiSTRAVA were funded by Moderna, Inc.
References
- 1.Bruxvoort KJ, Sy LS, Qian L, Ackerson BK, Luo Y, Lee GS, et al. Real-world effectiveness of the mRNA-1273 vaccine against COVID-19: Interim results from a prospective observational cohort study. Lancet Reg Health Am. 2022;6:100134. Epub 2021/12/02. doi: 10.1016/j.lana.2021.100134 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Paris C, Perrin S, Hamonic S, Bourget B, Roue C, Brassard O, et al. Effectiveness of mRNA-BNT162b2, mRNA-1273, and ChAdOx1 nCoV-19 vaccines against COVID-19 in healthcare workers: an observational study using surveillance data. Clin Microbiol Infect. 2021;27(11):1699 e5- e8. Epub 2021/07/16. doi: 10.1016/j.cmi.2021.06.043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Tenforde MW, Self WH, Adams K, Gaglani M, Ginde AA, McNeal T, et al. Association Between mRNA Vaccination and COVID-19 Hospitalization and Disease Severity. JAMA. 2021;326(20):2043–54. Epub 2021/11/05. doi: 10.1001/jama.2021.19499 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Pilishvili T, Fleming-Dutra KE, Farrar JL, Gierke R, Mohr NM, Talan DA, et al. Interim Estimates of Vaccine Effectiveness of Pfizer-BioNTech and Moderna COVID-19 Vaccines Among Health Care Personnel—33 U.S. Sites, January-March 2021. MMWR Morb Mortal Wkly Rep. 2021;70(20):753–8. Epub 2021/05/21. doi: 10.15585/mmwr.mm7020e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Mallapaty S. COVID vaccines slash viral spread—but Delta is an unknown. Nature. 2021;596(7870):17–8. Epub 2021/07/30. doi: 10.1038/d41586-021-02054-z [DOI] [PubMed] [Google Scholar]
- 6.Wu K, Werner AP, Moliva JI, Koch M, Choi A, Stewart-Jones GBE, et al. mRNA-1273 vaccine induces neutralizing antibodies against spike mutants from global SARS-CoV-2 variants. bioRxiv. 2021;2021.01.25.427948. Epub 2021/01/28. doi: 10.1101/2021.01.25.427948 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Scobie HM, Johnson AG, Suthar AB, Severson R, Alden NB, Balter S, et al. Monitoring Incidence of COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Status—13 U.S. Jurisdictions, April 4-July 17, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(37):1284–90. Epub 2021/09/17. doi: 10.15585/mmwr.mm7037e1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lopez Bernal J, Andrews N, Gower C, Gallagher E, Simmons R, Thelwall S, et al. Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant. N Engl J Med. 2021;385(7):585–94. Epub 2021/07/22. doi: 10.1056/NEJMoa2108891 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Twohig KA, Nyberg T, Zaidi A, Thelwall S, Sinnathamby MA, Aliabadi S, et al. Hospital admission and emergency care attendance risk for SARS-CoV-2 delta (B.1.617.2) compared with alpha (B.1.1.7) variants of concern: a cohort study. Lancet Infect Dis. 2022;22(1):35–42. Epub 2021/08/31. doi: 10.1016/S1473-3099(21)00475-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Brown CM, Vostok J, Johnson H, Burns M, Gharpure R, Sami S, et al. Outbreak of SARS-CoV-2 Infections, Including COVID-19 Vaccine Breakthrough Infections, Associated with Large Public Gatherings—Barnstable County, Massachusetts, July 2021. MMWR Morb Mortal Wkly Rep. 2021;70(31):1059–62. Epub 2021/08/06. doi: 10.15585/mmwr.mm7031e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Keehner J, Horton LE, Binkin NJ, Laurent LC, Alliance S, Pride D, et al. Resurgence of SARS-CoV-2 Infection in a Highly Vaccinated Health System Workforce. N Engl J Med. 2021;385(14):1330–2. Epub 2021/09/02. doi: 10.1056/NEJMc2112981 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Baden LR, El Sahly HM, Essink B, Follmann D, Neuzil KM, August A, et al. Phase 3 Trial of mRNA-1273 during the Delta-Variant Surge. N Engl J Med. 2021;385(26):2485–7. Epub 2021/11/03. doi: 10.1056/NEJMc2115597 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bruxvoort KJ, Sy LS, Qian L, Ackerson BK, Luo Y, Lee GS, et al. Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study. BMJ. 2021;375:e068848. Epub 2021/12/17. doi: 10.1136/bmj-2021-068848 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Nasreen S, Chung H, He S, Brown KA, Gubbay JB, Buchan SA, et al. Effectiveness of COVID-19 vaccines against symptomatic SARS-CoV-2 infection and severe outcomes with variants of concern in Ontario. Nat Microbiol. 2022;7(3):379–85. Epub 2022/02/07. doi: 10.1038/s41564-021-01053-0 [DOI] [PubMed] [Google Scholar]
- 15.Bajema KL, Dahl RM, Prill MM, Meites E, Rodriguez-Barradas MC, Marconi VC, et al. Effectiveness of COVID-19 mRNA Vaccines Against COVID-19-Associated Hospitalization—Five Veterans Affairs Medical Centers, United States, February 1-August 6, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(37):1294–9. Epub 2021/09/17. doi: 10.15585/mmwr.mm7037e3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Grannis SJ, Rowley EA, Ong TC, Stenehjem E, Klein NP, DeSilva MB, et al. Interim Estimates of COVID-19 Vaccine Effectiveness Against COVID-19-Associated Emergency Department or Urgent Care Clinic Encounters and Hospitalizations Among Adults During SARS-CoV-2 B.1.617.2 (Delta) Variant Predominance—Nine States, June-August 2021. MMWR Morb Mortal Wkly Rep. 2021;70(37):1291–3. Epub 2021/09/17. doi: 10.15585/mmwr.mm7037e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Self WH, Tenforde MW, Rhoads JP, Gaglani M, Ginde AA, Douin DJ, et al. Comparative Effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) Vaccines in Preventing COVID-19 Hospitalizations Among Adults Without Immunocompromising Conditions—United States, March-August 2021. MMWR Morb Mortal Wkly Rep. 2021;70(38):1337–43. Epub 2021/09/24. doi: 10.15585/mmwr.mm7038e1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Scott J, Richterman A, Cevik M. Covid-19 vaccination: evidence of waning immunity is overstated. BMJ. 2021;374:n2320. Epub 2021/09/25. doi: 10.1136/bmj.n2320 [DOI] [PubMed] [Google Scholar]
- 19.Andrews N, Tessier E, Stowe J, Gower C, Kirsebom F, Simmons R, et al. Duration of Protection against Mild and Severe Disease by Covid-19 Vaccines. N Engl J Med. 2022;386(4):340–50. Epub 2022/01/12. doi: 10.1056/NEJMoa2115481 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Koebnick C, Langer-Gould AM, Gould MK, Chao CR, Iyer RL, Smith N, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Perm J. 2012;16(3):37–41. Epub 2012/09/27. doi: 10.7812/tpp/12-031 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.California Department of Public Health. Reporting Requirements at a Glance [9 December 2021]. Available from: https://eziz.org/assets/docs/COVID19/IMM-1329.pdf
- 22.Kim DH, Schneeweiss S, Glynn RJ, Lipsitz LA, Rockwood K, Avorn J. Measuring Frailty in Medicare Data: Development and Validation of a Claims-Based Frailty Index. J Gerontol A Biol Sci Med Sci. 2018;73(7):980–7. Epub 2017/12/16. doi: 10.1093/gerona/glx229 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Miettinen OS. Theoretical Epidemiology: Principles of Occurrence Research in Medicine: New York: John Wiley & Sons. Print.; 1985. [Google Scholar]
- 24.California Department of Public Health. Updated COVID-19 Vaccine Eligibility Guidelines 2021. (updated 12 May 2021) [updated 12 May 20219 December 2021]. Available from: https://www.cdph.ca.gov/Programs/CID/DCDC/Pages/COVID-19/VaccineAllocationGuidelines.aspx [Google Scholar]
- 25.Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med. 2021;384(5):403–16. Epub 2020/12/31. doi: 10.1056/NEJMoa2035389 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Swift MD, Breeher LE, Tande AJ, Tommaso CP, Hainy CM, Chu H, et al. Effectiveness of Messenger RNA Coronavirus Disease 2019 (COVID-19) Vaccines Against Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Infection in a Cohort of Healthcare Personnel. Clin Infect Dis. 2021;73(6):e1376–e9. Epub 2021/04/27. doi: 10.1093/cid/ciab361 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Malden DE, Bruxvoort KJ, Tseng HF, Ackerson B, Choi SK, Florea A, et al. Distribution of SARS-CoV-2 Variants in a Large Integrated Health Care System—California, March-July 2021. MMWR Morb Mortal Wkly Rep. 2021;70(40):1415–9. Epub 2021/10/08. doi: 10.15585/mmwr.mm7040a4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Tang P, Hasan MR, Chemaitelly H, Yassine HM, Benslimane FM, Al Khatib HA, et al. BNT162b2 and mRNA-1273 COVID-19 vaccine effectiveness against the SARS-CoV-2 Delta variant in Qatar. Nat Med. 2021;27(12):2136–43. Epub 2021/11/04. doi: 10.1038/s41591-021-01583-4 [DOI] [PubMed] [Google Scholar]
- 29.Tseng HF, Ackerson BK, Luo Y, Sy LS, Talarico CA, Tian Y, et al. Effectiveness of mRNA-1273 against SARS-CoV-2 Omicron and Delta variants. Nat Med. 2022. Epub 2022/02/22. doi: 10.1038/s41591-022-01753-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Centers for Disease Control and Prevention. COVID-19 Vaccine Booster Shots 2022. [updated February 2; cited 2022 March 4]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/vaccines/booster-shot.html [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(PDF)
(PDF)
Data Availability Statement
Individual-level data reported in this study involving human research participants are not publicly shared due to potentially identifying or sensitive patient information. Upon request, and subject to review, KPSC may provide the deidentified aggregate-level data that support the findings of this study. Anonymized data (deidentified data including participant data as applicable) that support the findings of this study may be made available from the investigative team in the following conditions: (1) agreement to collaborate with the study team on all publications, (2) provision of external funding for administrative and investigator time necessary for this collaboration, (3) demonstration that the external investigative team is qualified and has documented evidence of training for human subjects protections, and (4) agreement to abide by the terms outlined in data use agreements between institutions. Interested researchers should contact Ana Florea, PhD, MPH, the corresponding author, at ana.florea@kp.org or Kristi Reynolds, PhD, MPH, Director of Epidemiologic Research, Department of Research and Evaluation, Kaiser Permanente Southern California at Kristi.Reynolds@kp.org.