Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2023 Nov 8;18(11):e0293449. doi: 10.1371/journal.pone.0293449

Risk of Coronavirus Disease 2019 (COVID-19) among those up-to-date and not up-to-date on COVID-19 vaccination by US CDC criteria

Nabin K Shrestha 1,*, Patrick C Burke 2, Amy S Nowacki 3, Steven M Gordon 1
Editor: Cecilia Acuti Martellucci4
PMCID: PMC10631626  PMID: 37939032

Abstract

Background

The CDC recently defined being “up-to-date” on COVID-19 vaccination as having received at least one dose of a COVID-19 bivalent vaccine. The purpose of this study was to compare the risk of COVID-19 among those “up-to-date” and “not up-to-date”.

Methods

Employees of Cleveland Clinic in Ohio, USA, in employment when the COVID-19 bivalent vaccine first became available, and still employed when the XBB lineages became dominant, were included. Cumulative incidence of COVID-19 since the XBB lineages became dominant was compared across the”up-to-date” and “not up-to-date” states, by treating COVID-19 bivalent vaccination as a time-dependent covariate whose value changed on receipt of the vaccine. Risk of COVID-19 by vaccination status was also evaluated using multivariable Cox proportional hazards regression adjusting for propensity to get tested for COVID-19, age, sex, most recent prior SARS-CoV-2 infection, and number of prior vaccine doses.

Results

COVID-19 occurred in 1475 (3%) of 48 344 employees during the 100-day study period. The cumulative incidence of COVID-19 was lower in the “not up-to-date” than the “up-to-date” state. On multivariable analysis, being “up-to-date” was not associated with lower risk of COVID-19 (HR, 1.05; 95% C.I., 0.88–1.25; P-value, 0.58). Results were very similar when those 65 years and older were only considered “up-to-date” after 2 doses of the bivalent vaccine.

Conclusions

Since the XBB lineages became dominant, adults “up-to-date” on COVID-19 vaccination by the CDC definition do not have a lower risk of COVID-19 than those “not up-to-date”, bringing into question the value of this risk classification definition.

Introduction

In April 2023 the Centers for Disease Control and Prevention (CDC) updated its guidance on Coronavirus Disease 2019 (COVID-19) vaccination to consider all individuals above the age of 6 to be “up-to-date” with COVID-19 vaccination only if they had received at least one dose of a COVID-19 bivalent vaccine [1]. By this definition, those who had not received a single dose of a COVID-19 bivalent vaccine would be considered not “up-to-date”. A recent study was unable to find the bivalent vaccine to be effective while the XBB lineages were the dominant circulating strains [2]. Given this lack of effectiveness of the bivalent vaccine against the XBB lineages, it was reasonable to question whether individuals “up-to-date” with a vaccine of questionable effectiveness were protected against COVID-19 compared to those not “up-to-date”, when the XBB lineages were the predominant circulating strains.

The purpose of this study was to evaluate whether individuals who were “up-to-date” on COVID-19 vaccination had a lower risk of COVID-19 than those not “up-to-date”.

Methods

Study design

This was a retrospective cohort study conducted at the Cleveland Clinic Health System (CCHS) in the United States.

Patient consent statement

The study was approved by the Cleveland Clinic Institutional Review Board as exempt research (IRB no. 22–917). A waiver of informed consent and waiver of HIPAA authorization were approved to allow the research team access to the required data.

Setting

Since the arrival of the COVID-19 pandemic at Cleveland Clinic in March 2020, employee access to testing has been a priority. Voluntary vaccination for COVID-19 began on 16 December 2020, and the monovalent mRNA vaccine as a booster became available to employees on 5 October 2021. The COVID-19 bivalent mRNA vaccine began to be offered to employees on 12 September 2022. The study start date was 29 January 2023, the date the XBB lineages first became the dominant circulating strains in Ohio.

Participants

CCHS employees in employment at any location in Ohio when the COVID-19 bivalent vaccine first became available (12 September 2022), and who were still employed when the XBB lineages became dominant (29 January 2023), were included in the study. Those for whom age and sex were not available were excluded.

Variables

Covariates collected were age, sex, job location, prior SARS-CoV-2 infection, and number of prior vaccine doses, as described in our earlier studies [2–5]. Institutional data governance rules related to employee data limited our ability to supplement our dataset with additional clinical variables. Vaccination status was “up-to-date” or “not up-to-date” based on whether or not a person had received at least one dose of a COVID-19 bivalent vaccine. For each subject, the propensity to get tested for COVID-19 was defined as the number of COVID-19 nucleic acid amplification tests (NAATs) done divided by the number of years of employment at CCHS during the pandemic. The pandemic phase during which a subject had his or her last prior episode of COVID-19 was also collected as a variable, based on which variant/lineages accounted for more than 50% of infections in Ohio at the time [6].

Outcome

The study outcome was time to COVID-19, the latter defined as a positive NAAT for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) any time after the study start date. Outcomes were followed until 10 May 2023, allowing for evaluation of outcomes up to 100 days from the study start date. Data were accessed on 11 May 2023. Authors had access to information that could identify individual participants. However all data were de-identified before analysis.

Statistical analysis

A Simon-Makuch hazard plot [7] was created to compare the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states, by treating vaccination status as a time-dependent covariate. A subject’s vaccination status was “not up-to-date” before receipt of a COVID-19 bivalent vaccine, and “up-to-date” after receipt of the vaccine. Subjects whose employment was terminated during the study period before they had COVID-19 were censored on the date of termination. Curves for the “not up-to-date” state were based on data while the vaccination status was “not up-to-date”. Curves for the “up-to-date” state were based on data from the date the vaccination status changed to “up-to-date”.

Multivariable Cox proportional hazards regression models were fit to examine the association of various variables with time to COVID-19. Vaccination status was included as a time-dependent covariate [8]. The possibility of multicollinearity in the models was evaluated using variance inflation factors. The proportional hazards assumption was checked using log(-log(survival)) vs. time plots. The hazard ratio for vaccination status provided a comparison of risk of COVID-19 based on vaccination status after adjusting for all other covariates.

The analysis was performed by N. K. S. and A.S.N. using the survival package and R version 4.2.2 (R Foundation for Statistical Computing) [8–10].

Results

Of 48 344 included subjects, 1445 (3%) were censored during the study because of termination of employment. By the end of the study, 12 841 (27%) were “up-to-date” on COVID-19 vaccination, of whom 11 187 (87%) received the Pfizer vaccine and 1654 (13%) received the Moderna vaccine. Those “up-to-date” by the end of the study included 37 of the 6174 who had previously been unvaccinated at the start of the study. A total of 1475 employees (3%) acquired COVID-19 during the 100 days of the study.

Baseline characteristics

Table 1 shows the characteristics of subjects included in the study. Notably, this was a relatively young population, with a mean age of 43 years. Among these, 22 407 (46%) had previously had a documented episode of COVID-19 and 16 262 (34%) had previously had an Omicron variant infection. 42 160 subjects (87%) had previously received at least one dose of vaccine and 44 432 (92%) had been previously exposed to SARS-CoV-2 by infection or vaccination. Altogether, 36 490 subjects (76%) were tested for COVID-19 by a NAAT at least once while employed at Cleveland Clinic. The propensity for COVID-19 testing ranged from 0 to 63.5 per year, with a median of 0.64 and interquartile range spanning 0.32 to 1.27 per year.

Table 1. Baseline characteristics of 48 344 employees of cleveland clinic in Ohio.

Characteristics Overall
Age in years, mean (SD) 42.5 (13.2)
Sex
 Female 35 953(74.4)
 Male 12 381 (25.6)
Location
 Cleveland Clinic Main 19 445 (40.2)
 Cleveland area regional hospitals 11 355 (23.5)
 Ambulatory centers 8443 (17.5)
 Cleveland Clinic Akron 3996 (8.3)
 Administrative centers 3971 (8.2)
 Cleveland Clinic Medina 1124 (2.3)
Prior documented COVID-19a 22 407 (46.4)
Prior Omicron variant infectionb 16 262 (33.6)
Most recent prior SARS-CoV-2 infection
 No documented infection 25 927 (53.6)
 During the pre-Omicron phase 6145 (12.7)
 During the Omicron BA.1/BA.2 dominant phase 8876 (18.4)
 During the Omicron BA.4/BA.5 dominant phase 5943 (12.3)
 During the Omicron BQ dominant phase 1443 (3.0)
Number of prior vaccine doses
 0 6174 (12.8)
 1 or 2 16 301 (33.7)
 3 14 731 (30.5)
 >3 11 128 (23.0)
Days since most recent infectionc, mean (SD) 365 (238)
Days since most recent vaccined, mean (SD) 370 (214)
Days since proximate SARS-CoV-2 exposuree, mean (SD) 306 (200)
Open in a new tab

Data are presented as no. (%) unless otherwise indicated.

Abbreviation: SD, standard deviation; COVID-19, Coronavirus Disease 2019; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus-2.

aDefined by a positive SARS-CoV-2 nucleic acid amplification test.

bDefined by a positive SARS-CoV-2 nucleic acid amplification test during a time when Omicron variants were the predominant strains in Ohio.

cAmong those with prior documented COVID-19.

dAmong those who received at least one dose of a COVID-19 vaccine.

eAmong those with prior documented COVID-19 or who received at least one dose of a COVID-19 vaccine.

Risk of COVID-19 based on vaccination status and prior infection

The risk of COVID-19 was lower in the “not up-to-date” state than in the “up-to-date” state, with respect to COVID-19 vaccination (Fig 1). When stratified by tertiles of propensity to get tested for COVID-19, the “up-to-date” state was not associated with a lower risk of COVID-19 than the “not up-to-date” state in any tertile (Fig 2).

Fig 1. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states with respect to COVID-19 vaccination.

Fig 1

Open in a new tab

Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility.

Fig 2. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states with respect to COVID-19 vaccination, stratified by tertiles of propensity to get tested for COVID-19.

Fig 2

Open in a new tab

Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. Solid lines represent the “up-to-date” states while dashed lines represent the “not up-to-date” states.

In contrast, consideration of prior infection provided a more accurate classification for risk of COVID-19. Those whose last prior SARS-CoV-2 infection occurred during the Omicron BQ or BA.4/BA.5 dominant phases had a substantially and significantly lower risk of COVID-19 than those whose last SARS-CoV-2 infection occurred in the early Omicron or pre-Omicron periods or those not previously known to have had COVID-19 (Fig 3). When stratified by most recent prior SARS-CoV-2 infection, there was no difference in risk of COVID-19 between the “up-to-date” and “not up-to-date” states within each most recent prior infection group, except for those not previously known to be infected, among whom the “up-to-date” state was associated with a higher risk of COVID-19 than the “not up-to-date” state (Fig 4).

Fig 3. Hazard plot comparing the cumulative incidence of COVID-19 stratified by the phase during which the last prior episode of COVID-19 occurred.

Fig 3

Open in a new tab

Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. Solid lines represent those last infected before the Omicron variants became dominant or those not previously known to have had COVID-19, while dashed lines represent those last infected while the Omicron variants were dominant.

Fig 4. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states with respect to COVID-19 vaccination, stratified by the phase during which the last prior episode of COVID-19 occurred.

Fig 4

Open in a new tab

Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. “Remote infection” includes infections during the pre-Omicron and Omicron BA.1/BA.2 dominant periods, i.e more than 218 days before the study start date. Solid lines represent the “up-to-date” states while dashed lines represent the “not up-to-date” states.

Multivariable analysis

In a multivariable Cox proportional hazards regression model, adjusted for propensity to get tested for COVID-19, age, sex, phase of most recent SARS-CoV-2 infection, and number of prior vaccine doses, being “up-to-date” on COVID-19 vaccination was not associated with a lower risk of COVID-19 than not being “up-to-date” (HR, 1.05; 95% C.I., 0.88–1.25; P-value, 0.58). Having had their last prior episode of COVID-19 while lineages of the Omicron variant were dominant was protective against COVID-19 with higher protection the more recent the infection. A higher number of prior vaccine doses was associated with higher risk of COVID-19. Point estimates of hazard ratios and associated 95% confidence intervals for the variables included in the unadjusted and adjusted Cox proportional hazards regression models are shown in Table 2. If number of prior vaccine doses was not adjusted for, being “up-to-date” on COVID-19 vaccination was associated with a higher risk of COVID-19 that not being “up-to-date” (HR, 1.30; 95% C.I., 1.16–1.45; P-value, <0.001). Results were very similar if those 65 years and older were only considered “up-to-date” after receipt of at least 2 doses of a bivalent vaccine (Table 1). Results were also very similar if an individual was only considered “up-to-date” 7 days after receipt of a bivalent vaccine (Table 2).

Table 2. Unadjusted and adjusted associations With time to COVID-19.

Variables Unadjusted HR (95% CI) P Adjusted HR (95% CI)a P
Vaccination status “up-to-date”b 1.28 (1.15–1.43) <0.001 1.05 (0.88–1.25) 0.58
Propensity to get tested for COVID-19c 1.07 (1.06–1.09) <0.001 1.09 (1.08–1.10) <0.001
Age 1.001 (0.997–1.005) 0.56 0.997 (0.993–1.001) 0.09
Male sex 0.83 (0.74–0.94) 0.004 0.80 (0.70–0.90) <0.001
Most recent prior SARS-CoV-2 infectiond
 During Pre-Omicron phase 1.11 (0.96–1.29) 0.14 1.07 (0.92–1.24) 0.38
 During Omicron BA.1/BA.2 dominant phase 0.91 (0.80–1.04) 0.18 0.83 (0.72–0.95) 0.006
 During Omicron BA.4/BA.5 dominant phase 0.31 (0.24–0.40) <0.001 0.27 (0.21–0.34) <0.001
 During Omicron BQ dominant phase 0.12 (0.05–0.27) <0.001 0.09 (0.04–0.21) <0.001
Number of prior vaccine dosese
 1 or 2 1.67 (1.36–2.07) <0.001 1.72 (1.39–2.13) <0.001
 3 1.94 (1.57–2.39) <0.001 2.15 (1.74–2.66) <0.001
 >3 2.16 (1.74–2.67) <0.001 2.26 (1.73–2.95) <0.001
Open in a new tab

Abbreviation: COVID-19, Coronavirus Disease 2019; HR, hazard ratio; CI, confidence interval; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus-2

aFrom a multivariable Cox-proportional hazards regression model.

bTime-dependent covariate

cCalculated as number of COVID-19 nucleic acid amplification tests done per year of employment at Cleveland Clinic during the course of the pandemic.

dReference: No documented prior infection.

eReference: Zero doses

Discussion

This study found that being “up-to-date” on COVID-19 vaccination, using the current CDC definition, was not associated with a lower risk of COVID-19 than not being “up-to-date”, while the XBB lineages were the dominant circulating strains of SARS-CoV-2, after adjusting for important factors that might influence risk of COVID-19.

The strengths of our study include its large sample size, and its conduct in a healthcare system that devoted resources to have an accurate accounting of who had COVID-19, when COVID-19 was diagnosed, who received a COVID-19 vaccine, and when. The study methodology, treating vaccination status as a time-dependent covariate, allowed for determination of vaccine effectiveness in real time. Adjusting for the propensity to get tested for COVID-19 should have mitigated against bias that might arise from infections being more likely to be detected in individuals who tested more frequently. Furthermore, those who received the COVID-19 bivalent vaccine did so with an expectation that it provided protection against COVID-19. So, if there was a difference in likelihood of getting tested before and after receiving the COVID-19 bivalent vaccine, one would expect relatively less testing in the “up-to-date” state than in the “not up-to-date” state, having the effect of undercounting infections in the “up-to-date” state, thereby falsely overestimating the beneficial effect of being “up-to-date”.

Our study was limited to examination of all detected infections. We were unable to distinguish between symptomatic and asymptomatic infections, and the rarity of severe illnesses precluded studying whether being “up-to-date” decreased severity of illness. Information on prior COVID-19 would have been incomplete, as many asymptomatic and mildly symptomatic infections would have been missed. Infections diagnosed by in-home testing without follow-up PCR testing would have been missed, but there is no reason to think these would have been missed in the “up-to-date” and “not up-to-date” states at rates different enough to change the results significantly. There may have been unconsidered variables that might have influenced the results. Lastly, our study was done in a healthcare population, and included no children and few elderly subjects, and the majority of study subjects would not have been immunocompromised.

Specific prior infections, prior vaccinations, time since proximate exposure to SARS-CoV-2 by prior infection or vaccination, number of prior vaccine doses, and interval between vaccines, all possibly contribute to determining an individual’s risk of acquiring COVID-19. It is difficult to account for the effects of all these factors in an individual. What this study clearly shows is that a simplistic categorization of being or not being “up-to-date” on COVID-19 bivalent vaccination did not provide an accurate classification of risk of acquiring COVID-19 while the XBB lineages were the dominant circulating strains. The questionable effectiveness of the vaccine, as has been previously demonstrated [2], is likely a significant contributor to the lack of accuracy of risk classification based on receipt of the vaccine.

This study’s findings question the wisdom of promoting the idea that every person needs to be “up-to-date” on COVID-19 vaccination, as currently defined, at this time. It is often stated that the primary purpose of vaccination is to prevent severe COVID-19 and death. We certainly agree with this, but it should be pointed out that there has not yet been a single study that has shown that the COVID-19 bivalent vaccine protects against severe disease or death caused by the XBB lineages of the Omicron variant. At least one prior study has failed to find a protective effect of the bivalent vaccine against the XBB lineages of SARS-CoV-2 [2]. Additionally, the finding of higher risk of COVID-19 with higher number of prior vaccine doses in this and our prior study [2], should be taken into consideration. People may still choose to get the vaccine, but an assumption that the vaccine protects against severe disease and death is not reason enough to unconditionally push a vaccine of questionable effectiveness to all adults.

In conclusion, this study found that being “up-to-date” on COVID-19 vaccination by the CDC definition was not associated with a lower risk of COVID-19 than not being “up-to-date”. This study highlights the challenges of counting on protection from a vaccine when the effectiveness of the vaccine decreases over time as new variants emerge that are antigenically very different from those used to develop the vaccine. It also demonstrates the folly of risk classification based solely on receipt of a vaccine of questionable effectiveness while ignoring protection provided by prior infection.

Data Availability

Data are available from https://osf.io/bufwc/.

Funding Statement

The authors received no specific funding for this work.

References

Decision Letter 0

Cecilia Acuti Martellucci

25 Aug 2023

PONE-D-23-21544Risk of Coronavirus Disease 2019 (COVID-19) among Those Up-to-Date and Not Up-to-Date on COVID-19 VaccinationPLOS ONE

Dear Dr. Shrestha,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 05 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Cecilia Acuti Martellucci, M.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This clearly written paper adds to some knowledge of the efficacy of Covid-19 bivalent vaccines. However, we do have some comments that we hope will help the authors to improve the quality of their work.

Title: As PLOS ONE is aimed at an international audience, the title should indicate that "up-to-date" and "not up-to-date" refer to US-CDC definitions.

Abstract: Please provide information on the timeframe and location (Ohio, USA) of the study.

Methods:

The paper lacks information on the timing of bivalent vaccination in the "vaccinated" group. Could you add a table or figure showing the weekly or monthly distribution of this vaccination? This is important to discuss the impact of waning protection. Perhaps the time since last vaccination could be added as an explanatory variable in the model.

Please provide information on the vaccination status of the "unvaccinated group" (which is partially vaccinated, if I understand correctly).

Discussion

It's worth noting that the reference [2] does not take into account the time since the last vaccination as a confounding factor of the time period or the variant.

In my opinion, the discussion focuses too much on the results of univariate or bivariate analysis. The authors should focus on discussing the multivariate Cox model, as this model was adjusted for confounding.

“There are two reasons why not being “up-to-date” on COVID-19 vaccination by the CDC

definition was associated with a lower risk of COVID-19.” Are you referring to the raw cumulative risk or the adjusted risk of the multivariate Cox model?

The fact that the "not up-to-date" group is less at risk in the multivariate analysis, despite adjustment for covariates (in particular propensity to be tested, and prior infection) is an important result that should be discussed. What are the authors' hypotheses to explain this?

Reviewer #2: The authors examined the risk of COVID-19 in the period when the XBB variant became dominant among 48344 healthcare workers, and specifically compared the risk among those who had received at least one dose of bivalent vaccine (up to date) versus those without any bivalent vaccines (not up to date). The authors found that adults with ‘not up to date’ vaccine had a lower risk of infection than those with ‘up to date’ bivalent vaccine. I have a few concerns and comments as outlined below:

1. I noticed in the authors’ previous publication (Nabin K Shrestha et al., Effectiveness of the Coronavirus Disease 2019 Bivalent Vaccine, Open Forum Infectious Diseases, 2023), the effectiveness of bivalent vaccine was compared with non-bivalent vaccine in XBB-dominant period, and found no significant differences in risk of infection. I am wondering what the additional value of this manuscript on top of that paper is? Are there any differences in data and methodology? Also, the previous paper found no difference, while the current manuscript reported a lower risk, any explanations for this?

2. In the discussion, the authors discussed two reasons why ‘not up to date’ was associated with a lower risk of infection. First, I agree that the bivalent vaccine was less effective against XBB than BA.1 or BA.5, however, monovalent vaccine was also less effective against XBB, and studies have found an even lower neutralization of XBB by monovalent vaccine than bivalent vaccine (e.g. Kurhade, C., Zou, J., Xia, H. et al. Nat Med, 2023; Zou J, Kurhade C, Patel S, et al. N Engl J Med, 2023). Second, in the multivariable model, the authors adjusted for the most recent prior SARS-CoV-2 infection. Assuming prior infection was correctly captured, the lower adjusted HR in ‘not up to date’ group should not be affected by prior infection. Therefore, I don’t think the reasons discussed can explain the risk difference?

3. The study outcome (XBB infection) and exposure (previous infection) are sorely based on NAAT testing in employment. Little information was found regarding the testing behaviour (e.g. was it symptomatic testing? routing testing?), which can potentially induce bias.

As mentioned in the manuscript, the number of NAATs done divided by the number of years of employment was only 0.64 (0.32-1.27), which means on average each participant only had less than one test per year. This low testing frequency will likely generate a lot of missed infections. This can also be seen from Table 1, over half (53.6%) did not have documented infection before the emergence of XBB variant, which was quite high compared with similar healthcare settings, given the high infection rate in Omicron period. Therefore, missed infections are very likely to occur. While this was mentioned in the limitations, it was not thoroughly discussed and could impact the validity of the conclusion. The lower risk could be caused by the ‘not up to date’ group having more missed Omicron infections than the ‘up to date’ group, which I think is the main reason for the observed lower risk. Therefore, this evidence cannot be used to support a low effectiveness of bivalent vaccine.

4. Given the above reason, I would consider it inappropriate to conclude in the discussion and call a bivalent vaccine “questionable effectiveness”. Although bivalent vaccines have lower neutralizing capabilities against XBB variant, there is evidence showing they can provide additional protection against XBB infections (e.g. Link-Gelles R, Ciesla AA, Roper LE, et al. MMWR Morb Mortal Wkly Rep. 2023), and was associated with a lower risk of infection or severe infection with XBB variants (Lin DY, Xu Y, Gu Y, Zeng D, Sunny SK, Moore Z. N Engl J Med. 2023). These studies should be discussed. I would agree with not giving repeated boosters to all adults, but vaccines/bivalent vaccines would remain beneficial to the elderly and vulnerable groups.

5. Among 48344 participants, were all of them vaccinated? Among those ‘not up to date’ cohort, how many participants received 1, 2, 3, and 4 doses of vaccine? Would the number of vaccinations influence the results?

6. The references in the introduction and discussion are a little bit sparse, more studies on bivalent vaccine and XBB variant should be referenced and discussed, including the papers I mentioned above.

Reviewer #3: The analysis is interesting and intriquing, presentation of the results is clear and easily readable. Please add the titles to the figures presented, the titles should represent the aim and the main content of the figures.

I`m a little bit sceptic bout one sentence in the Discussion part of the manuscript, that sounds like " People may still choose to get vaccine, but an assumption that the vaccine protects against severe disease and death is not enough to unconditionally push a vaccine of questionable effectiveness to all adults". First of all, presented study does not address that issues, i.e vaccine effectiveness against severe disease, so it is quite boldly to claim it in the manuscript.

Here are some new studies about that

https://pubmed.ncbi.nlm.nih.gov/37043647/

https://pubmed.ncbi.nlm.nih.gov/37561053/

It would be polite not to do wrong conclusions about vaccine effectiveness in combating severe COVID-19 infection.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Vincent Auvigne

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Nov 8;18(11):e0293449. doi: 10.1371/journal.pone.0293449.r002

Author response to Decision Letter 0

13 Sep 2023

Reviewer #1: This clearly written paper adds to some knowledge of the efficacy of Covid-19 bivalent vaccines. However, we do have some comments that we hope will help the authors to improve the quality of their work.

Title: As PLOS ONE is aimed at an international audience, the title should indicate that "up-to-date" and "not up-to-date" refer to US-CDC definitions.

Response: We added this to the title.

Abstract: Please provide information on the timeframe and location (Ohio, USA) of the study.

Response: We added the location, “Ohio, USA”, in the abstract. The relevant timeframe of the study (while the COVID-19 XBB lineages were dominant) is already mentioned in the abstract.

Methods:

The paper lacks information on the timing of bivalent vaccination in the "vaccinated" group. Could you add a table or figure showing the weekly or monthly distribution of this vaccination? This is important to discuss the impact of waning protection. Perhaps the time since last vaccination could be added as an explanatory variable in the model.

Response: The paper already provides this information. The numbers at risk in Figure 1, show the weekly distribution of bivalent vaccination. Those “up-to-date” were those who had received the bivalent vaccine. Those “not up-to-date” were those who had not. We but could not add time since last vaccination as an explanatory variable, because 6174 study participants had not received a vaccine prior to the study.

Please provide information on the vaccination status of the "unvaccinated group" (which is partially vaccinated, if I understand correctly).

Response: We did not have an “unvaccinated group” in the study. The reviewer may have meant to ask what proportion of previously unvaccinated individuals received the bivalent vaccine. 37 of the 6174 previously unvaccinated individuals received the bivalent vaccine. We have added this information in the first paragraph of the results.

Discussion

It's worth noting that the reference [2] does not take into account the time since the last vaccination as a confounding factor of the time period or the variant.

Response: Reference 2 adjusts for time since proximate SARS-CoV-2 exposure, which is scientifically more sound than adjusting for time since last vaccination. The study also adjusted for the variant.

In my opinion, the discussion focuses too much on the results of univariate or bivariate analysis. The authors should focus on discussing the multivariate Cox model, as this model was adjusted for confounding.

Response: The suggestion is valid. We have removed the entire paragraph that discusses the findings of the univariable analysis. We have added a new paragraph that discusses the findings of the multivariable analysis.

“There are two reasons why not being “up-to-date” on COVID-19 vaccination by the CDC

definition was associated with a lower risk of COVID-19.” Are you referring to the raw cumulative risk or the adjusted risk of the multivariate Cox model?

Response: This was referring to the comparison of the raw cumulative risk between those “up-to-date” and those “not up-to-date”. As stated in our response to the immediate prior comment, we have removed this entire paragraph.

The fact that the "not up-to-date" group is less at risk in the multivariate analysis, despite adjustment for covariates (in particular propensity to be tested, and prior infection) is an important result that should be discussed. What are the authors' hypotheses to explain this?

Response: In response to one of the reviewers’ comments we added number of prior vaccine doses as a covariate in the multivariable model. When we did this, the association of lower risk with not being “up-to-date” was no longer apparent. We accordingly changed the study’s conclusions.

We added the following as a new paragraph to discuss the findings on multivariable analysis:

‘Specific prior infections, prior vaccinations, time since proximate exposure to SARS-CoV-2 by prior infection or vaccination, number of prior vaccine doses, and interval between vaccines, all possibly contribute to determining an individual’s risk of acquiring COVID-19. It is difficult to account for the effects of all these factors in an individual. What this study clearly shows is that a simplistic categorization of being or not being “up-to-date” on bivalent vaccination did not provide an accurate classification of risk of acquiring COVID-19 while the XBB lineages were the dominant circulating strains. A significant contributor to the lack of accuracy of classification based on receipt of this vaccine is likely to be because the vaccine is of questionable effectiveness [2].’

Reviewer #2: The authors examined the risk of COVID-19 in the period when the XBB variant became dominant among 48344 healthcare workers, and specifically compared the risk among those who had received at least one dose of bivalent vaccine (up to date) versus those without any bivalent vaccines (not up to date). The authors found that adults with ‘not up to date’ vaccine had a lower risk of infection than those with ‘up to date’ bivalent vaccine. I have a few concerns and comments as outlined below:

1. I noticed in the authors’ previous publication (Nabin K Shrestha et al., Effectiveness of the Coronavirus Disease 2019 Bivalent Vaccine, Open Forum Infectious Diseases, 2023), the effectiveness of bivalent vaccine was compared with non-bivalent vaccine in XBB-dominant period, and found no significant differences in risk of infection. I am wondering what the additional value of this manuscript on top of that paper is? Are there any differences in data and methodology? Also, the previous paper found no difference, while the current manuscript reported a lower risk, any explanations for this?

Response: The purpose of the previous paper was to examine the effectiveness of the bivalent vaccine. The primary purpose of this study was not to examine the effectiveness of the vaccine, but to examine if the CDC classification into “up-to-date” or “not up-to-date” was a useful classification system.

Our previous study only included 2 weeks of data during the XBB dominant phase of the epidemic. This study included 12 weeks of data during the XBB dominant phase of the epidemic. With the revised multivariable analysis with addition of number of prior vaccine doses as a covariate, this study also did not find a difference, so the findings are still consistent with those of the prior study.

2. In the discussion, the authors discussed two reasons why ‘not up to date’ was associated with a lower risk of infection. First, I agree that the bivalent vaccine was less effective against XBB than BA.1 or BA.5, however, monovalent vaccine was also less effective against XBB, and studies have found an even lower neutralization of XBB by monovalent vaccine than bivalent vaccine (e.g. Kurhade, C., Zou, J., Xia, H. et al. Nat Med, 2023; Zou J, Kurhade C, Patel S, et al. N Engl J Med, 2023). Second, in the multivariable model, the authors adjusted for the most recent prior SARS-CoV-2 infection. Assuming prior infection was correctly captured, the lower adjusted HR in ‘not up to date’ group should not be affected by prior infection. Therefore, I don’t think the reasons discussed can explain the risk difference?

Response: We don’t dispute that the monovalent vaccine may have also been less effective against XBB than the earlier sub-lineages. However, when this study was done, all individuals were many months away from their last monovalent vaccine, and one should not expect any residual protection from the monovalent vaccine (as many papers have shown that protection from vaccine wanes within a few months). Moreover, the definition of being “up-to-date” was not determined by whether or not a person received a monovalent vaccine, nor by how many doses of monovalent vaccine a person had received. So the assertion that monovalent vaccine was also less effective against XBB, while true, has no bearing on the question examined in this study.

The reviewer is correct that because the multivariable model adjusted for most recent prior SARS-CoV-2 infection, prior infection should not be invoked to explain a lower HR in those “not up-to-date”. The reviewer is also correct that prior infection cannot explain a lower adjusted HR associated with the “not up to date” state, as the multivariable analysis adjusted for prior infection. The results of the revised multivariable analysis are different, and as discussed in our responses to the prior reviewer, we have removed this entire paragraph. We have added a new paragraph in its place, to discuss the results of the revised multivariable analysis.

3. The study outcome (XBB infection) and exposure (previous infection) are sorely based on NAAT testing in employment. Little information was found regarding the testing behaviour (e.g. was it symptomatic testing? routing testing?), which can potentially induce bias.

As mentioned in the manuscript, the number of NAATs done divided by the number of years of employment was only 0.64 (0.32-1.27), which means on average each participant only had less than one test per year. This low testing frequency will likely generate a lot of missed infections. This can also be seen from Table 1, over half (53.6%) did not have documented infection before the emergence of XBB variant, which was quite high compared with similar healthcare settings, given the high infection rate in Omicron period. Therefore, missed infections are very likely to occur. While this was mentioned in the limitations, it was not thoroughly discussed and could impact the validity of the conclusion. The lower risk could be caused by the ‘not up to date’ group having more missed Omicron infections than the ‘up to date’ group, which I think is the main reason for the observed lower risk. Therefore, this evidence cannot be used to support a low effectiveness of bivalent vaccine.

Response: It is not possible to retrospectively determine why individuals chose to get tested. One should not assume there would be bias by assuming that individuals “up-to-date” had different testing behavior than those “not up-to-date”. This study employed a time-dependent covariate to separate the “up-to-date” and “not up-to-date” state, so it would be the same individuals who would contribute data to the “up-to-date” state and the “not up-to-date” state. So it is not that different individuals contribute data to the different states. One could, of course, argue that getting the bivalent vaccine might have changed peoples’ testing behavior. If so, the most logical change one would expect in testing behavior in an individual is that one would be less likely to get tested for COVID-19 when “up-to-date” than when “not up-to-date”, because those who chose to receive the bivalent vaccine did so with the expectation that it would decrease their risk of COVID-19. If there was such a change in testing behavior, COVID-19 would have been less likely to have been detected in the “up-to-date” state than in the “not up-to-date” state. So it would not mask a beneficial effect of vaccination.

We agree that many infections were probably missed. By the time the later sub-lineages of the Omicron variant became dominant, COVID-19 was no longer the severe illness it was when it first appeared. At least some people would have chosen not to get tested when they had mild symptoms. However, this should not have affected the comparison between the “up-to-date” and “not up-to-date” states, because infections would have been missed in both states. As discussed in the previous section, if there was a difference in detection of infections in the two states because of testing behavior differences between the two states, it would likely be that there was comparatively less testing in the “up-to-date” state than in the “no up-to-date” state. If there was a bias in detecting COVID-19 based on difference in testing behavior, any such detection bias would have actually had the effect of masking some of the risk of COVID-19 among those “up-to-date”. We expanded the second paragraph of the discussion to explain this.

4. Given the above reason, I would consider it inappropriate to conclude in the discussion and call a bivalent vaccine “questionable effectiveness”. Although bivalent vaccines have lower neutralizing capabilities against XBB variant, there is evidence showing they can provide additional protection against XBB infections (e.g. Link-Gelles R, Ciesla AA, Roper LE, et al. MMWR Morb Mortal Wkly Rep. 2023), and was associated with a lower risk of infection or severe infection with XBB variants (Lin DY, Xu Y, Gu Y, Zeng D, Sunny SK, Moore Z. N Engl J Med. 2023). These studies should be discussed. I would agree with not giving repeated boosters to all adults, but vaccines/bivalent vaccines would remain beneficial to the elderly and vulnerable groups.

Response: We would like to clarify that this study does not draw the conclusion that the bivalent vaccine was of questionable effectiveness. This study concludes that the CDC classification into “up-to-date” and “not up-to-date” is not useful.

Our previous study failed to find the vaccine to be effective against the XBB lineages. That study was published in a major peer-reviewed infectious disease journal, and in the scores of communications we have received about the study not one provided a reasonable explanation for why that conclusion was not justified. We thus stand by the conclusion of the prior study that the vaccine was of questionable effectiveness. However, that conclusion pertains to our previous study, not to this one.

We have been aware of the two studies mentioned by the reviewer. These studies do not provide evidence that the bivalent booster provides additional protection against XBB variants.

The first study by Link-Gelles et al, published in MMWR, examined the effectiveness of the bivalent vaccine during a time when the BA.5 and XBB lineages were dominant. During the study period, a little over a third of infections were XBB/XBB.1.5-related infections. The majority were BA.5-related infections. We have shown in one of our earlier studies that the bivalent vaccine was effective in preventing infections caused by the BA.5 lineages. In the Link-Gelles study, the lack of effectiveness of the bivalent vaccine against XBB lineages (which comprised a minority of the infections) could have easily been masked by the effectiveness of the vaccine against the BA.5 lineages (which comprised the majority of the infections). Placing this study’s findings in context of other studies on the subject, one cannot use the Link-Gelles study’s findings to claim that the bivalent vaccine protects against XBB lineage infections.

The second study by Lin et al published in the NEJM does not examine the effectiveness of the bivalent vaccine against the XBB lineages of the Omicron variant. It examines the duration of protection provided by bivalent vaccination in two cohorts, the first one consisting of individuals who received the vaccine when the BA.4/BA.5 lineages were dominant, and the second one consisting of individuals who received the vaccine when the BQ or XBB lineages were dominant. Results of the second cohort showed that the bivalent vaccine was still about 20% effective 9-10 weeks after receipt of the vaccine. It should however, be noted that the majority of the study occurred before the XBB lineages became dominant. Only a very small proportion of the study occurred when the XBB lineages were dominant, and the XBB lineages only became dominant during the last two weeks of the study. There are data only for 2-3 weeks for those who received the bivalent vaccine during XBB lineage dominance, and lack of effectiveness against the XBB lineages could easily have been masked by effectiveness against the BQ lineages, as most of the data for the first three weeks since receipt of the bivalent booster would have consisted of data accrued while the BQ lineages were dominant. This study in no way shows that the bivalent vaccine protected against the XBB lineages of the Omicron variant.

There have been no studies that have actually shown that the bivalent vaccine is beneficial in the elderly or in the immunocompromised. It is widely understood that the elderly and the immunocompromised have weaker immune responses than an otherwise younger and healthier population. No vaccine has ever been shown to be beneficial in the immunocompromised when it is not beneficial in those who are not immunocompromised. The reason for strongly encouraging vaccination of the elderly and immunocompromised is not because they are expected to have a stronger immune response, but because they are at higher risk of adverse consequences of the infection and any protection, even if weaker than in a healthier population, is worth the effort of vaccination. But this, of course, assumes that the vaccine is effective among those able to mount an immune response. There is no logic to the assertion that a vaccine that is not effective in healthier individuals would miraculously be effective among the immunocompromised.

5. Among 48344 participants, were all of them vaccinated? Among those ‘not up to date’ cohort, how many participants received 1, 2, 3, and 4 doses of vaccine? Would the number of vaccinations influence the results?

Response: Of the 48344 participants, 6174 were not vaccinated at the start of the study. In table 1, we have added a breakdown of the number of vaccine doses received by the study participants at the start of the study. Adding the number of prior vaccine doses did indeed influence the results greatly! We have included this as a variable in the multivariable model. As we have found in prior studies, there was an association of increased risk of COVID-19 with higher number of prior vaccine doses even after adjustment for other factors. However, once this was added to the model, there was no longer an association of higher risk of COVID-19 with being “up-to-date” with the bivalent vaccine. Thank you for raising this question. It made a huge difference in the results.

Given the change in the results, we changed the reference level of the bivalent vaccinated status from “up-to-date” to “not up-to-date”, so that the results could be worded more simply. We also accordingly revised our conclusion to say that being “up-to-date” on COVID-19 vaccination by the CDC definition was not associated with a lower risk of COVID-19 than not being “up-to-date”.

6. The references in the introduction and discussion are a little bit sparse, more studies on bivalent vaccine and XBB variant should be referenced and discussed, including the papers I mentioned above.

Response: We acknowledge the suggestion to add additional references. Which references to include is a matter of writing style preference. We believe that we have included the references that support statements we have made in the introduction and discussion. We acknowledge that there are many other papers on the subject, but we did not think we have excluded any that would substantially alter the rationale for doing the study or placing the findings into context. Specifically, we chose not to reference the mentioned papers because their findings are either tangential to the issues being discussed or their claims are not supported by the data presented. Had we been able to find studies that found the bivalent vaccine to be effective against the XBB lineages, we would have certainly referenced them.

Reviewer #3: The analysis is interesting and intriquing, presentation of the results is clear and easily readable. Please add the titles to the figures presented, the titles should represent the aim and the main content of the figures.

Response: The figures have the appropriate titles. They may not have shown up properly in the version sent out for peer review. We understand that the figures will have the appropriate titles in the final printed version.

I`m a little bit sceptic bout one sentence in the Discussion part of the manuscript, that sounds like " People may still choose to get vaccine, but an assumption that the vaccine protects against severe disease and death is not enough to unconditionally push a vaccine of questionable effectiveness to all adults". First of all, presented study does not address that issues, i.e vaccine effectiveness against severe disease, so it is quite boldly to claim it in the manuscript.

Here are some new studies about that

https://pubmed.ncbi.nlm.nih.gov/37043647/

https://pubmed.ncbi.nlm.nih.gov/37561053/

It would be polite not to do wrong conclusions about vaccine effectiveness in combating severe COVID-19 infection.

Response:

We think that statement is very appropriate. In a prior study we failed to find the bivalent vaccine to be effective in preventing infection caused by the XBB lineages of the Omicron variant. We have not come across a single study that has truly shown the bivalent vaccine provided protection against severe infection caused by the XBB lineages. Moreover, in this and our prior study we found an association of increased risk of COVID-19 with increasing number of prior vaccine doses, in multivariable analyses. We have not said that people should not get the vaccine. We make the argument that one should not mandate a vaccine whose effectiveness is questionable.

In the next two paragraphs we specifically address the two papers suggested to be included as references, to explain why they were not included as references:

We have already described why the first paper (Lin et al NEJM 2023) does not make the case for bivalent vaccine effectiveness, in our response to comment # 4 from reviewer #2.

The findings as presented for the second paper also do not support the study’s assertion that the bivalent vaccine protected against COVID-19. Looking at table 2, which provides the main results of the study, a person who is paying attention will notice that the incidence rate per 100,000 was 0.587 in the reference group and 0.586 for the 15-60 day group. The incidence rate is almost identical in those groups. How does that square with the calculated relative vaccine effectiveness being 45.6%? There are similar questions for each of the groups. Perusal of the presented data shows that the incidence of severe COVID-19 was similar for each group (different time intervals since receipt of second bivalent booster) compared to the reference group (those who received only one booster dose), suggesting that receiving more than one booster dose provided no additional protection over receipt of only one booster dose. Even if we are misreading the results and there are no flaws in the methodology, this study was a comparison between the effectiveness of one booster dose versus multiple booster doses, not an evaluation of the effectiveness of the bivalent vaccine.

We agree that it is not helpful to draw wrong conclusions about vaccine effectiveness against severe COVID-19. This study does not make a conclusion about vaccine effectiveness against severe COVID-19. It only makes a conclusion about the usefulness of the “up-to-date” / “not up-to-date” classification.

Attachment

Submitted filename: response_to_reviewers.docx

Click here for additional data file. (24.4KB, docx)

Decision Letter 1

Cecilia Acuti Martellucci

13 Oct 2023

Risk of Coronavirus Disease 2019 (COVID-19) among Those Up-to-Date and Not Up-to-Date on COVID-19 Vaccination by US CDC Criteria

PONE-D-23-21544R1

Dear Dr. Shrestha,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Cecilia Acuti Martellucci, M.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Most of our comments have been taken into account by the authors. The article is now well focused on the evaluation of the US CDC vaccination criteria on the risk of Covid-19, which was its main objective.

Reviewer #2: The authors have addressed all my comments and I do not have any further comments. I believe the manuscript is much improved and the messages are clearly presented.

Reviewer #3: No comments here. Thanks for exciting research and staying committed to your idea. Your research adds freshness to the science and exactly enhance negotiations about vaccination plan implementation.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Vincent Auvigne

Reviewer #2: No

Reviewer #3: No

**********

Acceptance letter

Cecilia Acuti Martellucci

17 Oct 2023

PONE-D-23-21544R1

Risk of Coronavirus Disease 2019 (COVID-19) among Those Up-to-Date and Not Up-to-Date on COVID-19 Vaccination by US CDC Criteria

Dear Dr. Shrestha:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Cecilia Acuti Martellucci

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: response_to_reviewers.docx

    Click here for additional data file. (24.4KB, docx)

    Data Availability Statement

    Data are available from https://osf.io/bufwc/.

    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES