Skip to main content
NCBI home page
PLOS Global Public Health logoLink to PLOS Global Public Health
. 2022 Jul 19;2(7):e0000561. doi: 10.1371/journal.pgph.0000561

Definitions matter: Heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis

Philippe J Guérin 1,2,*, Alistair R D McLean 1,2, Sumayyah Rashan 1,2, AbdulAzeez Lawal 1,2, James A Watson 2,3, Nathalie Strub-Wourgaft 4, Nicholas J White 2,3
Editor: Stephen J Kerr5
PMCID: PMC10021556  PMID: 36962738

Abstract

Therapeutic efficacy in COVID-19 is dependent upon disease severity (treatment effect heterogeneity). Unfortunately, definitions of severity vary widely. This compromises the meta-analysis of randomised controlled trials (RCTs) and the therapeutic guidelines derived from them. The World Health Organisation ‘living’ guidelines for the treatment of COVID-19 are based on a network meta-analysis (NMA) of published RCTs. We reviewed the 81 studies included in the WHO COVID-19 living NMA and compared their severity classifications with the severity classifications employed by the international COVID-NMA initiative. The two were concordant in only 35% (24/68) of trials. Of the RCTs evaluated, 69% (55/77) were considered by the WHO group to include patients with a range of severities (12 mild-moderate; 3 mild-severe; 18 mild-critical; 5 moderate-severe; 8 moderate-critical; 10 severe-critical), but the distribution of disease severities within these groups usually could not be determined, and data on the duration of illness and/or oxygen saturation values were often missing. Where severity classifications were clear there was substantial overlap in mortality across trials in different severity strata. This imprecision in severity assessment compromises the validity of some therapeutic recommendations; notably extrapolation of “lack of therapeutic benefit” shown in hospitalised severely ill patients on respiratory support to ambulant mildly ill patients is not warranted. Both harmonised unambiguous definitions of severity and individual patient data (IPD) meta-analyses are needed to guide and improve therapeutic recommendations in COVID-19. Achieving this goal will require improved coordination of the main stakeholders developing treatment guidelines and medicine regulatory agencies. Open science, including prompt data sharing, should become the standard to allow IPD meta-analyses.

Introduction

Viral burdens in COVID-19 infection peak early, around the time of symptom onset, and then decline. In the minority of patients requiring hospitalisation and respiratory support, inflammatory processes dominate [1,2]. As a result, drugs used to treat COVID-19 may have varying efficacy depending on where the patient is in the disease course when the medicines are administered. Under this simple paradigm antiviral drugs would be expected to be most beneficial when administered as early as possible in the evolution of the individual infection, and less likely to benefit once inflammatory processes dominate later in the disease [3]. At this late stage [4], immune modulating and anti-inflammatory drugs are of proven benefit. Conversely, immune suppression could be harmful early in the infection by attenuating an effective early immune response to viral replication. The results of the seminal RECOVERY trial on dexamethasone provide some support for this paradigm, with evidence of treatment effect heterogeneity according to patient severity at randomisation, a proxy measurement of disease progression [5]. Low dose dexamethasone (6mg/day) resulted in lower 28-day mortality among hospitalised patients who were receiving either invasive mechanical ventilation at randomization (RR = 0.64; 95% CI, 0.51–0.81) or oxygen alone at randomization (RR = 0.82; 95% CI, 0.72–0.94). In contrast, among those receiving no respiratory support at randomisation, the mortality figures were in the direction of harm (RR = 1.19; 95% CI, 0.92–1.55, p<0.001 for trend across three groups). The WHO guidelines for the treatment of COVID-19 acknowledged this interaction by providing separate recommendations stratified by disease severity for corticosteroids, recommending their use in severe and critical patients whilst conditionally recommending against their use in non-severe patients [6]. Conversely, monoclonal antibodies targeting the virus spike proteins have shown benefit for mild to moderately ill patients, whilst their efficacy was not demonstrated in more severely ill cases [7,8].

In the absence of precise knowledge of when a patient was infected, clinicians use history and disease severity at presentation to assess the appropriate clinical management. Disease severity is a continuous spectrum but, as is common for many potentially severe infectious diseases, researchers have partitioned COVID-19 severity into discrete categories of mild, moderate, severe and critical. This is useful for triage and, if these definitions were consistent, would also be useful for comparison of clinical and epidemiological observations, investigations and trials–as in the COVID-19 Network Meta-Analysis (NMA) underpinning the WHO guidelines. Most groups use the oxygen saturation level (SpO2) at rest in ambient air; respiratory rate (breaths per minute) and–when available–the ratio of arterial oxygen partial pressure (mmHg) to fractional inspired oxygen (PaO2/FiO2) in their severity criteria. However, the thresholds used vary substantially. The US National Institutes of Health (NIH) and the World Health Organization (WHO) have proposed different sets of criteria to categorise patients by severity (Fig 1 and S1 Table). Most notably these criteria differ on the saturation of oxygen (SpO2) threshold to define a severe case (the NIH considers an individual with <94% SpO2 to be a severe case, whilst the WHO requires <90% SpO2). The living network meta-analysis underpinning the WHO treatment guidelines, referred hereafter as “WHO–COVID19 Living Network Meta-Analysis” [6,9] and the international COVID-NMA Initiative [10,11] both use different severity definitions ((Fig 1 and S1 Table).

Fig 1. Severity definitions from WHO, US NIH guidelines and organisations conducting review of evidence i.e. the WHO—COVID-19 Living Network Meta-Analysis and the COVID-NMA imitative (RR: Respiratory rate; SpO2: Blood Oxygen Saturation; PaO2/FiO2: Ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

Fig 1

Open in a new tab

When the efficacy of a drug varies depending upon disease severity at time of treatment, as in COVID-19, harmonised definitions of severity are essential to provide severity specific estimates of drug efficacy. So, if definitions of patient severity vary across published reports, stratified efficacy estimates are likely to be compromised. Severity specific estimates of drug efficacy are compromised further if definitions based on clinical observations are not accompanied by pertinent data on date of symptoms onset and/or oxygen saturation values. We sought to determine the feasibility of retrospectively classifying the disease severity of patients included in the COVID-19 clinical trials upon which the current WHO living therapeutic guidelines are based. This was obtained from published information by extracting severity components from the clinical trial publications included in the living network meta-analysis.

Materials and methods

Identification of studies

We evaluated the 85 trials that met the inclusion criteria for the WHO—COVID-19 Living Network Meta-Analysis for drug treatments for COVID-19 (update 2, published 17th of December 2020). From these 85 trials, we could extract information from 81 (S2 Table). The full texts of the remaining four were inaccessible to us by 1st May 2021.

Categorical indicators of disease severity

We extracted information on the following variables and categorical indicators of COVID-19 severity at baseline: outpatients; patients with pneumonia; patients receiving oxygen therapy; inpatients; ICU patients. We extracted whether a) the results section (including the baseline table) indicated that any participants had any of the indicators listed above; b) if any of the indicators were listed as an exclusion criterion; c) if any of the indicators were listed as a necessary inclusion criterion; and d) if any of the indicators were listed as a sufficient but not necessary inclusion criterion. We then determined whether in the study all, some or no participants had any of the indicators, or if it was not known if any participants had any of the indicators or not.

Continuous indicators of severity

We extracted information on the following continuous indicators of COVID-19 severity at baseline: days since symptom onset; oxygen saturation level (SpO2) at rest in ambient air; respiratory rate (breaths per minute); the ratio of arterial oxygen partial pressure (mm Hg) to fractional inspired oxygen (PaO2/FiO2). For all continuous indicators we extracted the following measures where available: median; lower quartile; upper quartile; mean; standard deviation; observed minimum in study population; observed maximum in study population; minimum inclusion criteria where the inclusion criteria were sufficient but not necessary; maximum inclusion criteria where the inclusion criteria were sufficient but not necessary; minimum inclusion criteria where the inclusion criteria were necessary; maximum inclusion criteria where the inclusion criteria were necessary; proportions and ranges of categorisations. Where the observed measures were presented separately by arm we extrapolated overall study information where possible (in the case of the minimum and maximum measures for continuous indicators). Where study level measures were not identifiable, measures reported for the arm with the larger sample size were used. In addition, we classified whether the observed measures we extracted are representative of the entire study population or a single study arm.

Network Meta-Analysis (NMA) classifications

We extracted the mean age, mortality and possible severity range of patients listed online by the WHO–COVID-19 Living Network Meta-Analysis group [6,9] which provide evidence to WHO (https://www.covid19lnma.com/drug-treatments-study-level). Where the WHO–COVID-19 Living Network Meta-Analysis group listed a study arm as having “1 or 2” deaths in it (Zhao 2020) we did not include this arm in our calculation of mortality. We also extracted the severity classifications according to the COVID-NMA Initiative [10,11] (https://covid-nma.com/living_data/). We used the data available at these sites as on the 4th of March 2021.

The severity of two studies [12,13] included in update 2 of the WHO—COVID-19 Living Network Meta-Analysis group site were not present on the website so these were extracted from the original manuscript. Where severity was captured as not reported (NR) by the WHO—COVID-19 Living Network Meta-Analysis group we considered that patients with this severity may have been present in the trial and therefore considered this as part of the possible severity range of patients.

Data analysis

Data were summarised using descriptive statistics including percentages and counts where appropriate. Figures were generated using Stata 17.0 (College Station, TX, USA).

Ethical approval

As all the data were anonymised, available in the public domain and aggregated without any personal information, ethics approval was deemed unnecessary.

Results

Severity classification according to NMA groups

Of the 81 studies (S2 Table), 24 (30%) studies included patients from one severity category only in the WHO—COVID-19 Living Network Meta-Analysis (5 studies mild patients only; 8 moderate only; 7 severe only; 4 critical only) ((Fig 2). The majority (57/81, 70%) of studies included patients with a range of clinical severities (12 mild-moderate; 3 mild-severe; 19 mild-critical; 5 moderate-severe; 8 moderate-critical; 10 severe-critical). The severity classification used by the COVID-NMA initiative usually did not align with that used by the WHO—COVID-19 Living Network Meta-Analysis group. Of 70 studies that had been classified by both sources (The COVID-NMA initiative did not list four trials, six trials had “unclear” severity and one trial was unclassified), only 26 (37%) (of studies were classified as having the same range of severity by both groups (S3 Table). In studies with a single patient severity category, as expected, mortality rates stratified roughly according to severity. However, in studies with multiple severity strata, there was considerable overlap in mortality across the different severity definitions with variation not explained by mean patient age (Fig 3).

Fig 2. Possible range of patient severity in studies as classified by WHO- COVID-19 Living Network Meta-Analysis group (black circles) and COVID-NMA initiative (red squares).

Fig 2

Open in a new tab

Matched severity definiton between the two groups (green triangle), unmatched (red circle).

Fig 3.

Fig 3

Open in a new tab

Trial mortality among trials with (A) only a single stratum of patient severity and (B) a range of possible patient severity as reported by https://www.covid19lnma.com/drug-treatments-study-level. Green circles are from trials with mean patient age<50 years; orange for trials with mean patient age 50–60 years; blue for trials with mean patient age >60 years. Boxes denote quartiles. Circle width corresponds to number of participants analysed (<75 participants; 75–150 participants; 150–300 participants; > = 300 participants from smallest to largest).

Categorical indicators of severity

We extracted information on whether patients included in the trials were inpatients; outpatients; hospitalised in ICU; diagnosed with pneumonia; and receiving oxygen therapy. The reporting of whether all, some or no patients had each categorical indicator of severity are shown in Fig 4. In the large majority of studies 65/81 (80%) all patients were inpatients, while in 7 (9%) all patients were outpatients, 5 (6%) contained some inpatients but it was unclear if any outpatients were included, while in the remaining 4 (5%) studies it could not be ascertained if outpatients or inpatients were included. In the majority (54/65, 83%) of non-outpatient studies it was unclear if any of the patients were in ICU at baseline. Four (5%) studies reported all patient were in ICU; six (7%) reported some; and 17 (21%) reported none. In 33/81 (41%) studies all patients had pneumonia, with some patients with pneumonia reported in 19 (23%) studies. It was unknown in the remaining 29 (36%) studies. In the majority (47/81, 58%) of studies it was unclear if any of the patients were on oxygen therapy at baseline. Only one (1%) study indicated that no patients were on oxygen therapy, one (1%) study indicated that all patients were on oxygen therapy, and the remaining 32/79 (40%) reported that some patients were receiving oxygen.

Fig 4. Heatplot of studies and whether any patients in the study were outpatients; inpatients; ICU patients; patients with pneumonia; patients receiving oxygen at baseline.

Fig 4

Open in a new tab

Continuous indicators

Of the 81 studies, the majority, 54 (66%) provided information on the intervals from symptom onset ((Fig 5); 37 studies (46%) presented days since symptom onset as quartiles of the patient population; 13 studies gave means and standard deviations; three studies reported the minimum eligible in terms of day of onset for inclusion; 13 studies reported the maximum time eligible; two studies reported the observed minimum and maximum; and one study reported proportions of individuals falling into categorisations. Of the studies that reported a minimum number of days since symptom onset threshold to be eligible for inclusion, two reported 3 days and one reported 7 days. Of the studies that reported a maximum number of days since symptom onset threshold to be eligible for inclusion, three reported 4 days, one reported 6 days, two reported 8 days, one reported 10 days, three reported 12 days, one reported 13 days and two reported 14 days.

Fig 5. Available information on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

Fig 5

Open in a new tab

Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. The same graph stratified by outpatient composition (all/none/unknown) is presented in S1–S3 Figs. SpO2: Blood Oxygen Saturation; PaO2/FiO2: Ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

Information on SpO2 was provided in 44 studies (54%). Eleven studies presented SpO2 quartiles of the patient population; 12 studies gave means and standard deviations; 14 studies reported the minimum eligible for inclusion and three studies reported the maximum eligible for inclusion; 18 studies reported a maximum threshold of SpO2 that was sufficient but not necessary for inclusion; no studies reported the observed minimum and maximum; and two studies reported proportions of individuals falling within various SpO2 ranges. There was substantial heterogeneity in SpO2 thresholds. Of the studies that reported a minimum SpO2 threshold required for inclusion three gave a threshold of 95%, six gave 94%, three gave 93%, one gave 90% and one gave 75%. Of the studies that reported a maximum SpO2 threshold to be eligible for inclusion two gave 94% and one gave 90%.

Only 41 of 81 (51%) studies reported information on the respiratory rates (RR) of their patients. There were 16 studies which presented RR quartiles of the patient population; 11 studies gave means and standard deviations; three studies reported the minimum eligible for inclusion and nine studies reported the maximum eligible for inclusion; nine studies reported a maximum threshold of RR that was sufficient, but not necessary, for inclusion and one study reported a minimum threshold of RR that was sufficient, but not necessary, for inclusion; seven studies reported proportions of individuals falling into categorisations. Of the studies that reported a minimum RR threshold to be eligible for inclusion there was one each for thresholds of 19, 25 and 30 breaths/minute. Of the studies that reported a maximum threshold to be eligible for inclusion one gave 23, four gave 29, three gave 30 and one gave 35 breaths/minute.

Only 29 of 81 (36%) studies reported information on PaO2/FiO2 ratios. Four studies presented PaO2/FiO2 quartiles of the patient population; five studies gave means and standard deviations; eight studies reported the minimum eligible for inclusion and five studies reported the maximum eligible for inclusion; thirteen studies reported a maximum threshold of SpO2 that was sufficient but not necessary for inclusion. Of the studies that reported a minimum threshold for inclusion, one gave a PaO2/FiO2 = 76, one gave 100, five gave 300 and one gave 301. Of the studies that reported a maximum threshold to be eligible for inclusion one gave 200, one gave 250 and three gave 299.

Discussion

Information on COVID-19 disease severity in clinical trials is critical for the interpretation of therapeutic responses. Unfortunately, the summary information reported in the majority of COVID-19 clinical trial publications is insufficient to determine retrospectively, with sufficient accuracy, the distribution of disease severities of the patients included in the studies. In many studies, the key measures of severity were not reported at all. When they were reported they were often incomplete or ambiguous, e.g. being on oxygen-therapy at admission. Many of the current definitions used for COVID-19 severity combine a mixture of AND and OR logical operators. For example, the WHO guideline definition considers a patient to have severe disease if they have a respiratory rate>30/minute or SpO2<90% on room air. These are not equivalent. A study that reports proportions of patients above and below these thresholds separately does not provide sufficient information to determine the proportion of patients meeting the criteria for severe infection. In the absence of this information in the trial reports, and with such variability in definitions, guidelines based on summary data are compromised. Analysis of the Individual Participant Data (IPD) is required to assess therapeutic responses in relation to disease severity.

The published literature contains a wide variety of COVID-19 severity threshold criteria, severity definitions and severity categories—many of which are arbitrary (i.e. they have not been calibrated either by mortality or complications). The WHO panel noted that “the oxygen saturation threshold of 90% to define severe covid-19 was arbitrary” [6]. The other indicator thresholds such as respiratory rate or PaO2/FiO2 are also arbitrary, and they are also not generally agreed upon. Providing two alternative criteria for severity is also potentially misleading. Although few would disagree that acute onset of hypoxia resulting in an oxygen saturation of <90% in a patient with previously normal lung function is a sign of severity, a rapid respiratory rate can reflect a number of processes including anxiety. To add to the confusion, during the course of the pandemic some of the definitions were changed e.g. the WHO clinical guidance for COVID-19 published on 27 May 2020 (version 3) defined severity of COVID-19 by clinical indicators, but modified the oxygen saturation threshold from 94% to 90% [14], in order to align with previous WHO guidance [15]. The current severity definitions used by the US-NIH and WHO are expert-based consensus definitions. A measure of severity should be based on available data and outcomes, and include disease specific factors linked to patient prognosis (independent of underlying patient factors). The large variation in mortality observed between trials supposedly including patients with similar levels of disease severity illustrates the problem. While this study did not specifically review the compliance to clinical trial reporting guidelines, several did not follow standard practices. Despite the urgency of reporting clinical outcomes during a pandemic, using CONSORT and other reporting guidelines should remain the standard and be enforced by Editors. In summary, the majority of the RCTs which formed the evidence base for the WHO therapeutic guidelines included mixed populations, with unclear distributions of severity and associated outcomes, and significant variability in observed mortalities even among groups classified as having similar severities.

This is a significant concern because in COVID-19 there is strong evidence for heterogeneity of treatment effects according to disease severity at the time of treatment, as seen with corticosteroids [2,5,7,8]. Fortunately, the large randomised controlled trials in hospitalised patients have provided robust evidence and thus guidance for the management of severe COVID-19, but substantial uncertainty remains for chemoprevention and the treatment of early COVID-19. Epidemiological and therapeutic assessments would benefit substantially from agreement on definitions of severity and full reporting of key clinical measures, so that the quality and thus value of meta-analyses can be improved.

The slow global roll-out of vaccines and the threat of new variants means that effective therapeutics are still needed urgently. Most therapeutic trials on COVID-19 have been on hospitalised patients and most trials reported to date contain insufficient information to classify accurately the range of disease severity at randomisation. Researchers and policy makers must be careful not to over-interpret currently available data. In particular, the extrapolation of “lack of benefit” observed in hospitalised severely ill patients on respiratory support to ambulant mildly ill patients is not warranted. Considering the heterogeneity of disease severity definitions reported here and the amalgamation of patient outcomes reported in the literature, individual patient data (IPD) meta-analyses are needed now to guide and improve therapeutic recommendations in COVID-19. To date, effective data sharing initiatives have been successfully established for longitudinal data, such as the ones from the International Severe Acute Respiratory Infections Consortium (ISARIC) and the Virus Covid-19 Registry [16,17]. Sharing clinical trial data is still, two years after the start of the pandemic, in its infancy. Some funders are making data sharing a requirement attached to their grants, but they rarely enforce it, and the timeline to do so is often left to investigators. In a recently published review of COVID-19 clinical trial registries, intention to share individual participant data was reported in a minority of studies (348/1314, 26%) [18]. Data sharing will only happen if both incentives for investigators and clear policies from funders, scientific journals, regulators, policymakers and other stakeholders are aligned. This paradigm shift is likely to be successful if the research community embraces the culture of “Open Science”. Even during a pandemic and its urgent need for openness, this work simply illustrated that open science should be a standard but remains to date a wishful policy. Conducting IPD meta-analyses on clinical trial data require international collaborations to ensure equitable data sharing, following FAIR principles (Fig 6). IPD meta-analyses are particularly important for assessing drug efficacy in the large majority of patients who do not yet require hospitalisation.

Fig 6. Proposed strategy to generate COVID-19 treatment guidelines.

Fig 6

Open in a new tab

Supporting information

S1 Fig. Available information among studies that only contain outpatients on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. SpO2: Blood Oxygen Saturation; PaO2/FiO2: ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

(TIFF)

Click here for additional data file. (4MB, tiff)
S2 Fig. Available information among studies that contain no outpatients on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. SpO2: Blood Oxygen Saturation; PaO2/FiO2: ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

(TIFF)

Click here for additional data file. (3.8MB, tiff)
S3 Fig. Available information among studies with unknown outpatient composition on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. SpO2: Blood Oxygen Saturation; PaO2/FiO2: ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

(TIFF)

Click here for additional data file. (3.9MB, tiff)
S1 Table. Severity definitions.

(DOCX)

Click here for additional data file. (22.3KB, docx)
S2 Table. Trial publication details.

(DOCX)

Click here for additional data file. (37.4KB, docx)
S3 Table. Agreement between WHO—COVID-19 Living Network Meta-Analysis and COVID-NMA initiative groups with respect to possible minimum and maximum severity of trial participants (n = 70).

(DOCX)

Click here for additional data file. (18.3KB, docx)

Acknowledgments

We thank Brittany Maguire for support on the methodological approach, Emile Guerin for his technical support on figures and Lucy Peers for graphic design. This work is commissioned by the COVID-19 clinical trial coalition (https://covid19crc.org/).

Data Availability

All the data used in this study are publicly available and properly cited. The analysis database if freely available “Guerin, P. (2022). Analysis Database for "Definitions matter: heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis", Harvard Dataverse. https://doi.org/10.7910/DVN/YORCZN.

Funding Statement

This work is partly supported by the COVID-19 Clinical Research Coalition, funded by the Federal Ministry of Education and Research (bundesministerium für bildung und forschung) through KfW, Germany (ref: 2020 62 156) to PG, and the Republic and the canton de genève, International Solidarity Service, Switzerland (ref convention 2020) to PG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Cevik M, Tate M, Lloyd O, Maraolo AE, Schafers J, Ho A. SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis. Lancet Microbe 2021;2(1):e13–e22. doi: 10.1016/S2666-5247(20)30172-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Paules CI, Fauci AS. COVID-19: The therapeutic landscape. Med (N Y). 2021. May 14;2(5):493–497. doi: 10.1016/j.medj.2021.04.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Gonçalves A, Bertrand J, Ke R, Comets E, de Lamballerie X, Malvy D et al. Timing of Antiviral Treatment Initiation is Critical to Reduce SARS-CoV-2 Viral Load. CPT Pharmacometrics Syst Pharmacol. 2020;9(9):509–14. doi: 10.1002/psp4.12543 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.García LF. Immune Response, Inflammation, and the Clinical Spectrum of COVID-19. Front Immunol. 2020;11:1441. doi: 10.3389/fimmu.2020.01441 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.The RECOVERY Collaborative Group, Horby P, Lim WS, Emberson JR, Mafham M, Bell JL et al. Dexamethasone in Hospitalized Patients with Covid-19. N Engl J Med. 2021;384(8):693–704. doi: 10.1056/NEJMoa2021436 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Siemieniuk RA, Bartoszko JJ, Ge L, Zeraatkar D, Izcovich A, Kum E et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ 2020;370:m2980 doi: 10.1136/bmj.m2980 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.ACTIV-3/TICO LY-CoV555 Study Group, Lundgren JD, Grund B, Barkauskas CE, Holland TL, Gottlieb RL et al. A Neutralizing Monoclonal Antibody for Hospitalized Patients with Covid-19. N Engl J Med. 2021;384(10):905–14. doi: 10.1056/NEJMoa2033130 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Chen P, Nirula A, Heller B, Gottlieb RL, Boscia J, Morris J, et al. SARS-CoV-2 Neutralizing Antibody LY-CoV555 in Outpatients with Covid-19. N Engl J Med. 2021;384(3):229–37. doi: 10.1056/NEJMoa2029849 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Rochwerg B, Siemieniuk RA, Agoritsas T, Siemieniuk RA, Agoritsas T, Askie L et al. A living WHO guideline on drugs for covid-19. BMJ 2020;370:m3379. doi: 10.1136/bmj.m3379 [DOI] [PubMed] [Google Scholar]
  • 10.Boutron I, Chaimani A, Devane D, Barnier J, Vuillemot R, Ferrand G et al. Interventions for the prevention and treatment of COVID‐19: a living mapping of research and living network meta‐analysis. Cochrane Database of Systematic Reviews 2020(11) CD013769 doi: 10.1002/14651858.CD013769 [DOI] [Google Scholar]
  • 11.The COVID-NMA initiative A living mapping and living systematic review of Covid-19 trials [Available from: https://covid-nma.com/ accessed 21/05/2021].
  • 12.Mitjà O, Corbacho-Monné M, Ubals M, Tebé C, Peñafiel J, Tobias A et al. Hydroxychloroquine for Early Treatment of Adults With Mild Coronavirus Disease 2019: A Randomized, Controlled Trial. Clin Infect Dis.2021. Dec 6;73(11):e4073–e4081 doi: 10.1093/cid/ciaa1009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Mitjà O, Corbacho M, Tebé C, Tobias A, Ballana E, Mitjà J et al. Hydroxychloroquine alone or in combination with Cobicistat-boosted Darunavir for treatment of mild Covid-19: a cluster-randomized clinical trial. 2020. [Available at SSRN: https://ssrn.com/abstract=3615997] 10.2139/ssrn.3615997. [DOI]
  • 14.WHO. Clinical management of COVID-19: interim guidance Geneva: World Health Organization; 2020. [Available at https://www.who.int/publications/i/item/clinical-management-of-covid-19, accessed 1 September 2020]. [Google Scholar]
  • 15.WHO. IMAI district clinician manual. Hospital care for adolescents and adults. Guidelines for the management of common illnesses with limited resources. Geneva: World Health Organization; 2011. [Available at https://apps.who.int/iris/rest/bitstreams/109693/retrieve]. [Google Scholar]
  • 16.ISARIC Clinical Characterisation Group. The value of open-source clinical science in pandemic response: lessons from ISARIC. Lancet Infect Dis. 2021. 21(12): p. 1623–1624. doi: 10.1016/S1473-3099(21)00565-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Walkey AJ, Kumar VK, Harhay MO, Bolesta S, Bansal V, Gajic O, et al., The Viral Infection and Respiratory Illness Universal Study (VIRUS): An International Registry of Coronavirus 2019-Related Critical Illness. Crit Care Explor, 2020. 2(4): p. e0113. doi: 10.1097/CCE.0000000000000113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.McLean ARD, Rashan S, Tran L, Arena L, Lawal A, Maguire BJ et al., The fragmented COVID-19 therapeutics research landscape: a living systematic review of clinical trial registrations evaluating priority pharmacological interventions. [version 1; peer review: 1 approved]. Wellcome Open Res. 2022. 7(24). [Available at 10.12688/wellcomeopenres.17284.1]. [DOI] [Google Scholar]
PLOS Glob Public Health. doi: 10.1371/journal.pgph.0000561.r001

Decision Letter 0

Stephen John Kerr

7 Feb 2022

PGPH-D-21-00411

Definitions matter: heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis

PLOS Global Public Health

Dear Dr. Guerin,

Thank you for submitting your manuscript to PLOS Global Public Health. The reviewers have recommended minor revisions. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by 22nd March 2022. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at globalpubhealth@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pgph/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Stephen Kerr, PhD

Academic Editor

PLOS Global Public Health

Journal Requirements:

1. Please provide separate figure files in .tif or .eps format only.  Please ensure that all files are under our size limit of 20MB.  

For more information about how to convert your figure files please see our guidelines: Once you've converted your files to .tif or .eps, please also make sure that your figures meet our format requirements

https://journals.plos.org/globalpublichealth/s/figures

2. Please update the completed 'Competing Interests' statement, including any COIs declared by your co-authors. If you have no competing interests to declare, please state "The authors have declared that no competing interests exist". Otherwise please declare all competing interests beginning with the statement "I have read the journal's policy and the authors of this manuscript have the following competing interests:"

3. We have noticed that you have uploaded supporting information but you have not included a list of legends.  Please add a full list of legends for all supporting information files (including figures, table and data files) after the references list. 

4. In the online submission form, you indicated that "All the data used in this study are publicly available and properly cited. However, more guided instruction to get access to the data for transparency and reproducibility will be provided on request.". All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information.

This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons by return email and your exemption request will be escalated to the editor for approval. Your exemption request will be handled independently and will not hold up the peer review process, but will need to be resolved should your manuscript be accepted for publication. One of the Editorial team will then be in touch if there are any issues.

5. Please amend your detailed Financial Disclosure statement. This is published with the article, therefore should be completed in full sentences and contain the exact wording you wish to be published.

ii). State the initials, alongside each funding source, of each author to receive each grant.

iii). State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

Reviewer 1.

Overall: This is a timely and informative study. The authors carefully reviewed the current treatment guidelines and the studies used to inform the guidelines and provided logical and evidence-based critiques on the inconsistency in definitions/categorizations of COVID 19 disease severity. I (the reviewer) consider myself as a population health researcher with limited clinical experience, but I may be able to provide perspectives that are representative of the potential readers of the journal. Another overall comment is that the figures in this study are very informative, but I am afraid they are not self-explanatory enough. In general, more detailed notes/legends will be very helpful. Please consider the following recommendations.

(I found it very inconvenient to write comments without the availability of line numbers. Please consider adding line numbers if the manuscript needs reviews in the future)

Introduction:

(1) The concepts of “disease severity” and “stage of the disease” have been used interchangeably in the current study. But to me, the stage of the disease seems to be more related to time/phase instead of severity, especially when being used to determine, for example, whether a patient is at the initial infection phase of the disease where the viral burden peaks, or at the later phase when inflammatory processes dominate. Further clarifying these concepts is needed because the introduction section particularly emphasized the importance of determining disease course, and how crucial it is in terms of therapeutic recommendations (the timing of when to administer medicines). In the intro, the authors may consider at least briefly answering why a more consistent and more precise severity assessment can help better identify the stage of the disease, and thus guide and improve therapeutic recommendations.

(2) Though already specified in the main text, from a reader’s perspective, it is still helpful and necessary to have notes explaining the abbreviation used in the figures. (For example, Figure 1, respiratory rate and oxygen saturation level.)

(3) In the last paragraph of the introduction, the aim of the study is phrased “We sought to determine the feasibility of determining retrospectively the disease severity of patients included in the COVID-19 clinical trials upon which the current WHO living therapeutic guidelines are based.” I personally think that this sentence can be potentially reframed and simplified. Also, the authors may reconsider whether the aim of the study is determining the feasibility of the current measures, or examining the consistency or comparability of the current measures.

Materials and Methods:

(4) Under the subtitle of “categorical indicators of disease severity”, since the categories indicate severity, it will be very helpful if they are ordered accordingly, starting from the least severe category to the most severe one.

Results:

(5) Figure 2 needs notes/legends. Please specify the meaning of red dots and green triangles. Please specify the difference between the red and black dash lines.

(6) Figure 3 needs notes/legends and reorganization. Please specify the meaning of the colored dots. Please relocate the letter A and letter B, so they do not overlap with the figure title.

(7) Figure 5 needs notes/legends. Please specify what the different colored arrows mean.

Discussion:

(8) The paper clearly pointed out the drawbacks of the current severity measurement. It might be obvious to the authors that how individual patient data meta-analysis will be able to contribute, but to people who are not very familiar with clinical research, more details on how the proposed solution will provide evidence for a consistent and accurate severity definition will be extremely helpful. In other words, I do think the current research revealed very important issues of the current literature, but I do not think it indicates the future direction of the research clearly enough.

Reviewer 2

This is a cleverly conceived methods paper which highlights the difficulties of both setting up trials for a new disease in the absence of clear understanding of the disease pathogenesis, and how interventions might vary in effectiveness at different disease stages, and making recommendations on therapeutics at a global level. It is useful to highlight how the deficiencies described in the paper have compromised meta-analysis and recommendations, but I would like the authors to expand the discussion on why this has happened, and strategies so that it would not happen again in future pandemics.

The abstract recommendations are a logical conclusion from the findings, but could there be a final line on how this could be achieved?

I think it would be useful to have the individual studies that have been used to generate the meta-analytic findings are listed as references in a supplement. There are no descriptions of what software was used to generate the figures and calculate the summary statistics, so it would be useful to include this information in the methods.

I agree with the authors that standard definitions is a very important initiative, but given that authors have not reported key clinical measures used in their studies, could the authors propose a strategy for achieving this? Poor reporting has been noted for many years, and the equator-network has sought to develop guidelines to improve reporting standards.

Many journals have accepted the manuscripts with incomplete reporting of key clinical measures, even though CONSORT and other reporting guidelines exist, and these guidelines ask for eligibility criteria, completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed. Part of the problem is therefore manuscripts accepting reports, when the ideal information in the reports is not given. For journals that ask require reporting guideline checklists, this responsibility ultimately is a failure of the editorial and review process. The discussion should note these deficiencies, and suggest a strategy for improvement.

I suspect some of this poor reporting reflects the haste of authors to publish their work, and the haste of journals to publish comparatively new information. International collaboration has been poor in many aspects of this pandemic, including investigating viral origins, and ensuring equitable access to vaccines.

I also agree that international collaborations are required to ensure equitable data sharing. Some, but not all journals require that data is available in a public registry, and authors are often reluctant to agree to this demand. Do the authors have suggestions on how this aspect could be improved?

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Global Public Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Global Public Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Overall: This is a timely and informative study. The authors carefully reviewed the current treatment guidelines and the studies used to inform the guidelines and provided logical and evidence-based critiques on the inconsistency in definitions/categorizations of COVID 19 disease severity. I (the reviewer) consider myself as a population health researcher with limited clinical experience, but I may be able to provide perspectives that are representative of the potential readers of the journal. Another overall comment is that the figures in this study are very informative, but I am afraid they are not self-explanatory enough. In general, more detailed notes/legends will be very helpful. Please consider the following recommendations.

(I found it very inconvenient to write comments without the availability of line numbers. Please consider adding line numbers if the manuscript needs reviews in the future)

Introduction:

(1) The concepts of “disease severity” and “stage of the disease” have been used interchangeably in the current study. But to me, the stage of the disease seems to be more related to time/phase instead of severity, especially when being used to determine, for example, whether a patient is at the initial infection phase of the disease where the viral burden peaks, or at the later phase when inflammatory processes dominate. Further clarifying these concepts is needed because the introduction section particularly emphasized the importance of determining disease course, and how crucial it is in terms of therapeutic recommendations (the timing of when to administer medicines). In the intro, the authors may consider at least briefly answering why a more consistent and more precise severity assessment can help better identify the stage of the disease, and thus guide and improve therapeutic recommendations.

(2) Though already specified in the main text, from a reader’s perspective, it is still helpful and necessary to have notes explaining the abbreviation used in the figures. (For example, Figure 1, respiratory rate and oxygen saturation level.)

(3) In the last paragraph of the introduction, the aim of the study is phrased “We sought to determine the feasibility of determining retrospectively the disease severity of patients included in the COVID-19 clinical trials upon which the current WHO living therapeutic guidelines are based.” I personally think that this sentence can be potentially reframed and simplified. Also, the authors may reconsider whether the aim of the study is determining the feasibility of the current measures, or examining the consistency or comparability of the current measures.

Materials and Methods:

(4) Under the subtitle of “categorical indicators of disease severity”, since the categories indicate severity, it will be very helpful if they are ordered accordingly, starting from the least severe category to the most severe one.

Results:

(5) Figure 2 needs notes/legends. Please specify the meaning of red dots and green triangles. Please specify the difference between the red and black dash lines.

(6) Figure 3 needs notes/legends and reorganization. Please specify the meaning of the colored dots. Please relocate the letter A and letter B, so they do not overlap with the figure title.

(7) Figure 5 needs notes/legends. Please specify what the different colored arrows mean.

Discussion:

(8) The paper clearly pointed out the drawbacks of the current severity measurement. It might be obvious to the authors that how individual patient data meta-analysis will be able to contribute, but to people who are not very familiar with clinical research, more details on how the proposed solution will provide evidence for a consistent and accurate severity definition will be extremely helpful. In other words, I do think the current research revealed very important issues of the current literature, but I do not think it indicates the future direction of the research clearly enough.

Reviewer #2: This is a cleverly conceived methods paper which highlights the difficulties of both setting up trials for a new disease in the absence of clear understanding of the disease pathogenesis, and how interventions might vary in effectiveness at different disease stages, and making recommendations on therapeutics at a global level. It is useful to highlight how the deficiencies described in the paper have compromised meta-analysis and recommendations, but I would like the authors to expand the discussion on why this has happened, and strategies so that it would not happen again in future pandemics.

The abstract recommendations are a logical conclusion from the findings, but could there be a final line on how this could be achieved?

I think it would be useful to have the individual studies that have been used to generate the meta-analytic findings are listed as references in a supplement. There are no descriptions of what software was used to generate the figures and calculate the summary statistics, so it would be useful to include this information in the methods.

I agree with the authors that standard definitions is a very important initiative, but given that authors have not reported key clinical measures used in their studies, could the authors propose a strategy for achieving this? Poor reporting has been noted for many years, and the equator-network has sought to develop guidelines to improve reporting standards.

Many journals have accepted the manuscripts with incomplete reporting of key clinical measures, even though CONSORT and other reporting guidelines exist, and these guidelines ask for eligibility criteria, completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed. Part of the problem is therefore manuscripts accepting reports, when the ideal information in the reports is not given. For journals that ask require reporting guideline checklists, this responsibility ultimately is a failure of the editorial and review process. The discussion should note these deficiencies, and suggest a strategy for improvement.

I suspect some of this poor reporting reflects the haste of authors to publish their work, and the haste of journals to publish comparatively new information. International collaboration has been poor in many aspects of this pandemic, including investigating viral origins, and ensuring equitable access to vaccines.

I also agree that international collaborations are required to ensure equitable data sharing. Some, but not all journals require that data is available in a public registry, and authors are often reluctant to agree to this demand. Do the authors have suggestions on how this aspect could be improved?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLOS Glob Public Health. doi: 10.1371/journal.pgph.0000561.r003

Decision Letter 1

PLOS Manuscript Reassignment

8 Jun 2022

Definitions matter: heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis

PGPH-D-21-00411R1

Dear Prof Guerin,

We are pleased to inform you that your manuscript 'Definitions matter: heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis' has been provisionally accepted for publication in PLOS Global Public Health.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact globalpubhealth@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Global Public Health.

Best regards,

Stephen Kerr

Academic Editor

PLOS Global Public Health

***********************************************************

Reviewer Comments (if any, and for reference):

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Does this manuscript meet PLOS Global Public Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Global Public Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: Thank you for responding to my comments. This is an important contribution to the scientific literature

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Fig. Available information among studies that only contain outpatients on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

    Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. SpO2: Blood Oxygen Saturation; PaO2/FiO2: ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

    (TIFF)

    Click here for additional data file. (4MB, tiff)
    S2 Fig. Available information among studies that contain no outpatients on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

    Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. SpO2: Blood Oxygen Saturation; PaO2/FiO2: ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

    (TIFF)

    Click here for additional data file. (3.8MB, tiff)
    S3 Fig. Available information among studies with unknown outpatient composition on days since symptom onset; SpO2; respiratory rate; PaO2/FiO2.

    Dashed vertical lines denote thresholds used by either the NIH; WHO; Covid-NMA or Magic NMA groups to categorise severity. Eligible range (AND) refers to a criterion that is part of an AND condition and therefore an individual falling into this range would also require at least one other criterion to be met; eligible range (OR) refers to a criterion that is part of an OR condition. SpO2: Blood Oxygen Saturation; PaO2/FiO2: ratio of arterial oxygen partial pressure (PaO2 in mmHg) to fractional inspired oxygen (FiO2 expressed as a fraction)).

    (TIFF)

    Click here for additional data file. (3.9MB, tiff)
    S1 Table. Severity definitions.

    (DOCX)

    Click here for additional data file. (22.3KB, docx)
    S2 Table. Trial publication details.

    (DOCX)

    Click here for additional data file. (37.4KB, docx)
    S3 Table. Agreement between WHO—COVID-19 Living Network Meta-Analysis and COVID-NMA initiative groups with respect to possible minimum and maximum severity of trial participants (n = 70).

    (DOCX)

    Click here for additional data file. (18.3KB, docx)
    Attachment

    Submitted filename: Reponse to reviewers_230522_Plos Global Health_submitted.docx

    Click here for additional data file. (21.7KB, docx)

    Data Availability Statement

    All the data used in this study are publicly available and properly cited. The analysis database if freely available “Guerin, P. (2022). Analysis Database for "Definitions matter: heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis", Harvard Dataverse. https://doi.org/10.7910/DVN/YORCZN.

    Articles from PLOS Global Public Health are provided here courtesy of PLOS

    RESOURCES