Abstract
Background
A key first step in optimising COVID-19 patient outcomes during future case-surges is to learn from the experience within individual hospitals during the early stages of the pandemic. The aim of this study was to investigate the extent of variation in COVID-19 outcomes between National Health Service (NHS) hospital trusts and regions in England using data from March–July 2020.
Methods
This was a retrospective observational study using the Hospital Episode Statistics administrative dataset. Patients aged ≥ 18 years who had a diagnosis of COVID-19 during a hospital stay in England that was completed between March 1st and July 31st, 2020 were included. In-hospital mortality was the primary outcome of interest. In secondary analysis, critical care admission, length of stay and mortality within 30 days of discharge were also investigated. Multilevel logistic regression was used to adjust for covariates.
Findings
There were 86,356 patients with a confirmed diagnosis of COVID-19 included in the study, of whom 22,944 (26.6%) died in hospital with COVID-19 as the primary cause of death. After adjusting for covariates, the extent of the variation in-hospital mortality rates between hospital trusts and regions was relatively modest. Trusts with the largest baseline number of beds and a greater proportion of patients admitted to critical care had the lowest in-hospital mortality rates.
Interpretation
There is little evidence of clustering of deaths within hospital trusts. There may be opportunities to learn from the experience of individual trusts to help prepare hospitals for future case-surges.
Keywords: COVID-19, Coronavirus, Mortality, Unwarranted variation, Variability
Research in context.
Evidence before this study
We searched PubMed on 20th January, 2021 for articles that documented risk factors for COVID-19-related in-hospital mortality using search terms “SARS-CoV-2″ OR “COVID-19″ AND “mortality” AND “hospital” in the article title or abstract. Of the 2175 papers identified, we found 135 publications identified as pertaining to "England" OR "United Kingdom". Of these, 88 were original research studies involving patient data of which 19 investigated mortality in COVID-19 patients and only one examined differences in COVID-19 in-hospital mortality rates between hospital trusts. Most studies focused on specific patient cohorts and none covered an entire hospital population nationally over an extended period.
Added value of this study
To our knowledge, this is the largest analysis of between hospital COVID-19 related mortality rates in any country or region published to date. After adjustment for covariates, variation of mortality rates for hospitalised COVID-19 patients was relatively modest between the seven English regions (range: 23.6% to 31.4%) and 126 non-specialist NHS hospital trusts included in the analysis. Hospitals with larger bed bases and a higher proportion of patients admitted to critical care had lower mortality rates.
Implications of all the available evidence
Our study emphasises that there was no unwarranted variation of COVID-19 in-patient outcomes between hospital trusts during the first five months of the COVID-19 pandemic in England. However, those institutions with greater resources, in terms of beds, had better outcomes.
Alt-text: Unlabelled box
1. Introduction
The COVID-19 pandemic has strained hospital systems globally, with a rapid and prolonged increase in patient flow to hospitals. Hospital management strategies and patient outcomes have varied internationally and are a function of differing infection rates, public health structures and healthcare system organisation [1]. Furthermore, analysis of COVID-19 related deaths in several countries demonstrate differences in patient outcomes regionally which are in part due to demographic differences in the population [2]. However, limited analysis has been published regarding inter-hospital variation in COVID-19 mortality rates. Although this is a sensitive topic, identifying inter-hospital variation, and then exploring reasons why it exists with clinicians, offers an opportunity to learn from previous practice.
In England, patient outcomes within National Health Service (NHS) hospital trusts during the pandemic will have been mediated by established patient risk factors (e.g. age, sex, frailty, ethnicity and comorbidities) and hospital factors (location, population density). Furthermore, the different and complex methods used for reporting patient death can make it difficult to determine mortality rates accurately [3]. Early reports suggested that some NHS hospital trusts in England had much higher death rates than others [4].
The Getting Right First Time programme in England aims to drive clinical improvement by understanding variation in practice and outcomes [5]. Our previous work using the Hospital Episodes Statistics (HES) dataset has provided an understanding of the patient factors that affected COVID-19 related morality rates and the reduction in adjusted probability of in-hospital deaths as the pandemic progressed [6]. Using the same comprehensive dataset, we aimed to investigate the nature of variation in COVID-19 in-hospital mortality rates regionally and between NHS hospital trusts.
2. Methods
2.1. Ethics
Consent from individuals involved in this study was not required for this analysis of routine administrative data. Ethical approval was not sought for the present study because it did not directly involve human participants. This study was completed in accordance with the Helsinki Declaration as revised in 2013. The analysis and presentation of data follows current NHS Digital guidance for the use of HES data for research purposes and is anonymised to the level required by ISB1523 Anonymisation Standard for Publishing Health and Social Care Data [7,8].
2.2. Study design and data collection
This was a retrospective analysis of HES data. In England, NHS hospitals are run by trusts which typically serve a geographically defined catchment population. The population of England in mid-2019 was estimated to be 56,286,961 covered by 135 acute non-specialist trusts, and 17 acute specialist trusts [9]. Typically, each trust operates between one and four hospitals. Organisation of services varies widely between trusts. Some have multiple emergency admission sites which operate largely autonomously with regard to service provision, some operate a single site for emergency admissions, with intra-trust transfer as appropriate and others have single treatment escalation sites. As such, we have chosen to consider the trust as the unit of interest with regard to variability assessment.
HES data are collected by NHS Digital for all NHS-funded patients admitted to hospitals in England. The data are collected primarily to allow NHS trusts to be reimbursed for providing hospital care. Data are entered by trained coders in each hospital trust and data collection is mandatory.
2.3. Timing, case ascertainment, inclusion and exclusion criteria
We reviewed HES data for all completed episodes of hospital care in England with a discharge date from March 1st to July 31st 2020 that involved a diagnosis of COVID-19 confirmed by test: International Statistical Classification of Disease and Related Health Problems 10th edition (ICD-10) code U071 (presence of COVID-19 has been confirmed by laboratory testing).
The data collection period was defined in terms of the discharge date (alive or following in-hospital death) rather than the admission date, since our interest was in completed episodes of care, where the outcome (death or discharge) was known. Patients aged < 18 years were excluded. All patients meeting these criteria were included, regardless of the length of stay.
Where a patient had multiple admissions during the study period, only the chronologically last admission was retained. This ensured that all admissions were independent of one another at a patient level. It also avoided biasing the in-hospital mortality data, since only the final admission could result in death; the outcome of earlier admissions being fixed. As such, transferred patients were recorded as under the care of the destination trust. The data extraction process is summarised in Fig. 1.
Fig. 1.
Data extraction process.
2.4. Outcomes
The primary outcome was in-hospital mortality as recorded by the Office for National Statistics (ONS) with COVID-19 as the primary cause of death. An in-hospital death was recorded if the date of death was the same as or +/- one day of the date of hospital discharge recorded in HES.
2.4.1. Secondary outcomes
-
1.
Death with COVID-19 as the primary cause of death during hospital stay or within 30 days of discharge (as recorded by ONS) in patients discharged prior to June 30th, 2020.
-
2.
Length of hospital stay in patients who survived to discharge and who died in hospital.
-
3.
Admission to critical care during hospital stay.
2.5. Covariates
Age: Categorised as 18–39 years, 40–49 years, 50–59 years, 60–69 years, 70–79 years and ≥ 80 years for exploratory analysis and treated as continuous for modelling. The age bands were selected based on previous work in this field by our team and other researchers [10,6].
Sex: Male or female.
Ethnicity: Coded in broad categories to reflect those used by NHS Digital: White, Mixed, Black or Black British, Asian or Asian British (sub-divided into South Asian (Bangladeshi, Indian and Pakistani) and Other Asian), other and not stated.
Deprivation: Recorded using the Index of Multiple Deprivation (IMD) for the Lower Super Output Area (LSOA) of the patients' home address, with scores categorised into quintiles based on national averages.
Comorbidities: These were the 14 comorbidities used to construct the Charlson Comorbidity Index (peripheral vascular disease, congestive heart failure, acute myocardial infarction, cerebrovascular disease, dementia, chronic pulmonary disease, connective tissue disease/rheumatic disease, peptic ulcer, liver disease (mild and moderate/severe), diabetes (with and without chronic complications), paraplegia/hemiplegia, renal disease, cancer (primary and metastatic), HIV/AIDS) [11]. The comorbidity was deemed present if it was recorded in HES as a secondary diagnosis in the index admission or as a primary or secondary diagnosis in any admission during the previous year, in accordance with the recommendations of Quan et al. [12].
Temporal trends: Categorised into day or month of discharge. Monthly categorisation was used for ease of interpretation of tabulated data.
2.6. Grouping variables
Hospital trust: Including specialist hospitals and non-NHS trusts where the patient was NHS-funded.
NHS region: London, South-East, South-West, East of England, Midlands, North-West and North-East & Yorkshire. Patients who were in-patients at a non-NHS Hospital were excluded from this analysis.
NHS hospital trust baseline bed-base: Data on the number of overnight general and acute beds and critical care beds available within each trust were taken from national data produced by NHS England [13]. Overnight general and acute beds data were the average number of beds available for April-June 2020 inclusive. Critical care bed capacity data were taken from 27th February 2020 (the last point such data were published) [14]. For both measures, the trust level data were split into quartiles for analysis. On examination of the data profile, quartiles were considered to give the maximum information possible without overly categorising the data. For the trusts included in this study the acute and general bed-base per trust varied from 185 to 1936 and the critical care bed-base from 0 to 163).
NHS hospital trust proportion of patients admitted to critical care: The proportion of patients admitted to critical care for each trust was calculated and the data categorised into quartiles.
NHS hospital trust median length of stay: The median length of stay for each trust was calculated and the data categorised into quartiles. Where values were identical either side of the quartile threshold the nearest point where length of stay changed was used.
2.7. Statistical analysis
Data were extracted onto a secure encrypted server controlled by NHS England and NHS Improvement. Analysis within this secure environment took place using standard statistical software: Microsoft Excel (Microsoft Corp, Redmond, WA, USA), Stata (StataCorp LLC, College Station, TX, USA) and Alteryx (Alteryx Inc, Irvine, CA, USA).
In descriptive analysis, data were categorised as detailed above and summarised in terms of frequency and percentage. Based on previous work by our team [6], a multilevel (mixed-effects) logistic regression model of patient-related and temporal factors associated with in-hospital mortality was constructed for the current dataset using the 'melogit' command in Stata. A two-level intercept only model was constructed, allowing adjustment for clustering of patients within hospital trusts. The model included the covariates age, day of discharge, IMD deprivation score (all modelled as continuous variables, with restricted cubic splines used as appropriate), ethnicity, sex, obesity and the Charlson Comorbidity Index items listed above. The model is summarised in terms of odds ratios and 95% confidence intervals. A 95% confidence interval not crossing the value 1 was taken as an indication of statistical significance.
Adjusted in-hospital mortality rate for each NHS hospital trust, region and for each quartile of baseline overnight general and acute bed capacity, critical care bed capacity, median length of stay and proportion of patients admitted to critical care were calculated from the model fixed effects using the 'margins' command in Stata. As a sensitivity analysis, adjusted in-hospital mortality rates were also calculated with region and bed-base added as additional covariates to assess the impact they had on our findings. Interaction terms for time (day of discharge) with bed-base and critical care admissions were investigated, but not included in the final model as they did not add significantly to the model beyond the main effects. Marginal values were calculated for the variable in question by adding it to the model as a fixed effect and holding its value constant across all cases and then repeating this for all values of the variable. This allowed the estimation of adjusted in-hospital mortality rates based on the sample distribution of all the covariates within the model. Marginal estimation was preferred over conditional (e.g. at the covariate mean) estimation to better reflect the wider data structure. For this part of the analysis, non-NHS trusts, specialist trusts and trusts with fewer than 40 discharges or 10 deaths during the study period were excluded. Specialist trusts (e.g. non-acute, mental health, community and single-speciality trusts) were identified as those not contributing to the NHS England's daily COVID-19 situation reports [15]. Adjusted in-hospital mortality rates for each trust are presented as a caterpillar plot.
The extent of the variation between trusts is presented as a funnel plot of standardised mortality ratios (SMRs) for each trust. Following modelling for fixed effects only, the predicted in-hospital mortality rate for each trust was calculated using the 'predict’ command in Stata. SMRs were calculated as the ratio of observed to predicted deaths per trust. Funnel plot control limits were created at two and three standard errors from the target (SMR = 1). Standard errors were calculated based on the assumption of a Poisson distribution. Adjustment for over-dispersion due to unmeasured variables (e.g. illness severity) was made using the multiplicative method discussed by Spiegelhalter with winsorisation of the most extreme upper and lower 20% of values based on naïve Z-scores [16]. The SMR for each trust was plotted against expected deaths.
Missing data were rare, and given the large sample size, no attempt was made to impute missing values. Where data were missing the numbers involved are stated. For the covariate ethnicity, a number of patients chose not to state their ethnicity and these data are treated as if missing, although an answer was recorded in all cases. Although ethnicity was the variable with the most missing data, the extent of this was relatively modest compared to other studies of COVID-19 mortality [10].
2.8. Role of the funding source
The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
3. Results
The data extraction process is summarised in Fig. 1. Between 1st March and 31st July 2020, there were 86,356 patients discharged from hospitals in England with a test-confirmed diagnosis of COVID-19. Across the same period, there were 22,975 patients with a diagnosis of COVID-19 on clinical grounds (without test confirmation). There was a trend towards a greater proportion of all cases being clinically diagnosed across the study period (21.6% in March, 16.0% in April, 25.8% in May, 27.5% in June, 33.2% in July).
Of the 86,356 with test-confirmed COVID-19, 26,929 (31.2%) died in hospital of all causes and 22,944 (26.6%) died in hospital with test-confirmed COVID-19 identified as the primary cause of death.
3.1. Variation in crude in-hospital mortality rates and length of stay by NHS England region
For analysis by region, exclusion of nine (n = 237 patients) non-NHS hospital trusts gave a dataset of 86,119 patients across 174 NHS hospital trusts. The demographic profile of patients and their period of discharge are summarised in Table 1. Patients in London were noticeably younger and more likely to be male than in all other regions. London, the Midlands, North-East & Yorkshire and North-West had the highest proportion of patients in the most deprived quintile. London and the Midlands saw the most ethnically diverse patients, with just over half of all patients in London being from the White ethnic group. In all regions the greatest number of discharges for patients with confirmed SARS-CoV2 infection was during April.
Table 1.
COVID-19 patient characteristics across NHS regions in England.
| Region | London | South East | South West | East of England | Midlands | North East and Yorkshire | North West |
|---|---|---|---|---|---|---|---|
| Number of patients | 18,667 | 10,333 | 4148 | 9071 | 17,317 | 13,042 | 13,541 |
| Age band (years) | |||||||
| 18–39 | 1702 (9.1%) | 603 (5.8%) | 264 (6.4%) | 487 (5.4%) | 1119 (6.5%) | 805 (6.2%) | 767 (5.7%) |
| 40–49 | 1735 (9.3%) | 749 (7.2%) | 297 (7.2%) | 639 (7.0%) | 1200 (6.9%) | 770 (5.9%) | 869 (6.4%) |
| 50–59 | 3056 (16.4%) | 1269 (12.3%) | 537 (12.9%) | 987 (10.9%) | 1938 (11.2%) | 1509 (11.6%) | 1629 (12.0%) |
| 60–69 | 3224 (17.3%) | 1432 (13.9%) | 603 (14.5%) | 1285 (14.2%) | 2475 (14.3%) | 1823 (14.0%) | 1972 (14.6%) |
| 70–79 | 3645 (19.5%) | 2187 (21.2%) | 912 (22.0%) | 1931 (21.3%) | 3857 (22.3%) | 2964 (22.7%) | 3099 (22.9%) |
| ≥ 80 | 5305 (28.4%) | 4093 (39.6%) | 1535 (37.0%) | 3742 (41.3%) | 6728 (38.9%) | 5171 (39.6%) | 5205 (38.4%) |
| Sex | |||||||
| Female | 7727 (41.4%) | 4608 (44.6%) | 1795 (43.3%) | 3983 (43.9%) | 7848 (45.3%) | 6099 (46.8%) | 6160 (45.5%) |
| Male | 10,940 (58.6%) | 5725 (55.4%) | 2353 (56.7%) | 5088 (56.1%) | 9469 (54.7%) | 6943 (53.2%) | 7381 (54.5%) |
| Deprivation quintile (missing = 1709) | |||||||
| 1 (most deprived) | 4716 (26.3%) | 956 (9.4%) | 529 (12.9%) | 1165 (13.1%) | 5130 (30.1%) | 4515 (34.8%) | 4752 (35.6%) |
| 2 | 5586 (31.1%) | 1741 (17.2%) | 779 (18.9%) | 1808 (20.3%) | 3539 (20.8%) | 2845 (22.0%) | 2733 (20.5%) |
| 3 | 3575 (19.9%) | 2122 (20.9%) | 1012 (24.6%) | 2182 (24.5%) | 3148 (18.5%) | 2124 (16.4%) | 2020 (15.1%) |
| 4 | 2328 (13.0%) | 2340 (23.1%) | 928 (22.6%) | 1936 (21.7%) | 2961 (17.4%) | 2014 (15.5%) | 2096 (15.7%) |
| 5 (least deprived) | 1748 (9.7%) | 2984 (29.4%) | 863 (21.0%) | 1827 (20.5%) | 2272 (13.3%) | 1461 (11.3%) | 1740 (13.0%) |
| Ethnicity (not stated = 10,137) | |||||||
| White | 8119 (51.7%) | 7798 (87.1%) | 3315 (93.5%) | 6954 (87.9%) | 12,716 (82.6%) | 10,929 (91.1%) | 11,420 (90.5%) |
| Black or Black British | 2826 (18.0%) | 202 (2.3%) | 73 (2.1%) | 271 (3.4%) | 622 (4.0%) | 177 (1.5%) | 235 (1.9%) |
| South Asian | 1754 (11.2%) | 326 (3.6%) | 42 (1.2%) | 333 (4.2%) | 1441 (9.4%) | 561 (4.7%) | 572 (4.5%) |
| Other Asian | 1048 (6.7%) | 254 (2.8%) | 48 (1.4%) | 124 (1.6%) | 203 (1.3%) | 98 (0.8%) | 109 (0.9%) |
| Mixed | 170 (1.1%) | 74 (0.8%) | 21 (0.6%) | 63 (0.8%) | 90 (0.6%) | 45 (0.4%) | 61 (0.5%) |
| Other | 1773 (11.3%) | 294 (3.3%) | 48 (1.4%) | 162 (2.0%) | 327 (2.1%) | 184 (1.5%) | 217 (1.7%) |
| Month of discharge | |||||||
| March | 3169 (17.0%) | 858 (8.3%) | 354 (8.5%) | 706 (7.8%) | 1740 (10.0%) | 712 (5.5%) | 740 (5.5%) |
| April | 11,598 (62.1%) | 5509 (53.3%) | 2273 (54.8%) | 4627 (51.0%) | 9030 (52.1%) | 6787 (52.0%) | 6908 (51.0%) |
| May | 2611 (14.0%) | 2468 (23.9%) | 1080 (26.0%) | 2393 (26.4%) | 3900 (22.5%) | 3610 (27.7%) | 3571 (26.4%) |
| June | 899 (4.8%) | 1047 (10.1%) | 347 (8.4%) | 979 (10.8%) | 1864 (10.8%) | 1397 (10.7%) | 1659 (12.3%) |
| July | 390 (2.1%) | 451 (4.4%) | 94 (2.3%) | 366 (4.0%) | 783 (4.5%) | 536 (4.1%) | 663 (4.9%) |
Two-hundred and thirty-seven NHS-funded patients were at a non-NHS provider and are excluded from this analysis.
Table 2 summarises patient outcomes for each region. Crude in-hospital mortality rates were highest in the East of England and lowest in the South West and London, partly reflecting the age structure of admitted patients. Fig. 2 summarises the in-hospital mortality rate over time for each region. The trend in each region was remarkably similar, with all trusts showing a steady decline in-hospital mortality rates over time. In-hospital mortality rates by region and age band are presented in Fig. 3. Using this categorisation, the difference in-hospital mortality rates between regions seen in Table 2 was much less obvious. However, across the older age bands, where most deaths were concentrated, the East of England, London and the North-West had the highest in-hospital mortality rates and the South West the lowest in-hospital mortality rate.
Table 2.
COVID-19 patient outcomes across NHS regions in England.
| Region | London | South East | South West | East of England | Midlands | North East and Yorkshire | North West |
|---|---|---|---|---|---|---|---|
| All patients | |||||||
| Number of patients | 18,667 | 10,333 | 4148 | 9071 | 17,317 | 13,042 | 13,541 |
| In-hospital mortality | 4710 (25.2%) | 2655 (25.7%) | 940 (22.7%) | 2710 (29.9%) | 4694 (27.1%) | 3488 (26.7%) | 3710 (27.4%) |
| Median length of stay in patients who survived to discharge | 8 (4 to 16) | 9 (4 to 19) | 8 (3 to 18) | 8 (3 to 16) | 8 (4 to 17) | 8 (3 to 16) | 9 (4 to 21) |
| Median length of stay in patients who died in hospital | 7 (4 to 14) | 9 (4 to 17) | 8 (4 to 16) | 7 (4 to 15) | 8 (4 to 15) | 7 (4 to 14) | 9 (4 to 18) |
| Admitted to critical care during hospital stay | 2759 (14.8%) | 1080 (10.5%) | 388 (9.4%) | 985 (10.9%) | 1598 (9.2%) | 1173 (9.0%) | 1225 (9.0%) |
| Mortality if admitted to critical care | 1025 (37.2%) | 365 (33.8%) | 112 (28.9%) | 414 (42.0%) | 606 (37.9%) | 447 (38.1%) | 465 (38.0%) |
| Patients discharged up to 30th June 2020 | |||||||
| Number of patients | 18,277 | 9882 | 4054 | 8705 | 16,534 | 12,506 | 12,878 |
| In-hospital mortality | 4698 (25.7%) | 2592 (26.2%) | 937 (23.1%) | 2678 (30.8%) | 4611 (27.9%) | 3433 (27.5%) | 3637 (28.2%) |
| Death in hospital or within 30 days of hospital discharge | 5010 (27.4%) | 2844 (28.8%) | 1024 (25.3%) | 2928 (33.6%) | 5073 (30.7%) | 3817 (30.5%) | 3932 (30.5%) |
*Two-hundred and thirty-seven NHS-funded patients were at a non-NHS provider and are excluded from this analysis of the full dataset. Two-hundred and twenty-five NHS-funded patients were at a non-NHS provider and are excluded from this analysis of the March-June dataset.
Fig. 2.
Variation in crude in-hospital mortality rates by fortnight of discharge for each region.
Fig. 3.
Variation in crude in-hospital mortality rates by age band for each region.
Length of stay was similar between regions, with some evidence of longer stay in those that survived to discharge. In those discharged on or before 30th June 2020 (n = 83,061), there were 22,623 (27.2%) deaths in hospital and a further 2047 (2.5%) deaths within 30 days of discharge. London had the highest proportion of patients admitted to critical care (14.8%), with all other regions admitting between 9% and 11% of patients to critical care.
3.2. Adjusted in-hospital mortality rates for regions and trusts and by trust baseline bed-base, critical care admissions and length of stay
In multilevel logistic regression modelling, the intra-class correlation coefficient was 0.5% for region and 2.9% for trust, suggesting very little clustering of deaths by region or by trust. The model with adjustment for clustering by trust is summarised in Table 3. Based on a fixed effects model, adjusted in-hospital mortality rates by region are presented in Table 4. The South West had the lowest adjusted in-hospital mortality rate, followed by London, whilst the East of England had the highest in-hospital mortality rate, followed by the North West. Table 4 presents data for the adjusted in-hospital mortality rate by quartile of baseline overnight general and acute beds and critical care beds. There was a trend towards lower in-hospital mortality rates at trusts with a larger baseline bed-base according to both measures. Trusts with a higher proportion of admissions to critical care also had lower adjusted in-hospital mortality rates than trusts with lower critical care admission rates. There was no evidence of an association between median length of stay and adjusted in-hospital mortality rates.
Table 3.
Multilevel logistic regression model predicting in-hospital mortality.
| Variable | Odds ratios (95% CIs) | Significance (p-value) |
|---|---|---|
| Age | ||
| Spline 1 | 1.073 (1.070 to 1.077) | < 0.001 |
| Spline 2 | 0.976 (0.973 to 0.979) | < 0.001 |
| Day of discharge | ||
| Spline 1 | 0.985 (0.982 to 0.988) | < 0.001 |
| Spline 2 | 0.889 (0.868 to 0.911) | < 0.001 |
| Spline 3 | 1.289 (1.227 to 1.354) | < 0.001 |
| Sex | ||
| Female | 1 (reference) | |
| Male | 1.469 (1.416 to 1.523) | < 0.001 |
| IMD score | 1.002 (1.000 to 1.003) | 0.010 |
| Ethnicity | ||
| White | 1 (reference) | |
| Black | 1.018 (0.931 to 1.114) | 0.696 |
| South Asian | 1.291 (1.188 to 1.403) | < 0.001 |
| Other Asian | 1.111 (0.967 to 1.276) | 0.136 |
| Mixed | 1.321 (1.033 to 1.689) | 0.026 |
| Other | 1.041 (0.933 to 1.162) | 0.470 |
| Charlson Comorbidity Index items* | ||
| Peripheral vascular disease | 1.150 (1.070 to 1.235) | < 0.001 |
| Congestive heart failure | 1.515 (1.446 to 1.588) | < 0.001 |
| Acute myocardial infarction | 1.044 (0.987 to 1.105) | 0.131 |
| Cerebrovascular disease | 1.006 (0.948 to 1.068) | 0.834 |
| Dementia | 1.371 (1.309 to 1.434) | < 0.001 |
| Chronic pulmonary disease | 1.106 (1.063 to 1.151) | < 0.001 |
| Connective tissue disease/rheumatic disease | 1.256 (1.142 to 1.381) | < 0.001 |
| Peptic ulcer | 1.083 (0.883 to 1.328) | 0.445 |
| Mild liver disease | 0.981 (0.878 to 1.097) | 0.736 |
| Moderate or severe liver disease | 2.765 (2.338 to 3.269) | < 0.001 |
| Diabetes without chronic complications | 1.150 (1.105 to 1.198) | < 0.001 |
| Diabetes with chronic complications | 1.284 (1.167 to 1.414) | < 0.001 |
| Paraplegia and hemiplegia | 1.076 (1.015 to 1.139) | 0.013 |
| Renal disease | 1.141 (1.117 to 1.166) | < 0.001 |
| Primary cancer | 1.459 (1.364 to 1.561) | < 0.001 |
| Metastatic carcinoma | 1.262 (1.138 to 1.399) | < 0.001 |
| Obesity | 1.535 (1.433 to 1.643) | < 0.001 |
Models are based on data for 74,819 patients with no missing data. A stable odds ratio for the comorbidity HIV/AIDS could not be calculated due to small numbers.
For Charlson Comorbidity Index items the reference category is patients without the specified comorbidity. For Charlson Comorbidity Index items relating to liver disease, diabetes and cancer three mutually exclusive categories were used. IMD = Index of Multiple Deprivation. CI = Confidence interval. A 95% CI not crossing the value 1 was taken as an indication of statistical significance.
Table 4.
Adjusted in-hospital mortality rates by region, by trust baseline bed-base, by length of stay and by proportion of patients admitted to critical care.
| Adjusted in-hospital mortality rate (95% CI) | |
|---|---|
| Region | |
| London | 25.6% (24.9 to 26.3) |
| South East | 27.2% (26.3 to 28.0) |
| South West | 23.6% (22.3 to 24.9) |
| East of England | 31.4% (30.5 to 32.4) |
| Midlands | 27.0% (26.4 to 27.7) |
| North East and Yorkshire | 28.0% (27.2 to 28.7) |
| North West | 28.8% (28.1 to 29.6) |
| Average number of overnight general and acute beds (April–June, 2020) | |
| 1st Quartile (185 to 436) | 29.6% (28.7 to 30.4) |
| 2nd Quartile (444 to 623) | 28.6% (27.9 to 29.2) |
| 3rd Quartile (636 to 866) | 27.3% (26.6 to 27.9) |
| 4th Quartile (887 to 1936) | 27.3% (26.9 to 27.8) |
| Number of critical care beds (27th February 2020) | |
| 1st Quartile (0 to 12) | 30.9% (30.1 to 31.7) |
| 2nd Quartile (13 to 18) | 28.9% (28.2 to 29.5) |
| 3rd Quartile (19 to 33) | 28.1% (27.5 to 28.6) |
| 4th Quartile (34 to 163) | 25.9% (25.5 to 26.4) |
| Proportion of patients admitted to critical care (%) | |
| 1st Quartile (0 to 7.3) | 29.9% (29.3 to 30.6) |
| 2nd Quartile (7.4 to 9.9) | 29.4% (28.7 to 30.1) |
| 3rd Quartile (10.0 to 11.9) | 26.4% (25.8 to 27.0) |
| 4th Quartile (12.0 to 28.8) | 26.4% (25.8 to 26.9) |
| Median length of stay (days) | |
| 1st Quartile (0 to 6.5) | 28.3% (27.4 to 29.3) |
| 2nd Quartile (7) | 27.5% (27.0 to 28.1) |
| 3rd Quartile (8) | 27.7% (27.2 to 28.2) |
| 4th Quartile (9–15) | 28.0% (27.4 to 28.6) |
In-hospital mortality is adjusted for age, sex, deprivation, ethnicity, date of discharge and co-morbidity. The figures in brackets indicate the data range for each quartile. CI = Confidence interval.
In sensitivity analysis the same analysis was conducted with the additional covariates of general and acute bed-base (in the model of adjusted in-hospital mortality rates by region) and region in the other models. The data are presented in Table 5 and are very similar to the data presented in Table 4 suggesting very little evidence that region or bed-base had a notable confounding effect on the results presented.
Table 5.
Sensitivity analysis of adjusted in-hospital mortality rates by region, by trust baseline bed-base, by length of stay and by proportion of patients admitted to critical care.
| Adjusted in-hospital mortality rate (95% CI) | |
|---|---|
| Region* | |
| London | 25.9% (25.2 to 26.6) |
| South East | 27.8% (26.9 to 28.7) |
| South West | 24.1% (22.7 to 25.5) |
| East of England | 31.6% (30.7 to 32.6) |
| Midlands | 27.8% (27.2 to 28.5) |
| North East and Yorkshire | 28.0% (27.3 to 28.8) |
| North West | 28.8% (28.1 to 29.6) |
| Average number of overnight general and acute beds (April-June, 2020)⁎⁎ | |
| 1st Quartile (185 to 436) | 29.7% (28.8 to 30.5) |
| 2nd Quartile (444 to 623) | 28.5% (27.8 to 29.2) |
| 3rd Quartile (636 to 866) | 27.3% (26.7 to 28.0) |
| 4th Quartile (887 to 1936) | 27.3% (26.8 to 27.7) |
| Number of critical care beds (27th February, 2020)⁎⁎ | |
| 1st Quartile (0 to 12) | 31.0% (30.2 to 31.8) |
| 2nd Quartile (13 to 18) | 28.5% (27.9 to 29.2) |
| 3rd Quartile (19 to 33) | 27.8% (27.2 to 28.4) |
| 4th Quartile (34 to 163) | 26.3% (25.8 to 26.8) |
| Proportion of patients admitted to critical care (%)⁎⁎ | |
| 1st Quartile (0 to 7.3) | 30.0% (29.4 to 30.7) |
| 2nd Quartile (7.4 to 9.9) | 29.5% (28.8 to 30.2) |
| 3rd Quartile (10.0 to 11.9) | 26.5% (25.9 to 27.0) |
| 4th Quartile (12.0 to 28.8) | 26.2% (25.6 to 26.8) |
| Median length of stay (days)⁎⁎ | |
| 1st Quartile (0 to 6.5) | 28.5% (27.5 to 29.4) |
| 2nd Quartile (7) | 27.4% (26.8 to 27.9) |
| 3rd Quartile (8) | 27.6% (27.1 to 28.2) |
| 4th Quartile (9–15) | 28.2% (27.6 to 28.8) |
In-hospital mortality is adjusted for age, sex, deprivation, ethnicity, date of discharge and co-morbidity and quartile of general and acute beds.
In-hospital mortality is adjusted for age, sex, deprivation, ethnicity, date of discharge and co-morbidity and region. CI = Confidence interval.
Adjusted in-hospital mortality rates for the 126 non-specialist NHS trust that had sufficient activity to be include in the analysis are summarised in Fig. 4. The variation in SMRs across trusts is summarised as a funnel plot in Fig. 5. No SMRs fell outside the upper outer control limit. Two trusts had SMRs below the lower outer control limit, suggesting relatively good outcomes, although both were very close to the limit.
Fig. 4.
Adjusted probability of in hospital mortality for each trust.
Fig. 5.
Funnel plot of variation in standardised in-hospital mortality ratio by NHS hospital trust.
4. Discussion
We present data for all hospital activity involving test-confirmed COVID-19 in England over the first five months of the pandemic. To our knowledge, this is the first study examining regional and hospital trust level outcomes for the patient population in England. Although there will have been differences in the approaches utilised by trusts during the pandemic, variation in-hospital mortality rates between NHS regions and trusts was relatively modest.
The rate of COVID-19 infections has been variable both temporally and spatially in England, with London, the Midlands and the North West regions of the country experiencing a higher overall burden earlier in the pandemic [17,18]. These differences in COVID-19 activity will have resulted in an asymmetrical impact on hospital services [15]. This is evident from our data on the numbers of hospitalised COVID-19 cases and the proportion of deaths in each region. Difference in timing of case surges may have meant that regions affected later in the pandemic benefitted from the learning shared by those regions that experienced high COVID-19 activity in March and early April. Furthermore, differences in patient case-mix will have influenced outcomes especially in regions with older and/or frailer patient cohorts, such as the East of England. However, after adjusting for patient factors, there was limited inter-regional variation in COVID-19-related in-hospital mortality rates. A possible exception is the South West of England, which experienced relatively low COVID-19 activity compared to other regions. Encouragingly, reductions in-hospital mortality rates as the pandemic progressed followed a similar trend in all regions.
A cross-sectional observational study from Brazil reported substantial regional variation in-hospital mortality from COVID-192 which has been supported by a further study using data from Brazil's Unified Health System [19]. The authors of both studies speculate that this may be due to higher comorbidity burden and reduced access to healthcare, notably critical care, in lower socio-economic groups in the Northern regions of Brazil. Although access to healthcare in England may vary with socio-economic group, ethnicity and region, it is unlikely to be of the same magnitude as in countries such as Brazil where the divide between private and public healthcare is more pronounced.
One of the main findings from our study was that, after adjustment for other covariates, there was limited differences in COVID-19 related in-hospital mortality rates between NHS hospital trusts. Our study covers the initial case-surge in all regions of England and a period of relatively low COVID-19 hospital activity during June and July. By studying this five-month period, inter-trust differences in-hospital mortality rates due to differential infection rates should be lower than if we had focussed only on the peaked period of March-April. The role of national guidance and rapid learning within regional networks as new information became available may have helped in producing a relatively consistent picture across England.
There has been a paucity of published studies comparing COVID-19 outcomes between hospitals. In the UK, there are multiple sources of COVID-19 related in-hospital mortality information. However, these have undergone changes in structure and format as the pandemic has progressed [3]. Since April 2020, NHS England has produced daily reports of the number of COVID-19 related hospital deaths at national, regional and trust levels. Information is acquired through a direct reporting system called the COVID-19 Patient Notification System (CPNS) which aims to record deaths as close as possible to the time they occur [20]. However, these reports present crude numbers of deaths without the context of other COVID-19 hospital activity.
A previous study of between-centre COVID-19 in-hospital mortality in the intensive care unit (ICU) patient population in England found that the magnitude of the between-centre variation was comparable to the strongest fixed effects predictors (immunosuppressive disease, chronic cardiorespiratory/renal disease) [4]. This analysis utilised de-identified COVID-19 Hospitalisation in England Surveillance System (CHESS) data [21] from Public Health England from 94 NHS trusts across England and the period covered was the peak of the pandemic. Furthermore, whereas HES should include all hospital discharges, the frequency of reports from hospital trusts for CHESS was variable [22]. Likewise, mortality rates in critical care settings may be dependant on how critical care services within regional networks are organised, with some trusts acting as regional centres and likely to see a higher proportion of patients at high risk of poor outcomes. For this reason, we have not presented trust level data for critical care admission or critical care-specific mortality rates. Since all transfers should be within a region, regional level critical care data are presented and should be informative.
In a study of COVID-19 mortality rates in the USA using a large health insurer database, Asch and colleagues report significant variation in risk-standardised event rates (30 day mortality or referral to a hospice) between 955 healthcare institutions during January–June 2020 [23]. Although they did not find an association between outcomes and hospital resources (i.e. critical care beds, academic status), hospitals with higher surrounding community case rates had higher risk-standardised event rates. They speculate that the differences may represent variation in quality of care [24] but accept that some of this variation may reflect differences in hospital admission thresholds, which is also true of our study. Furthermore, the majority of hospitals in their study (557 (58.3%)) had fewer than 10 COVID-19 patients in a given 3 month period (January 1–April 30 and/or May 1–June 30) and therefore these low patient numbers may account for some variation.
Although much of the differences in-hospital mortality rates between trusts in our study are likely to be due to random variation, there are a number of factors not accounted for in our analysis that could explain some of the observed variation. These include disease severity at admission and organisational factors related to delivery of care (e.g. critical care provision within a trust). Greater illness severity at presentation is thought to be associated with poorer outcomes in patients with COVID-19 [25,26]. However, illness severity at presentation is likely to have varied over time and our analysis is one of very few to adjust for temporal trends. Early in the pandemic, when awareness of the disease was low and community transmission rates were high, patients are likely to have been more severely ill at presentation. Geographical factors including at risk population density surrounding a hospital [27], and proximity of neighbouring acute hospitals will also have influenced COVID-19 patient flow and potential for patient surges.
Interestingly, there were few deaths in the 30-day period immediately following discharge. A proportion of this group of patients who died following discharge may include patient discharged as part of palliative care strategies. The practice of individual trusts in this regard, will also have influenced in-hospital mortality rates.
Although the effect was modest, trusts with the largest bed-base had the lowest in-hospital mortality rates. This was most evident when considering the baseline critical care bed capacity than the acute and general bed capacity. Likewise, trusts with the highest proportion of patients admitted to critical care also had the lowest in-hospital mortality rates. Hospitals throughout England increased critical care capacity rapidly but were constrained by restrictions in baseline physical resources, equipment, environment and the workforce skill-mix and flexibility. Mirroring our findings, a study of inter-hospital variation in treatment and outcomes of COVID-19 patients in the USA found that patients who were admitted to hospitals with the fewest ICU beds had a higher risk of death [28]. As would be expected, hospital trusts with a greater baseline resource appear to be more likely to be able to cope with case-surges and so retain a clinically appropriate threshold for admission to critical care. Nevertheless, the relationship between mortality and bed-base is likely to be a complicated one. Admissions policies, strain on services, clinical practice and treatment options and illness severity are likely to have varied over time and between trusts.
HES data provides a complete record of all COVID-19 related hospital activity in England. However, there are inherent limitations in using HES data due to the reliance on hospitals completing administrative information accurately. Although HES data are entered by trained coders, who are independent of clinicians, they rely in patient notes for information and only if this is recorded accurately will HES data be reliable. HES provides only limited clinical information, and no record of how acutely unwell a patient was during their stay. Furthermore, coding practices in relation to COVID-19 may have changed as the pandemic progressed. To mitigate against this, we have limited identification of COVID-19 patients to ICD-10 code U071 where the presence of COVID-19 has been confirmed by laboratory testing. We were concerned that a lack of clear guidance on where to use the ICD-10 code U072 (diagnosis of COVID-19 based on clinical signs and symptoms) early in the pandemic could have biased our findings. However, in doing so we have under-reported COVID-19 hospital activity. We acknowledge that for some of the patients included in our dataset, COVID-19 will not have been the reason for admission or a major complication. This would be more of a problem later in our data collection period, where subsequent admissions unrelated to COVID-19 may still contain the code for COVID-19. However, we restricted deaths to patients where test-confirmed COVID-19 was the primary cause of death. Although we excluded a small number of patients who were still in hospital on 31st July 2020, we have no reason to suspect that they were in any way unrepresentative of all COVID-19 patients and that this would have biased our findings. Finally, we used critical care bed-base data from late February. We are aware that many trusts were able to rapidly increase their critical care bed-base. Nevertheless, human, equipment and infrastructure resource constraints will have limited the extent of this.
We found little evidence of substantial inter-hospital variation in COVID-19 patient outcomes in England. However, hospitals with greater baseline resources have lower rates of COVID-19 deaths. There are opportunities to learn from the experience of individual trusts and regional networks to help prepare the NHS for clinical management of COVID-19 patients during future case-surges. The GIRFT programme has used these data to engage hospital trusts in a cross-speciality learning programme to understand the clinical and strategic approaches that were successful in the management of COVID-19 patients during the 1st wave [29]. Comprehending the experience of staff and wider systems at a region, trusts and hospitals level during the first case-surge of the pandemic will be important in ensuring that lessons are learned.
Contributorship
This study was designed and organised by AN, WKG, JD and TWRB. Data cleaning, analysis was by WKG, supported by JD and AN. Writing of the first draft was by AN and WKG. All authors critically reviewed the manuscript and agreed to submission of the final draft.
Data sharing
Request for any underlying data cannot be granted by the authors as HES data were obtained and analysed under data sharing agreement from NHS Digital where conditions of use (and further use) apply. HES data can be obtained by direct application to NHS Digital.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Declaration of Competing Interest
The authors declare that there is no conflict of interest.
Acknowledgments
We acknowledge NHS Digital for permission to use their data in this report. The GIRFT programme is providing a framework for examining contemporary clinical practice in unprecedented detail and breadth. We also thank all staff within individual NHS trusts who collected and entered the data used in this study.
References
- 1.Xiong Y., Wang Y., Chen F., Zhu M. Spatial statistics and influencing factors of the COVID-19 epidemic at both prefecture and county levels in Hubei Province, China. Int J Environ Res Public Health. 2020;17 doi: 10.3390/ijerph17113903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Baqui P., Bica I., Marra V., Ercole A., van der Schaar M. Ethnic and regional variations in hospital mortality from COVID-19 in Brazil: a cross-sectional observational study. Lancet Glob Health. 2020 doi: 10.1016/S2214-109X(20)30285-0. published online Aug 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Mortality data review - NHS Digital. https://digital.nhs.uk/coronavirus/coronavirus-data-services-updates/mortality-data-review#download-the-mortality-data-review (accessed Aug 9, 2020).
- 4.Qian Z., Alaa A.M., van der Schaar M., Ercole A. Between-centre differences for COVID-19 ICU mortality from early data in England. Intensive Care Med. 2020 doi: 10.1007/s00134-020-06150-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Barratt H., Turner S., Hutchings A. Mixed methods evaluation of the Getting it Right First Time programme - improvements to NHS orthopaedic care in England: study protocol. BMC Health Serv Res. 2017;17:1–8. doi: 10.1186/s12913-017-2012-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Navaratnam A.V., Gray W.K., Day J., Wendon J., Briggs T.W.R. Patient factors and temporal trends associated with COVID-19 in-hospital mortality in England: an observational study using administrative data. Lancet Respir Med. 2021 doi: 10.1016/S2213-2600(20)30579-8. published online Feb 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.NHS Digital. Hospital episode statistics (HES) analysis guide. London, UK, 2018.
- 8.NHS Digital. Information Standard Board for Health and Social Care. Anonymisation Standard for Publishing Health and Social Care Data Specification (Process Standard). London, UK, 2013.
- 9.People, population and community - Office for National Statistics. https://www.ons.gov.uk/peoplepopulationandcommunity (accessed March 16, 2021 ).
- 10.Williamson E.J., Walker A.J., Bhaskaran K. OpenSAFELY: factors associated with COVID-19 death in 17 million patients. Nature. 2020 doi: 10.1038/s41586-020-2521-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Charlson M.E., Pompei P., Ales K.L., MacKenzie C.R. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 12.Quan H., Li B., Couris C.M. Updating and validating the charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173:676–682. doi: 10.1093/aje/kwq433. [DOI] [PubMed] [Google Scholar]
- 13.Statistics» Statistical work areas. https://www.england.nhs.uk/statistics/statistical-work-areas/ (accessed Oct 10, 2020 ).
- 14.Statistics» Critical care bed capacity and urgent operations cancelled 2019-20 data. https://www.england.nhs.uk/statistics/statistical-work-areas/critical-care-capacity/critical-care-bed-capacity-and-urgent-operations-cancelled-2019-20-data/ (accessed Sept 12, 2020 ).
- 15.Statistics» COVID-19 hospital activity. https://www.england.nhs.uk/statistics/statistical-work-areas/covid-19-hospital-activity/ (accessed Aug 9, 2020 ).
- 16.Spiegelhalter D.J. Funnel plots for comparing institutional performance. Stat Med. 2005;24:1185–1202. doi: 10.1002/sim.1970. [DOI] [PubMed] [Google Scholar]
- 17.Coronavirus (COVID-19) in the UK: Cases. https://coronavirus.data.gov.uk/cases (accessed Aug 9, 2020 ).
- 18.No Title. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/908434/Disparities_in_the_risk_and_outcomes_of_COVID_August_2020_update.pdf (accessed Aug 19, 2020 ).
- 19.de Andrade C.L.T., de Aguiar Pereira C.C., Martins M., Lima S.M.L., Portela M.C. COVID-19 hospitalizations in Brazil's unified health system (SUS) PLoS ONE. 2020;15 doi: 10.1371/journal.pone.0243126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Statistics» COVID-19 Daily Deaths. https://www.england.nhs.uk/statistics/statistical-work-areas/covid-19-daily-deaths/ (accessed Oct 11, 2020 ).
- 21.SGSS and CHESS data - NHS Digital. https://digital.nhs.uk/about-nhs-digital/corporate-information-and-documents/directions-and-data-provision-notices/data-provision-notices-dpns/sgss-and-chess-data (accessed Oct 11, 2020 ).
- 22.No Title. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/920373/Weekly_COVID19_Surveillance_Report_week_36_UPDATED.pdf (accessed Nov 1, 2020 ).
- 23.Asch D.A., Sheils N.E., Islam M.N. Variation in US hospital mortality rates for patients admitted with COVID-19 during the first 6 months of the pandemic. JAMA Intern Med. 2020 doi: 10.1001/jamainternmed.2020.8193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Jha A.K., Li Z., Orav E.J., Epstein A.M. Care in U.S. hospitals — the hospital quality alliance program. N Engl J Med. 2005;353:265–274. doi: 10.1056/NEJMsa051249. [DOI] [PubMed] [Google Scholar]
- 25.Knight S.R., Ho A., Pius R. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO clinical characterisation protocol: development and validation of the 4C mortality score. BMJ. 2020;370 doi: 10.1136/bmj.m3339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Gupta R.K., Harrison E.M., Ho A. Development and validation of the ISARIC 4C deterioration model for adults hospitalised with COVID-19: a prospective cohort study. Lancet Respir Med. 2021 doi: 10.1016/S2213-2600(20)30559-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Nicodemo C., Barzin S., Cavalli N. Measuring geographical disparities in England at the time of COVID-19: results using a composite indicator of population vulnerability. BMJ Open. 2020;10 doi: 10.1136/bmjopen-2020-039749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gupta S., Hayek S.S., Wang W. Factors associated with death in critically Ill patients with coronavirus disease 2019 in the US. JAMA Intern Med. 2020 doi: 10.1001/jamainternmed.2020.3596. published online July 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Getting it Right First Time programme. Clinical practice guide for improving the management of adult COVID-19 patients in secondary care. 2020. https://www.gettingitrightfirsttime.co.uk/wp-content/uploads/2020/12/Covid19-Clinical-Practice-Guidance-S-FINAL.pdf (accessed Jan 22, 2021).