The impact of remote home monitoring of people with COVID-19 using pulse oximetry: A national population and observational study

Summary

Background

Remote home monitoring of people testing positive for COVID-19 using pulse oximetry was implemented across England during the Winter of 2020/21 to identify falling blood oxygen saturation levels at an early stage. This was hypothesised to enable earlier hospital admission, reduce the need for intensive care and improve survival. This study is an evaluation of the clinical effectiveness of the pre-hospital monitoring programme, COVID oximetry @home (CO@h).

Methods

The setting was all Clinical Commissioning Group (CCG) areas in England where there were complete data on the number of people enrolled onto the programme between 2nd November 2020 and 21st February 2021. We analysed relationships at a geographical area level between the extent to which people aged 65 or over were enrolled onto the programme and outcomes over the period between November 2020 to February 2021.

Findings

For every 10% increase in coverage of the programme, mortality was reduced by 2% (95% confidence interval:4% reduction to 1% increase), admissions increased by 3% (-1% to 7%), in-hospital mortality fell by 3% (-8% to 3%) and lengths of stay increased by 1·8% (-1·2% to 4·9%). None of these results are statistically significant, although the confidence interval indicates that any adverse effect on mortality would be small, but a mortality reduction of up to 4% may have resulted from the programme.

Interpretation

There are several possible explanations for our findings. One is that CO@h did not have the hypothesised impact. Another is that the low rates of enrolment and incomplete data in many areas reduced the chances of detecting any impact that may have existed. Also, CO@h has been implemented in many different ways across the country and these may have had varying levels of effect.

Funding

This is independent research funded by the National Institute for Health Research, Health Services & Delivery Research programme (RSET Project no. 16/138/17; BRACE Project no. 16/138/31) and NHSEI. NJF is an NIHR Senior Investigator.

Research in Context.

Evidence before this study

Existing evidence before this study and the search strategy used to obtain this evidence has been published previously by the authors in a systematic review. We searched MEDLINE, CINAHL PLUS, EMBASE, TRIP, medRxiv and Web of Science for articles and preprints from January 2020 to February 2021. Papers were selected if they focussed on the monitoring of confirmed or suspected patients with COVID-19. The search algorithm used combinations of the following terms: "COVID-19″, "severe acute respiratory syndrome coronavirus", “2019-nCoV", "SARS-CoV-2″, “Wuhan”, “coronavirus”, “virtual ward”, “remote monitoring”, “virtual monitoring”, “home monitoring”, “community monitoring”, “early monitoring,” “remote patient monitoring”, “pre-hospital monitoring”, “Covidom”, “My m health”, “GetWell Loop” “silent hypoxaemia” “pulse oximetry”. Previous quantitative studies have assessed remote oximetry monitoring services for COVID-19 patients mostly at individual sites and focussed on their safety. However, their effectiveness has been little studied. This may reflect the challenges of identifying reliable counterfactuals during a rapidly evolving pandemic.

Added value of this study

This study is part of a wider mixed methods evaluation that followed the rapid implementation of remote monitoring across the English NHS during the Winter of 2020/21. Previous studies have evaluated remote monitoring of COVID-19 patients using oximetry at a local level, some being targeted at people of particularly high risk. This study adds evidence towards the performance of such programmes at a national level.

Implications of all the available evidence

There is some existing evidence that remote monitoring of COVID-19 patients can be locally effective although we have not been able to replicate such findings at a wider level. Missing data and lower coverage of the service than expected may have influenced our results, and the effectiveness of some local programmes could have been lost amongst the analysis of national data. Future implementation requires better data collection strategies which could be focussed within fewer local areas, and effective learning from areas that have achieved better population coverage.

Alt-text: Unlabelled box

Introduction

During the early months of the COVID-19 pandemic, many patients with COVID-19 were admitted to hospital having deteriorated several days after they were first diagnosed. Many of these patients had “silent hypoxia” (low blood oxygen saturation levels without typical symptoms or awareness) and, once at hospital, often required intensive treatment with a high risk of mortality.¹ This motivated health services to try and detect such cases at an earlier stage by monitoring blood oxygen levels in people diagnosed with COVID-19 at home using pulse oximetry. This could reassure people who did not need to go to hospital, whilst more quickly identifying individuals with dangerously low blood oxygen saturations (<92%).^2,3

In the English National Health Service (NHS), remote home monitoring using pulse oximetry started to be implemented within some areas during the first wave of the pandemic in the UK.⁴ This was followed by a national implementation during the Winter of 2020/21.⁵ The service was known as COVID Oximetry @home (CO@h) and by the end of January 2021 it was operating in all clinical commissioning areas of England.

The way different areas organised and operated the service varied. People testing positive for COVID-19 would be sent a pulse oximeter for use at home and readings would be sent to local healthcare staff. The process of reporting readings was sometimes facilitated by smartphone technology or reported via telephone, depending on the location and the preferences of the patient.⁴ Some sites started by only enrolling individuals aged 65 or over, or who were deemed extremely clinically vulnerable. Others extended enrolment to a wider age group, and often these criteria changed over time.⁶

One aim of CO@h was to reduce mortality through earlier identification of deterioration. Furthermore, it was hypothesised that if fewer COVID-19 patients were admitted to hospital with advanced disease, and critically low oxygen levels, there may be a reduction in the use of critical care facilities, fewer deaths within hospital and shorter lengths of stay. The anticipated impact on numbers of hospital admissions was less certain since the aim of the programme was not to reduce admissions, but to make sure people who needed to be in hospital were admitted sooner. However, any consequence on the number and mix of patients admitted for COVID-19 would be useful to understand as remote monitoring may have different impacts on different types of individual.

Earlier studies of the use of oximetry for remote monitoring within England during the country's first wave focussed on aspects of safety and implementation, but were unable to establish reliable comparators for measuring impact.^7,8

Faced with this lack of evidence as to the likely effectiveness of CO@h, the two rapid evaluation teams commissioned by the National Institute for Health Research (NIHR) were requested by NHS England and NHS Improvement to undertake a mixed methods study of the service.⁹ This study included evaluations of clinical effectiveness, costs, the processes of implementation and patient and staff experiences, and was one of three evaluations simultaneously requested by NHS England/Improvement.9, 10, 11

This paper presents findings from the clinical effectiveness workstream of the study addressing the specific research questions:

• 1.What is the impact of CO@h on mortality?
• 2.What is the impact of CO@h on the incidence of hospital admission for COVID-19 or suspected COVID-19 and on the characteristics of those admitted?
• 3.For these admissions, what is the impact on in-hospital mortality and length of stay?

Our quantitative approach used combinations of unlinked, aggregated population-level data and hospital administrative data. In doing so we were able to undertake a rapid analysis that not only complemented the other evaluations but provided valuable insight in the future evaluation of similar programmes implemented at scale.

Methods

Study design

The study of overall mortality and admissions was designed as an area-level analysis combining aggregated data from different sources. Considering these data as time series, we investigated “dose-response” relationships¹² between the evolving coverage of the programme within each area and outcome. We analysed four outcomes: mortality from COVID-19, hospital admissions for people with confirmed or suspected COVID-19, in-hospital mortality for these admissions and their lengths of stay. For the in-hospital outcomes, we used an observational design relating in-hospital mortality and lengths of stay at an individual patient level to the degree of coverage of the CO@h programme within the area at the time of admission.

Setting and participants

The setting was all Clinical Commissioning Group (CCG) areas in England where there was complete data on the number of people enrolled onto the programme (onboarded) between 2nd November 2020 and 21st February 2021. (CCGs are NHS organisations that organise the delivery of primary care services within a specific geographic area. At the time of the study there were 135 in England.) The study populations included anyone with a laboratory-confirmed positive test for COVID-19 and any hospital admission for COVID-19 or suspected COVID-19. We also limited the analysis to people aged 65 or over, as this population was eligible for CO@h across all CCGs and both coverage and frequency of outcomes within this group were higher. Implementation amongst younger age groups across the country was much more variable.

Data and variables

For our analysis we used data from several sources (see supplementary material, Table S1). Data on numbers of new cases of COVID-19 and deaths were acquired from Public Health England (now the UK Health Security Agency). New cases were laboratory-confirmed and deaths were those either within 60 days of the first positive test or where COVID-19 was mentioned on the death certificate.¹³ If someone had more than one positive test within the previous seven days, then only one was counted.¹⁴ These data were aggregated by week, age band and CCG. The selected age bands were 65 to 79 and 80 plus. Numbers of people onboarded to CO@h were sourced from a bespoke national data collection for the programme and aggregated by the team at Imperial College London undertaking one of the other two simultaneous evaluations. Due to small numbers, aggregation was performed by fortnight, rather than week, and by the same age bands and by CCG. To comply with data protection rules, these data were also rounded to the nearest five individuals, or, for smaller values, labelled as between one and seven.

Data on hospital admissions and outcomes were obtained from Hospital Episode Statistics (HES). Although most of the non-hospital data was available weekly, we aggregated to fortnightly data in order to match the aggregation of the onboarding data. We restricted our statistical analysis to the period between 2 November 2020 and 21 February 2021 when numbers of cases and outcomes were at their peak. Also, outside that period there were too many low numbers at our chosen level of granularity.

Coverage of CO@h was measured as numbers enrolled onto the programme within each CCG every fortnight divided by the number of new cases detected in that fortnight. To be able to calculate this by CCG, we required the onboarding data within a CCG to be complete. CCGs providing complete onboarding data were identified by NHS Digital. As part of the wider mixed methods study, the team selected 28 study sites for surveys, interviews and to obtain data on costs, most of which were CCGs that provided complete data. For the costing part, sites were independently asked how many people they had onboarded, and we used this information to validate the reports of completeness from the national programme and to include additional CCGs where the numbers onboarded were broadly similar or greater. Further information about this process is included in Section 2 of the supplementary material. Where numbers onboarded were between one and seven, we assigned a value of four, being the mid-point within the range.

We estimated coverage in two ways. One was to calculate it for each CCG regardless of whether a service was operating at the time, and this was used in our analysis. However, to understand what coverage was achievable once a service was implemented, we also estimated coverage within individual CCGs over periods when we knew a service was operating there. For this we only included fortnights over which a service was operating within the CCG for the entirety.

The proportion of hospital beds occupied by COVID-19 patients was used as a measure of local system pressures and sourced from publicly available routine data.¹⁵ By the end of February 2021, most hospital trusts were operating step-down virtual wards whereby COVID-19 patients could be discharged early with a pulse oximeter and monitored at home in a similar way to the CO@h service.¹⁶ Due to the potential influence of these virtual wards on hospital outcomes their existence was incorporated as a confounding variable in our analyses of length of stay and in-hospital mortality.

Comparisons between included and excluded CCGs

We compared population characteristics and COVID-19 incidence rates between the CCGs we included, because their data was believed to be complete, and the remaining CCGs to test how representative the included CCGs were. The mean values and proportions associated with each CCG were treated as the separate observations. Normality was assessed by viewing Q-Q plots of the variables and comparisons were carried out using Student t-test, or Mann-Whitney U-tests where data were skewed. We also investigated their geographical spread.

Analysis of mortality

Because we only had aggregate data for deaths, new COVID cases and people onboarded to CO@H, our approach was to calculate coverage rates for CO@H over time and then investigate relationships between levels of coverage and mortality by age band within each CCG. To do this we adopted a two-stage approach. The first stage was to estimate denominators representing exposure, the second was to use these as offset variables in negative binomial regression models, relating mortality to coverage of the CO@H programme by age group. We included a further variable for the month to allow for changes in relationships as the second wave progressed. To account for CCG-level effects we used general estimating equation (GEE) approaches with an exchangeable correlation structure.¹⁷ This approach accommodates the fact that mortality within a single CCG is likely to be correlated and GEEs ensure that correlation is accounted for by adjusting parameter estimates and standard errors.

The need to estimate denominators arose because we were not able to directly link the new cases and mortality data. When a death occurs, the median time between a new case arising and death is about two weeks, although some may have been diagnosed only in the previous week, and some three weeks or more before. We therefore developed a preliminary set of regression models relating mortality to new cases, with new cases lagged at different times, in order to establish the contributions of the lagged variables. These then determined weights which we used to aggregate new cases into a denominator. Assuming that there was no lag between diagnosis and exposure to the programme, we applied the same weights to the onboarding data to establish a weighted coverage variable appropriate to the mortality observed at each time. A more detailed description of this approach is provided in Section 3 of the supplementary material.

Other options for lagging the time between diagnosis, onboarding and mortality were tested in sensitivity analysis and reported in the supplementary material.

Analysis of hospital admissions

Hospital admissions over the study period were extracted from Hospital Episode Statistics (HES). We considered any admission where COVID-19 or suspected COVID-19 appeared as a diagnosis in the first episode of care, whether as a primary or secondary diagnosis (ICD-10 codes U07.1 and U07.2). If a patient was readmitted with one of these diagnoses within a 28-day period, we only considered the first admission. To match the onboarding data, numbers were aggregated by age band and fortnight.

We undertook a similar procedure for hospital admissions as for mortality, although with different weights, since the time between diagnosis and admission tended to be shorter.

Again, for our sensitivity analysis, we tested different options for lagging the time between diagnosis, onboarding and outcomes. We also tested the option of only including admissions where COVID-19 or suspected COVID-19 was the primary diagnosis.

Separate models were developed to evaluate any impact of CO@h on the characteristics of patients admitted in terms of age, sex, deprivation, Charlson Score (a measure of the severity of co-morbidities) and ethnicity. Our dependant variables for these characteristics were mean age of admissions by CCG, numbers of female admissions, numbers living in the most deprived quintile, defined by the Index of Multiple Deprivation (IMD), numbers with Charlson scores greater than five and numbers reported with non-white ethnicity. For age, we performed ordinary linear regression relating the mean age to coverage and month accounting for CCG-level effects using GEE approaches, as before. For the other characteristics we use Poisson regression to relate each dependant variable to coverage, age band and month and accounting for CCG-level effects in a similar way. For the Poisson regression models the natural logarithm of the number of admissions was used as an offset variable.

Analysis of in-hospital outcomes

To analyse outcomes for COVID-19 patients admitted to hospital, we used individual-level Hospital Episode Statistics (HES). To measure in-hospital mortality we included any death that was reported within the same hospital spell. To investigate the impact on in-hospital mortality, we created logistic regression models relating mortality to the weighted coverage for the relevant CCG with individual patient characteristics as confounders. Values for the weighted coverage corresponded to those calculated for hospital admissions. Again, we used general estimating equation (GEE) approaches to account for CCG-level effects. Length of stay was defined as the number of days between admission and discharge from the same hospital or death within that hospital. We used negative binomial regression models¹⁸ to analyse the impact on lengths of stay of the weighted coverage for the relevant CCG, again with individual patient characteristics as confounders. Stays longer than 60 days were trimmed to 60 days to mitigate the influence of very long stays. Because we used negative binomial models, ratios in outcomes led to our deriving the impact on length of stay as a percentage change rather than a number of days.

Using rounded data

To accommodate the uncertainty caused by the rounding of the onboarding data, we ran all our statistical models multiple times, each time randomly sampling onboarded numbers from the range of feasible values (treating the distributions as uniform). Based on the similarity of results with each simulation, we deemed it sufficient to perform 1000 runs for each model. The simulation results were then pooled to obtain overall effect sizes. All statistical analyses were performed using SAS version 9.4.¹⁹

Patient and public involvement

Members of the study team met to discuss the study with service users and public members of the NIHR BRACE Health and Care Panel and patient representatives from NIHR RSET. Although mostly used for the qualitative evaluations in the wider study, meetings were held during the data analysis phase to share learning and cross-check our interpretations of findings.

Data governance and ethics

The receipt of aggregated data from Public Health England was governed by a data sharing agreement. Receipt of aggregated onboarding data from Imperial College was governed by their separate data sharing agreement with NHS Digital. The access and use of HES was governed by an existing data sharing agreement with NHS Digital covering NIHR RSET analysis (DARS-NIC-194,629-S4F9X). Since we were using combinations of aggregated data and datasets for which we already had approval to use, no ethics committee approval was needed for this analysis. No patient consent was required for this study.

Role of the funding source

The funders had no role in study design, data collection, analysis and interpretation and the decision to publish the manuscript. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR, NHSE&I, NHS Digital or the Department of Health and Social Care. CSJ and TG had access to individual HES records and the raw data on new cases and deaths under the terms of the data sharing agreements with NHS Digital and Public Health England, respectively. All authors had access to aggregated data from these sources as well as all other data used in the study. All authors decided to submit the manuscript for publication.

Results

Data completeness and coverage

Over the period of analysis, we judged that onboarding data was complete for 37 CCGs (27% of the total number of 135 CCGs across England).

The included CCGs had no notable differences in mean age, proportions of non-White population or proportions resident in most deprived areas when compared to the remaining 98 that were not included; although included CCGs had a lower incidence of positive test results (Table 1). There were also regional differences: no CCGs from the East NHS Region were included, and only one from the North East and Yorkshire (Figure 1). The South West, North West and Midlands were the best represented regions.

Table 1.

Characteristics of the populations resident within the CCGs included in the analysis compared with those that were excluded. (Samples are the proportions and rates observed within each CCG).

	Included CCGs (n = 37)		Excluded CCGs (n = 98)
	Mean	(Standard error)	Mean	(Standard error)	P-value for difference between groups*
Mean proportion aged 65 or more	18.1%	(0.7%)	17.9%	(0.5%)	0.77
Mean proportion aged 80 or more	4.8%	(0.2%)	4.8%	(0.1%)	0.91
Mean proportion in non-white ethnic groups	17.3%	(2.7%)	17.7%	(1.8%)	0.97
Mean CCG-level Index of Multiple Deprivation	22.7	(1.3)	22.6	(0.8)	0.96
Incidence of laboratory-confirmed COVID-19 per 1000 people
All ages	41.9	(1.7)	46.8	(1.4)	0.03
Age 65 or over	30.4	(1.3)	35.0	(1.2)	0.01

^⁎Two-sample T-tests. Mann-Whitney test for ethnic groupings

Data sources:

Ethnicity – NHS digital: https://digital.nhs.uk/data-and-information/publications/statistical/mi-ethnic-category-coverage/current.

Deprivation – Ministry of Housing, Communities and Local Government: https://www.gov.uk/government/statistics/english-indices-of-deprivation-2019.

Ages – NHS Digital: https://digital.nhs.uk/data-and-information/publications/statistical/patients-registered-at-a-gp-practice/november-2020.

Incidence of COVID-19 – Public Health England (now UK Health Security Agency).

Figure 1.

Number of Clinical Commissioning Groups (CCGs) within each NHS region with complete and incomplete data between 2 November 2020 and 21 February 2021 (% complete shown in each bar).

Fortnightly coverage of CO@h amongst people aged 65 or over within the 37 CCGs is shown in Figure 2. Rates were particularly low earlier in the period because many sites had not commenced implementation. The numbers of CCGs where there were sites onboarding patients within each fortnight is shown along the horizontal axis. Sites within nine (24%) CCGs were operating services in the first fortnight, which had risen to 33 (89%) within the fortnight beginning 28 December. Services were operating within all CCGs during the fortnight beginning 11 January 2021. The median coverage only exceeded 10% in the final fortnight, although, from the end of November, the maximum was consistently above 30%, with one or two CCGs each fortnight achieving much higher rates than the rest. The overall coverage over the period across all 37 CCGs was 5.9%. If we exclude fortnights during which services were either not operating or operating for only part of the fortnight, the overall coverage was 8.7% with only one CCG averaging a rate of more than 30%.

Figure 2.

Variation in coverage of Covid Oximetry @home (CO@h) by fortnight amongst people aged 65 or over with laboratory-confirmed COVID-19 across the 37 Clinical Commissioning Groups (CCGs) included in the analysis.

Summary data and outcomes

Summary data and outcomes for the period from 2 November 2020 to 21 February 2021 for the 37 CCGs by age band are shown in Table 2. Although there were many more positive tests recorded amongst the under 65′s, there were approximately half as many deaths and equivalent numbers of hospital admissions. Coverage after implementation was low across both age groups and highest amongst those aged 65 to 79. Time series plots of numbers of new diagnoses, numbers onboarded and outcomes are shown in Section 4 of the supplementary material (Figs. S2–S4). Correlations between repeated measures are also shown in the supplementary material (Table S3).

Table 2.

Outcomes across the 37 CCGs included in the analysis between 2nd November 2020 and 21 February 2021.

	Age band		Both age groups
	65 to 79	80+
Positive tests and onboarding over the whole period
Number of new positive diagnoses	41,437	27,944	69,381
Number onboarded to CO@h*	2734	1382	4116
Coverage	6.60%	4.95%	5.93%
Positive tests and onboarding since implementation⁎⁎
Number of new positive diagnoses	26,507	18,636	45,143
Number onboarded to CO@h*	2610	1320	3930
Coverage	9.85%	7.08%	8.71%
Mortality
Number of deaths	4269	8699	12,968
% of new diagnoses over the period⁎⁎⁎	10.30%	31.13%	18.69%
Hospital admissions
Number of hospital admissions	12,351	12,128	24,479
% of new diagnoses over the period⁎⁎⁎	29.81%	43.40%	35.28%
Number of hospital admissions (excluding readmissions within 28 days)	10,895	10,683	21,578
% of new diagnoses over the period⁎⁎⁎	26.29%	38.23%	31.10%
Within hospital outcomes
Number of in-hospital deaths	2748	3948	6696
In-hospital deaths per admission	22.25%	32.55%	27.35%
Median length of stay in days (IQR)	8 (3 to 15)	9 (4 to 17)	8 (4 to 16)
Number staying 2 weeks or more	3489	4134	7623
Proportion staying 2 weeks or more	28.25%	34.09%	31.14%

^⁎Onboarded numbers aggregated from fortnightly counts at CCG level rounded to the nearest 5 or given the value 4 if between 1 and 7.

^⁎⁎Only includes fortnights for which a service was implemented within the CCG over the whole period.

^⁎⁎⁎Ignoring lags between new diagnoses and deaths/admission.

The characteristics of the admitted patients are shown in Table 3. There were roughly equal numbers of these patients in each age band and about half the patients were admitted to hospitals where more than 23% of occupied beds had COVID patients. Coverage of the CO@h programme was estimated as less than 5% in areas where 65% of the admitted patients were resident.

Table 3.

Characteristics of patients admitted with confirmed or suspected COVID-19 over the study period within the 37 included CCGs and values of other explanatory variables on admission.

Explanatory variable	Value	Number of patients	%
Ethnicity	White	19,811	88.99%
	Non-white	2450	11.01%
Charlson Index	5 or less	19,442	79.42%
	Greater than 5	5037	29.58%
Sex	Male	12,807	52.32%
	Female	11,672	47.68%
Deprivation	Most deprived quintile	5295	21.75%
	Excluding most deprived quintile	19,053	78.25%
Age	65 to 79	12,351	50.46%
	80+	12,128	49.54%
Virtual ward active on admission	Yes	9537	41.29%
	No	13,562	58.71%
COVID bed occupancy on admission	< 15%	5932	24.23%
	15 - 22%	6131	25.05%
	23 - 29%	5944	24.28%
	> 29%	6472	26.44%
Weighted coverage*	< 5%	15,914	65.01%
	5 - 10%	4182	17.08%
	> 10%	4383	17.91%

^⁎Estimated coverage at the time the patient was enroled to the CO@h programme.

Mortality and hospital admissions

Results from our models for mortality and hospital admission are shown in Table 4. For every 10% increase in coverage, mortality fell by 2% (relative risk = 0.98, 95% confidence interval: 0.96 to 1.01) and admissions increased by 3% (relative risk = 1.03, 95% confidence interval: 0.99 to 1.07), but neither result is statistically significant. There is, however, a significantly higher risk of mortality and admission amongst the older age group (p < 0.001) and higher risk of mortality in the months following November 2020 (p < 0.001 for each month in comparison to November). Graphical presentations of these results are shown in Section 5 of the supplementary file (Figs. S5–S8).

Table 4.

Results of the multivariate analysis for mortality and hospital admission: relative risks associated with each factor in the model.

Variable		Relative risk of death (95% confidence interval)		Relative risk of hospital admission (95% confidence interval)
Coverage	Every 10% increase in coverage	0.98	(0.96, 1.01)	1.03	(0.99, 1.07)
Age	80+ v 65 to 79	3.06	(2.94, 3.17)	1.55	(1.48, 1.63)
Month	Dec 2020 v Nov 2020	1.25	(1.17, 1.34)	1.06	(1.01, 1.11)
	Jan 2021 v Nov 2020	1.24	(1.14, 1.34)	1.03	(0.97, 1.11)
	Feb 2021 v Nov 2020	1.40	(1.27, 1.55)	0.97	(0.88, 1.07)

We also found no significant evidence of a relationship between the coverage of CO@h and the ethnicity, sex, deprivation or health status of patients admitted to hospital with COVID-19 or suspected COVID-19 (Table 5). The impact on age is of borderline statistical significance: for each 10% increase in coverage admitted patients are on average 4 months younger (95% confidence interval: −7.9 months to 0.2 months, p = 0.07).

Table 5.

Impact of increases in coverage of oximetry on the characteristics of people admitted to hospital with COVID-19 or suspected COVID-19.

Characteristic	% change with each 10% increase in coverage (95% confidence interval)
Proportion non-white ethnicity	0.43%	(−8.99%, 10.82%)
Proportion female	0.07%	(−1.93%, 2.11%)
Proportion resident in most deprived areas (by quintile)	0.30%	(−7.19%, 8.41%)
Proportion with Charlson scores > 5	−2.75%	(−6.37%, 1.00%)
	Change in mean age with each 10% increase in coverage(95% confidence interval)
Mean age	−3.9 months	(−7.9, 0.2)

In-hospital outcomes

The results of our analysis of in-hospital outcomes are shown in Table 6. For every 10% increase in coverage, in-hospital mortality fell by 3% (relative risk = 0.97, 95% confidence interval: 0.92 to 1.03) and length of stay increased by 1.8% (95% confidence interval: −1.2% to 4.9%). For patients staying the median length of 8 days, this equates to an extra 0.14 days, on average. Again, neither result is statistically significant. Non-White ethnicity and existing COVID bed occupancy on admission were both associated with shorter lengths of stay. Again, graphical presentations of these results are shown in the supplementary file.

Table 6.

Results of the multivariate analysis for in-hospital mortality and length of stay. Effects of each factor on the odds of mortality and proportionate change in length of stay.

Variable		Odds ratio associated with in-hospital mortality (95% confidence interval)		Relative change in length of stay (95% confidence interval)
Coverage	Every 10% increase in coverage	0.97	(0.92, 1.03)	1.8%	(−1.2%, 4.9%)
Age	80+ v 65 to 79	1.73	(1.63, 1.84)	6.5%	(3.1%, 9.7%)
Month	Dec 2020 v Nov 2020	1.27	(1.14, 1.42)	−4.2%	(−9.5%, 1.5%)
	Jan 2021 v Nov 2020	1.12	(1.00, 1.26)	−11.5%	(−14.9%, −8.1%)
	Feb 2021 v Nov 2020	0.88	(0.76, 1.03)	−19.7%	(−27.4%, −11.1%)
Sex	Female	0.71	(0.66, 0.76)	2.5%	(0.0%, 5.0%)
Charlson score	Greater than 5	2.14	(1.96, 2.33)	1.1%	(−2.2%, 4.6%)
Ethnicity	Non-white ethnicity	1.24	(1.07, 1.42)	−12.0%	(−16.2%, −7.6%)
Deprivation	Lowest IMD quintile	0.98	(0.89, 1.07)	0.0%	(−3.0%, 3.1%)
Has a virtual ward		0.96	(0.88, 1.04)	1.0%	(−6.5%, 9.1%)
COVID bed occupancy in the trust on admission	Every 10% increase in occupancy	1.04	(0.97, 1.11)	−5.2%	(−7.6%, −2.7%)

Sensitivity analysis

Results from our sensitivity analyses are shown in Section 6 of the supplementary material (Tables S4–S6). None of the changes we made in our assumptions affected our findings with respect to the association between coverage of remote oximetry and outcomes.

Discussion

In this study we have found no evidence of a significant association between the coverage of pulse oximetry, as implemented by the CO@h programme, and COVID-associated mortality or COVID-associated admission to hospital. However, our results indicate that any adverse effect on mortality would be small, and a mortality reduction of up to 4% may have resulted from the programme. For such hospital admissions, we found no evidence of a significant relationship to in-hospital mortality or length of stay. Also, for COVID-associated admissions, there appears to have been no associated change in the patient characteristics we have tested.

There are several possible explanations for these results, and it would be premature to conclude that the COVID oximetry @home programme has had no impact. Firstly, limitations in data completeness meant that we were only able to analyse onboarding data for one quarter of CCGs across England. This, combined with lower than expected coverage within these CCGs, meant that our ability to detect any possible impact was smaller than anticipated. This also meant that some areas of the country were under-represented. Secondly, individual level associations may not have been seen at the aggregate level. Thirdly, qualitative findings from our wider study revealed that COVID oximetry @home was implemented in a variety of ways by different services within CCGs, some of which may have had more impact on outcomes than others.⁶ The possible impact of the intervention on hospital admissions was always uncertain, since the aim of the programme was to ensure people who needed hospital treatment were admitted at the right time. Also, once in hospital, the determinants of length of stay are complex and multifactorial, and may have varied during the time course of the second wave of COVID-19 infection in England.

The anticipated value of our approach with aggregated data was that it would complement the other two simultaneous quantitative evaluations of CO@h,^10,11 could be carried out more rapidly and, if COVID-19 continues to stretch national health services, it could be more readily repeated as new data become available, provided the right information is routinely collected at source.

We were able to handle small number suppression and the rounding of aggregate data by multiple random sampling throughout the range of possible values and it was encouraging to discover that this uncertainly did not have a large impact on results.

Using aggregated data has some limitations because it does not enable us to trace direct links between the onboarding of an individual and their outcomes. There may also be an ecological fallacy, where individual-level effects are not observable in the aggregated data.²⁰ However, obtaining linked individual-level data is a complex and potentially long process that may not always be feasible when there is a need to provide rapid feedback to a developing programme and where resources are stretched. Unfortunately, however, our ability to provide rapid feedback was compromised by delays in obtaining onboarding data which proved an understandable challenge for local services in the midst of a pandemic.

This has been part of a larger mixed methods study that has added insight into some of our findings and provided locally collected bespoke data against we could verify information we received centrally about coverage and data completeness. However, these checks could only be made against the 28 study sites and we were not able to verify the data from the other CCGs in the same way. Findings from surveys and interviews have helped interpret what we have found and, conversely, our data analyses have helped provide a context against which to understand the relative importance of the qualitative findings.^6,21

The qualitative study has also shed light on the different ways that the services had been implemented.⁶ However, with just 12 of the 28 study sites being amongst the 37 included in our analysis, in combination with low coverage, it has not been possible to undertake a quantitative study of the relative performance of the different types of implementation.

We anticipated that finding a suitable comparator group during the national implementation of a programme was likely to be problematic, and we therefore avoided this problem by treating the relationship between coverage and outcome as a dose-response. However, the power to detect any impact in such an analysis depends on the level of coverage which, in practice, was lower than we hoped.

During the period of our analysis the vaccination programme was starting, and by the end of our study period 88% of people aged 65 or over had at least one dose.²² Although this study investigates outcomes of people after being diagnosed with COVID-19, there is evidence that vaccination changes the subsequent risks of mortality and hospital admission²³ which could have had a confounding effect on our analysis.

In the context of fortnightly data, we assumed a minimal lag between the diagnosis of COVID-19 and onboarding to CO@h. However, there was evidence from sites that they sometimes encountered delays in identifying positive cases,⁶ although the overall impact on this assumption is uncertain.

Prior to this study, very little was known about the quantitative impact of the use of pulse oximeters for remote home monitoring of people diagnosed with COVID-19. One of the other evaluations of the CO@h programme in England (not yet peer-reviewed) also found no significant impact on mortality or health service utilisation.¹⁰ However, the study did find reductions in mortality and increases in hospital attendance (yet with lower use of critical care) amongst people enroled onto the programme after attending the Emergency Department (ED).²⁴ A study of 4384 high risk patients receiving home monitoring of vital signs, including pulse oximetry, in one region of Galicia, Spain, found lower admissions, lengths of hospital stays and in-hospital mortality when compared with other local regions.²⁵ A retrospective cohort study from South Africa evaluated the use of pulse oximetry for people diagnosed with COVID-19 to read their own blood oxygen levels without remote monitoring by local health services. As with the Spanish study, the implementation was limited to people deemed to be of high clinical risk.²⁶ They found a significant improvement in mortality but no impact on admissions to hospital: the reduction in mortality being explained by earlier admission. A recent study of CO@h carried out at one site demonstrated reductions in 30-day mortality and lengths of stay amongst people admitted to hospital.²⁷ This, however, is currently a pre-print prior to peer review and lacks some details about the comparability of the control group. In another study implemented in the UK during the first wave, patients with suspected COVID-19 attending ED were discharged home with an oximeter. They observed a reattendance rate of 4.7% compared to 22.7% amongst a retrospective control group.²⁸ However, this was a younger cohort (median age 41 years) and the absolute numbers of reattendance were small (nine in all). Other studies have reported on the safety of similar programmes, but have lacked comparators.^8,29, 30, 31, 32

At the start of this study we anticipated the services would have higher coverage and complete data would be available from more CCGs. Although the use of aggregated population-level data can enable more rapid evaluation of a new service, these two elements had an influence on the power of the analysis to detect an impact. The resulting shortfall in expected data reflects the challenges of trying to centrally manage a bespoke data collection while services are already stretched. However, sufficient quantities of data are vital to determining whether a service is effective, so it is important to understand how this can be improved, for example, by concentrating data collection in a few sites and using routinely collected data wherever possible.

Furthermore, low coverage raises questions about capacity of both staff and resources in the midst of high infection rates and how it is possible to secure the best value from such a service under the circumstances. The fact that at least one CCG managed to achieve reasonably good coverage indicates the possibility for learning from others.

This study provides an evaluation of the national implementation of remote home monitoring of pulse oximetry for people diagnosed with COVID-19 across the English NHS. Although we detected no significant impact on outcomes, there are potential explanations for this finding that are unrelated to the effectiveness of the programme. Taking due account of populations that may respond less well to oximetry, there is no evidence that future implementation of similar programmes would be unsafe. However, the challenges of providing sufficient data so that effectiveness can be adequately measured need to be overcome.

Declaration of interests

Funding for this study was provided by the National Institute for Health Research (NIHR) and NHS England and Improvement and payments made to the institutions of all the authors. The authors have no other competing interests.