What data can reveal and conceal

On March 15, 2020, data started coming in from the COVID-19 Hospitalisations in England Surveillance System (CHESS). Unlike the patchy, delayed numbers in the existing database, CHESS provided a comprehensive overview of COVID-19 patients needing critical care. The results were alarming. Whereas the previous database had implied low numbers of admissions without a clear increasing trend, CHESS showed more and more people turning up critically ill across the country. The earlier numbers had been an illusion; the UK was already in the middle of a large epidemic.

In May, 2020, the statistician David Spiegelhalter described the UK Government's daily presentation of statistics as context-free “number theatre”. There were tallies of tests and deaths, but the lagged, incomplete values did not fully reflect the underlying reality. In Covid by Numbers: Making Sense of the Pandemic with Data, Spiegelhalter and Anthony Masters tour the wide range of COVID-19 statistics that have since accumulated, with a focus on the UK. As well as routine surveillance of COVID-19 cases and SARS-CoV-2 genome sequences, the book draws on data from ongoing studies such as the RECOVERY trial and the Office for National Statistics community infection survey, and large-scale analysis of health records through projects like OpenSAFELY.

When I teach students outbreak analysis, many are surprised to discover the ambiguities and assumptions behind oft-quoted “facts” about novel pathogens. Knowledge about COVID-19 has arguably accumulated faster than for any infectious disease in history. However, the volume of overlapping—and at times seemingly contradictory—research can easily be overwhelming. Spiegelhalter and Masters tackle the problem by providing bite-sized answers to key pandemic questions, many of which are not as straightforward as they might seem. The result is a valuable overview of COVID-19 statistics and how to navigate them. Rather than just quoting numbers, Spiegelhalter and Masters discuss how to think about epidemic data. They cover a range of tools, including log-plots, which enable comparisons of exponential growth, and age-stratified analysis to weigh up the benefits and side-effects of vaccines.

An inevitable limitation of a book about COVID-19 numbers is that it can only show plots and tables for numbers that exist. After all, data only reflect the part of reality that someone has bothered to measure. As the pandemic response turns towards pandemic recovery, we should avoid assuming that recorded outcomes are the only ones that matter. “There will be an invisible pool of continued ill-health”, Spiegelhalter and Masters warn.

Focusing on the wealth of UK data can also leave the impression that the pandemic is similarly documented elsewhere. Globally, data are scarce for some COVID-19 tragedies. Analysing satellite imagery of burial sites in Aden, Yemen, and Mogadishu, Somalia, epidemiologists have estimated mortality might have been 60% higher than pre-pandemic levels. Looking only at official mortality data, we would not realise these cities have been among the worst-affected areas globally. Spiegelhalter and Masters are correct that it will take years to fully understand the impact of the pandemic, but time is running out to address these data gaps before biological markers—and collective memories—wane locally.

As well as the availability of datasets, there is also the question of what countries do with them. During the summer of 2020, I often wondered what would have happened if there had been better situational awareness in the UK during February and March, 2020. What if people had been able to see the hundreds of outbreaks that—as genomic data would later reveal—were being seeded as infections arrived from Spain, Italy, and France? The events of autumn would provide the answer, as the epidemic grew and grew like a slow-motion replay of spring, this time in high definition.

Despite impressive COVID-19 vaccine trial results in November, 2020, which created a clear incentive to suppress infections while vaccinating risk groups, countries across Europe suffered large second waves over winter. Meanwhile, some of the places that generated these crucial trial data are yet to benefit like others have. Several major vaccine trials have been done in South Africa, for example, but in mid-September, 2021, only 13% of the country's population had been fully vaccinated. While science may aspire to stand on the shoulders of giants, vaccine inequity has trampled on the hopes of the vulnerable.

No doubt many books will be written on the COVID-19 pandemic. Some will focus on local policies and problems, whereas others will cover global infections and accompanying injustices. Each will inevitably document and debate the events of 2020 onwards from a different angle. But if they want to get the statistics straight, their authors may want to read Covid by Numbers first.