Learning from the initial deployment of digital contact tracing apps

Smartphone applications for digital contact tracing were a truly innovative intervention in the COVID-19 pandemic. These apps aim to automatically identify and rapidly notify individuals who have been in close contact with people who have tested positive for SARS-CoV-2. These apps were deployed in many countries in record time to support or complement traditional contact tracing systems. Epidemiological models showed the potential impact of this intervention under specific epidemic conditions,1, 2, 3 but months into the apps' deployment, real-world evaluations are now required to identify shortcomings and look for improvements.

In The Lancet Public Health, Florian Vogt and colleagues⁴ report the findings from a prospective study of the deployment of COVIDSafe, Australia's national COVID-19 tracing app, in the state of New South Wales (NSW). The app was based on a centralised approach, in which the information of contacts is uploaded to a common database and accessed by public health authorities for risk assessment and notification. Analysing data from the first months of app deployment between May and November, 2020, the authors report that the app was used by 137 cases—ie, 22% of all 619 detected adult cases. Altogether, 205 contacts were recorded, among whom 79 (39%) qualified as close contacts after validation by public health staff. These results correspond to 39% positive predictive value for digital contact identification. 17 of these contacts were detected by the app only and constitute the yield of the app. As a comparison, conventional contact tracing identified more than 25 300 close contacts during the same period. Based on the low positive predictive value and yield, the authors concluded that the app did not provide a meaningful contribution to the COVID-19 response in NSW during the period.

Previous real-world evaluation studies have provided more positive conclusions regarding the role of apps in the COVID-19 response.5, 6, 7 A pilot deployment of an early version of the UK app was conducted in the Isle of Wight.⁶ The analysis of incidence data showed that the reproductive ratio decreased significantly immediately after the app launch. Later, the national deployments of contact tracing apps in Switzerland and in England and Wales provided further evidence of the usefulness of the intervention.5, 7 Both the UK and Swiss apps are based on a decentralised approach, in which the data collected stay on the user's device and notifications occur automatically, without the involvement of public health staff.⁸ The study of the Swiss app argued that digital contact tracing can be effective in detecting exposed contacts, who then become COVID-19 positive, with a yield similar to conventional contact tracing.⁷ In England and Wales, the analysis encompassed the first 3 months of the app's deployment.⁵ The authors of the UK study combined two epidemiological approaches relying on diverse information and concluded that between 284 000 and 594 000 cases could have been averted by the use of the app—ie, approximately one case averted for each case using the app. Taken together, these three studies suggest that digital contact tracing can be an effective public health tool.

All the examples discussed differ substantially in the nature of the digital contact tracing program, the epidemiological context, and the metric used to evaluate the intervention. This makes their direct comparison extremely hard. Digital contact tracing is indeed a complex intervention.⁹ Its performance depends on the functioning of the technology, but also on adoption by the users and how it integrates into the public health response. Vogt and colleagues⁴ discuss in detail the issues that might have limited the impact of COVIDSafe, thus providing precious lessons learnt. The authors acknowledge that if a better real-world piloting of the app had been done before its launch, this might have helped to improve app performances. In addition, contacts identified by the COVIDSafe app underwent validation by public health staff before further investigation. This was time-consuming for the public health staff and probably undermined the fundamental potential advantage of the app—to be fast and scalable. Vogt and colleagues also warn that their conclusions might not apply directly elsewhere. Indeed, evaluating the contribution of an app within an epidemic response plan will crucially depend on the epidemic context—eg, the extent and speed of the epidemic—and the interplay with other interventions in place. The epidemic in NSW was under control during most of the study period. Therefore, conventional contact tracing was extraordinarily effective, with more than 40 close contacts identified per COVID-19 case on average.⁴ This is in sharp contrast with the fewer than two close contacts per case identified by conventional contact tracing in the UK study.⁵ The contact tracing authorities in NSW were never overwhelmed during the study period and, therefore, the relative interest and the cost–benefit balance of digital contact tracing were probably less apparent than in other countries.

Setting clear criteria for evaluating digital contact tracing apps and a common glossary for comparison is a research priority.9, 10 This is essential for assessing the complex technological, public health, and social factors involved in the intervention and learning from different countries' experiences. The optimal implementation of a digital contact tracing app must account for the epidemic context and deal with acceptability, privacy, and the respect of civil liberties. Careful evaluations of the various digital contact tracing apps developed and deployed in tens of countries across the world since the beginning of the COVID-19 pandemic must contribute to the debate.

We declare no competing interests.