Table 2.

Summary of outcomes and other key findings

	Primary outcomes		Secondary outcomes
	Number or proportion of contacts identified (observed or required for outbreak control)	Number or proportion of subsequently diagnosed cases identified	Impact on R0 (or other indicators of outbreak control)	Uptake	Resource requirements or cost-effectiveness	Other ethical issues	Lessons learnt from implementation of the intervention
Fully automated contact tracing
Bulchandani et al (2020)28	Not reported	Not reported	Estimated that approximately 90% app ownership required for epidemic control if 50% of transmission is asymptomatic but no presymptomatic transmission; multiple input parameter values assessed	Not reported	Not reported	Not reported	Not reported
Ferretti et al (2020)14	Multiple values; related to delay to intervention (eg, if 60% of cases are isolated, >65% of contacts need to be identified and quarantined to bring R0<1)	Not reported	High uptake and compliance, and decreased delays to notification (presented in a 1 day increments, from 3 days down to no delay) and quarantine improve the likelihood of reaching R0<1	Not reported	Not reported	Propose that such a scheme should fulfil eight requirements in order to be ethical (appendix p 8)	N/A (modelling study)
Hinch et al (2020)29	Baseline scenario of 3 day doubling time and 80% app uptake among smartphone owners means approximately 10–15 million people would be quarantined (at any given time, with variation over time); in addition to the population over 70 years who were assumed to practise shielding	Not reported	All app configurations showed a substantial reduction in new cases and hospital and intensive care unit admissions, and a substantial number of lives saved over a 150 day period from the start of a 35 day lockdown, increasing incrementally as uptake increases from 0% to 80% of smartphone owners; direct contact tracing (only first-order contacts quarantined) suppressed the epidemic only under optimistic epidemic growth assumptions (3·5 day doubling time, 5·0 day generation time); recursive tracing (with first-order contacts’ household members also quarantined) reached epidemic control under pessimistic assumptions but leads to only a 50% reduction in numbers quarantined compared with full lockdown; the authors estimate that test-based quarantine release would require approximately 100 000–200 000 tests per day	80% usage to reach suppression under scenarios 3 and 5 with epidemic doubling time of 3·5 days; lower uptake rates reduced incidence	Not reported	Not reported	N/A (modelling study)
Kim and Paul (2020)31	Not reported	Not reported	Under a range of assumptions, the percentage of the population needed to be enrolled in automated contact tracing for outbreak control (Re<1) was estimated (eg, 40–60% uptake required for Re<1, assuming a 30% mean transmission probability per contact event, if 75–95% actively confirm when they get infected; appendix p 6)	Not reported	Not reported	Not reported	N/A (modelling study)
Kucharski et al (2020)32	Median number of (physical and conversational) contacts successfully traced and quarantined (assuming 90% of those traced are quarantined) per case (modelled results) was 4 (mean 14) via automated contact tracing, 21 (mean 32) via manual contact tracing (acquaintances only), 28 (mean 39) with all contacts traced manually	Not reported	App-based contact tracing would require a high level of coverage to ensure Re<1; smaller relative reduction in Re than manual contact tracing for either all contacts or acquaintances only, varying with proportion symptomatic and relative role of asymptomatic transmission; automated tracing alone was estimated to reduce Re by 44%, whereas manual tracing of all contacts reduced Re by 61% (assuming transmission occurs only through physical or conversational contacts that can be recalled)	Estimated 53% uptake (as an input parameter) across UK population in baseline optimistic scenario, on the basis of 75% uptake × 71% smartphone ownership	Not reported	Not reported	N/A (modelling study)
Xia and Lee (2020)34	Not specified; assumes 100% identification of contacts	Not reported	Assessment of the minimum uptake required to reach R0<1; estimated at 95–100% (if only 2–10% of cases are detected because of a large proportion being mild or asymptomatic)	Not reported	Not reported	Not reported	N/A (modelling study)
Yasaka et al (2020)35	Not specified	Not reported	No summary estimates presented; based on results presented graphically, an estimated 65–90% of the population would be infected at the epidemic curve peak with 0% uptake of the automated contact-tracing app, and 15–50% infected with an adoption rate of 75%	Speculated that absence of a user registration process will improve adoption rates	Suggested that because of decreased data sharing, overheads for government agencies would be minimal	Not reported	N/A (modelling study)
Partly automated contact tracing
Danquah et al (2019)36	Mean 36 per case with app-based system; 16 per case with paper-based system	Not specified; data suggested no identified contacts developed Ebola (however, only 69% [384 of 556] of registered contacts were visited)	69% (384 of 556) of registered contacts for 16 confirmed cases were documented as visited under the app-based system compared with 39% (157 of 407) of contacts for nine confirmed cases for whom paper forms were returned under the paper-based system (out of 25 confirmed cases in this system; paper forms were not returned for the other 16 cases)	Not reported	Training contact tracers took 3 days; contact tracers reported that the app-based system was faster and more accurate; reduced travel time (by 5–6 h per coordinator per day); battery charging and technical support were both important	Not reported	Technical issues included poor network coverage, battery life, and quality of phones; further training on syncing data between phone and server needed; compensation and planning for phone charging, including travel to charging booths; and more refresher training for contact tracing and monitoring
Li et al (2017)37	Not reported	Reduced delay to intervention by the hospital infection control team by 0·5–4·0 h	Not reported	Not reported	230–476 h of contact tracing time saved per annum (baseline unclear)	Not reported	Authors stated that implementation of system took substantial time and effort from users; and workflow changes needed when specific data analytics were not available within the software
Schafer et al (2016)38	More than 50 000 contacts recorded on system for >100 000 cases by end of 2015	Not reported	Not reported	Widely used by contact tracers in these settings, percentage not specified	Resources required included technical expertise for training provision, data management staff who were technically skilled and trained (the authors noted that these skills were often scarce locally), and time and expertise required for set-up and maintenance of the database and network; network connectivity and electricity supply were also important resources, and recurrent outages made data entry and transmission challenging	Not reported	Successful use of app required organised flow of contact information between data managers and contact tracers, and a concerted effort to use the app (often not the case); Epi Info Viral Hemorrhagic Fever app designed to require minimal IT support but multiuser data entry expanded complexity and support requirements (minimal, if any, IT support was available); accommodating many countries’ needs within one software was challenging, particularly as it was not possible to customise the software for each country
Sacks et al (2015)39	210 contact tracers monitoring 9162 contacts	Not reported	Not reported	210 of 366 who were trained on CommCare actively used it	Total time to establish programme was 10–13 weeks (training contact tracers took 2–3 days); smartphones, SIM cards, 500 MB data per month, and solar phone chargers donated for staff use; need for technical support	Focused on the ethical issues around adopting new technology during complex emergency; authors argue this disruption might be required, but that feasibility and risks should be carefully considered and, where implemented, accompanied with close managerial oversight	Availability and cost of expertise for specific software used; recruitment of local technology expert youth volunteers helped to accelerate phone configuration; actual use of data by government staff members to inform action was low; clearer initial standards (for contact-tracing protocols and metrics) could have accelerated the design process
Tom-Aba et al (2015)40	Not reported	Not reported	Reduced delay to evacuation of symptomatic contacts from their homes to an isolation facility from 3–6 h to 1 h, and the proportion of contacts followed up increased from a variable baseline of between 90% and 99% (before the partly automated system was introduced) to consistently 100% afterwards, with associated benefits for outbreak control	Not reported	Costs of android phones, tablets, laptops, data plans, and high-speed internet; time costs of trained personnel	Not reported	Not reported
Automated contact detection in a relevant disease context (without subsequent contact tracing or contact notification)
Aiello et al (2016)41	453 281 total Bluetooth contacts between iEpi substudy smartphones only (62·5 contacts per phone per day) and 1 591 741 with other devices (219·4 contacts per phone per day) over 78 days	N/A	Not reported	Not reported	Availability and training of a large study staff (size not specified); system required mapping, debugging, data cleaning, and verification	Not reported	95% (281 of 295) of participants who completed the exit survey reported joining the study because of the cash incentive
Al Qathrady et al (2016)42	353 458 encounter records from 34 225 users in 1 week	Not reported	All contacts within the simulated institutional outbreak could be traced on the basis of one infectious case having been identified (for an infection with a latent period of 1 day and infectious period of 2 days); further detail not provided	Not reported	Not reported	Not reported	Accuracy of the contact-tracing system in some buildings decreased with coverage, but increased in others, because of differences in the encounter patterns within each building, and the source node chosen
Voirin et al (2015)43	18 765 contact events recorded among 84 individuals over 11 days (cumulative duration 251 h)	N/A	Not reported	Not reported	Not reported	Not reported	Most contacts between nurses or between nurses and a patient; influenza transmission is difficult to predict from contact data alone

Additional details can be found in the appendix (appendix pp 8–10). N/A=not applicable. R₀=basic reproduction number. R_e=effective reproduction number.