MyCOVIDRisk app: development and utilisation of a COVID-19 risk assessment and mitigation application

INTRODUCTION

The transmission dynamics of SARS-CoV-2 are complex. They depend on factors that enhance or protect against fomite, large droplet and aerosol transmission, as well as local prevalence of disease. The public face challenges in understanding and making educated decisions about daily activities, prompting perspective pieces such as ‘We’ve been left to calculate our virus risk on our own. We’re terrible at it.’¹ Mobile apps could play an important role in helping individuals understand infection risk from everyday activities. Current COVID-19 risk apps range from predictive models estimating the risk of critical illness, to symptom checkers and workplace guides.

Here we present the development and implementation of MyCOVIDRisk app, intended to both inform Americans of the risk incurred when engaging in different activities and to guide on risk-reduction measures. Our objective was to create a tool that was freely accessible to the public, incorporated up-to-date information on local disease prevalence² and helped people easily understand how to reduce risk without divulging personal information. The hypothesis was that if individuals could continue to engage in enjoyable low-risk activities, we could reduce community transmission while also minimising anxiety, isolation and so-called pandemic fatigue.

METHODS

Literature review

The idea of MyCOVIDRisk was conceived in July 2020 following conversations on social media about challenges with estimating risk and the cognitive burden of making these calculations with little knowledge several times a day. We reviewed peer-reviewed and grey literature to identify published studies on: (1) transmission dynamics and protective measures, (2) COVID-19 risk scores, and (3) risk assessment apps or websites. We aimed to identify infection (1) risk factors, (2) sources of reliable prevalence data, (3) attack rates associated with different activities and (4) studies modelling the effect of mitigation factors.

Model development

In contrast to explanatory statistical modelling that focuses on testing hypotheses, our goal was to create a predictive model for the purpose of forecasting the value of a new observation (whether a person will develop COVID-19). We aimed to identify a simple model that would roughly predict infection risk and also make the development process shorter and less complicated. With this knowledge in mind, we undertook four steps common to predictive model development: data understanding, model assembly, model audit and model delivery. We had an a priori understanding of variables that needed to be included to predict transmission risk based on existing literature, clinical care and public health guidelines, and supplemented this experiential data with a literature review (data understanding). Based on the literature review, we identified high-quality models for estimating transmission and mitigation (model assembly). We consulted with experts in biostatistics, epidemiology and mathematics to inform adjustments to the model and to provide independent assessments model validity (model audit). Due to continued lack of accurate population data on transmission patterns, prospective or retrospective model validation based on real-world data was not possible at the time of model creation. After we completed fine tuning of the model and received feedback on the app design, we deployed the app and shared it publicly together with documentation and communication of the scientific premise of the model (model delivery).

Creation of app wireframe

Design decisions were made based on behaviour change theory, theories of ‘persuasive technology,’ principles of user-centred design, and prior experience in development of effective and engaging digital health technologies, to ensure the app was usable for individuals of all ages and digital literacy.^3–5 To maximise persuasiveness, we designed the app to be used in two stages: the first stage requires user input to specify details of the planned activity, and the second stage allows the user to choose options to reduce transmission risk (online supplemental file 1). To reduce user fatigue and improve engagement, we limited scrolling, avoidable clicking and the number of input screens prior to the preliminary output. Consistent with best practices for digital health behaviour change, we provided person-alisation and interactivity, used multiple techniques for engagement, and included both positive and negative feedback. We worked with a UX expert to design icons that were visually appealing and inclusive.

App analytics

We obtained basic usage statistics from Google Analytics (14 October–18 December 2020) and back-end app data (1 October–18 December 2020). Although formal user feedback was not solicited after launch, unsolicited feedback was received through our website, email and informal conversations.

RESULTS

Findings of our review included a risk chart ranking day-to-day activities into categories of risk and COVID-19 risk apps⁶; however, the apps required users to share demographic information, chronic health conditions or health records.^7–10 Many COVID-19 apps were designed to show risk of critical, fatal illness or hospitalisation. Other than the transmission estimator by Jimenez, we did not find other COVID-19 tools that calculated projected risk of daily activities.¹¹

Based on our initial review, risk factors included location near high COVID-19 prevalence counties, indoor activities,¹² poor ventilation, long durations of visits, physical exertion and close proximity to others.¹³ Mitigation factors included wearing a mask, distancing, reducing activity time, washing hands, increasing ventilation and wearing eye protection.^14–19

Model

Based on our literature review, the most accurate model of transmission dynamics was identified as the box model of airborne transmission, developed by Miller et al and instrumentalised in the COVID-19 Aerosol Transmission Estimator by Jose Jimenez.^{11 20} Using Jimenez’s estimator, we calculated the probability of infection given user entered data, local prevalence, and then used odds ratios (ORs) reported in the literature to calculate posterior probabilities of infection with mitigation measure use.¹⁶ After consultation with external experts, in the absence of a clear consensus of how to calculate risk, we assumed that the individual protective measures were independent events with independent effects on probability (eg, allowing multiplication of effects). Regarding risk levels, we considered a 5% risk of infection (eg, the attack rate for a family member) as ‘very high,’²¹ and the risk of fatality when flying in an aeroplane (assuming travelling by plane eight times a year during a 75-year lifespan) as ‘very low’.²² Parameters for other user inputs—quanta (infectious particle transmission rate), building ventilation rates, event venue size—were sourced from peer-reviewed literature and expert consensus.^{11 23}

App overview

Based on iterative user feedback on the initial app wireframe, we reordered screens; redesigned input icons; changed input screens for activity time and per cent of people masked; included check-boxes to ensure its use in the USA and moved detailed modelling information to optional screens.

Analytics on use

MyCOVIDRisk app was launched on 1 October 2020. As of 18 December 2020, the app was accessed over 1 million times by users in over 112 countries (96.5% in the USA). Within the USA, first-time users accounted for 84.5% of access (table 1). Of activities selected, meeting at friend’s house was most common (22.6% of respondents), followed by shopping (20.2%) and taking a walk (11.6%). Activities related to dining (restaurant: 8.7%, bar: 1.7%) and self-care (salon: 7.0%, gym: 5.7%) were less common (online supplemental file 2). Planned gathering sizes varied widely between users with the majority of calculations (55.6%) involving groups of 1–10 people. Of those using mitigation steps, almost all (99.9%) selected social distancing and 83.7% planned to wear a mask (table 2).

Table 1.

Estimated demographics based on the subset of Google users with demographic data available to Google Analytics (14 October–18 December 2020, total N=410 118)

Characteristic	n (%)
New	346 550 (84.5)
Returning	63 568 (15.5)
Age (years)
18–24	41 421 (10.1)
25–34	118 934 (29.0)
35–44	76 282 (18.6)
45–54	74 231 (18.1)
55–64	59 467 (14.5)
64+	39 781 (9.7)
Device
Mobile	289 133 (70.5)
Desktop	104 990 (25.6)
Tablet	15 994 (3.9)
Session information
Average duration	1:22 min
Average # sessions/user	1.4

Table 2.

Selected mitigation measures among the subset of Google users completing mitigation steps (1 October–18 December 2020, based on back-end application data, N=170 142)

Mitigation step	n (%)
Social distancing	169 972 (99.9)
3 ft	62 102 (36.5)
6 ft	90 685 (53.3)
9 ft	17 354 (10.2)
Washing hands	149 725 (88.0)
Mask	142 409 (83.7)
Homemade	68 227 (40.1)
Fit
Loose	9868 (5.8)
Tight	58 359 (34.3)
Layers
One	10 889 (6.4)
Two	57 338 (33.7)
Surgical	48 320 (28.4)
N95	25 691 (15.1)
Eye protection	28 583 (16.8)

The majority of users exited the application after receiving an initial risk assessment. The selections of the subset of Google users who continued on to input desired mitigation measures are summarised.

Tracking user risk assessments before and after selection of mitigation steps revealed that the majority of users received a ‘low-risk’ or ‘very low-risk’ assessment even before mitigation steps were selected. Those that received a ‘high-risk’ score most often were able to achieve ‘low risk’ after selecting mitigation steps (online supplemental file 3).

DISCUSSION

The MyCOVIDRisk app was created within 3 months in response to the public health imperative for accurate, comprehensible risk assessment information. Its high utilisation, despite lack of formal advertising, demonstrates demand for and accessibility of this simple risk assessment and mitigation tool.

Using health apps to increase public health awareness and reduce misinformation should be part of a comprehensive public health strategy to address epidemics or pandemics. Over 81% of adult Americans have smart-phones and one in five uses health apps.^{24 25} To design and launch a useful, usable application requires not just scientific evidence, but also the ability to incorporate principles of user-centred design and science communication. Behaviour change is essential to reducing SARS-CoV-2 transmission. Elements of behaviour change related to this work include: (a) helping people understand transmission (here is your MyCOVIDRisk score), (b) creating social norms (people want to reduce risk), (c) giving people an action (take these mitigation steps like mask-wearing to reduce risk), (d) making change easy (easily choose a safer activity). MyCOVIDRisk is easy to use, has widespread uptake and illustrates the importance of multiple layers of protection. Additionally, it is updated with real-time prevalence data using Application Programming Interfaces (APIs) and is less expensive than other traditional public messaging campaigns. Using health apps to increase public health awareness and reduce misinformation should be part of a comprehensive strategy to address pandemics.

Future work should include considerations of how to disseminate and motivate use of the app by those who may be sceptical or unaware, and how to enhance use of mitigation steps. Changes in knowledge, behavioural intention and actual behaviours are still unknown. Additional modifications could include enhancement of more complex risk modelling (eg, travel, doctor’s visits), ‘behavioural nudges’ or linkages to testing. Limitations include that it may be inaccessible to those at highest risk: Black, Hispanic, Native Americans and older adults have decreased access to broadband WIFI (although national studies suggest similar rates of smartphone access and health app usage). We hope to translate the app and ensure cultural relevance to diverse groups.

Although the risk model would ideally be validated prospectively, continued lack of accurate data on exposure histories of those diagnosed with COVID-19 makes this challenging. Effect estimates of mitigation measures were partly based on observational studies of other beta-coronaviruses, due to limited data available for SARS-CoV-2. We purposefully provided quintiles of risk rather than exact estimates, recognising continued scientific debate about precise transmission dynamics. Although we may be overestimating the benefit of multiple protective measures, research shows that when layers of protection are used the risk approaches zero.^26–29 We would encourage scientists to contact us to pressure-test our model using assumptions about viral transmission dynamics.

CONCLUSION

MyCOVIDRisk could serve as a model of mobile apps that enhance public awareness and gamify risk mitigation. Although the impact of the app on COVID-19 fatigue and anxiety has not yet been elucidated, apps such as MyCOVIDRisk may help the public make more nuanced decisions that allow safe activities to continue when pandemics last for months.

Summary box.

What are the new findings?

• A simple web-based mobile application to estimate risk of COVID-19 transmission is feasible and acceptable among the US public.

• Transmission risk can be estimated for app users based on local prevalence of disease, type of activity and mitigation measures employed, without collecting personal health information.

How might it impact on healthcare in the future?

• Health apps that are free, publicly available, and incorporate evidence-based research could reduce COVID-19 fatigue and safety measure compliance by allowing individuals to make their own risk assessments and enjoy low-risk activities safely.

• Social media may be a useful tool to obtain early user feedback and promote health tools during a public health emergency.