Identifying and addressing digital health risks associated with emergency pandemic response: Problem identification, scoping review, and directions toward evidence-based evaluation

Abstract

Background and objective

The COVID-19 pandemic has accelerated digital health applications in multifaceted disease management dimensions. This study aims (1) to identify risk issues relating to the rapid development and redeployment of COVID-19 related e-health systems, in primary care, and in the health ecosystems interacting with it and (2) to suggest evidence-based evaluation directions under emergency response.

Method

After initial brainstorming of digital health risks posed in this pandemic, a scoping review method was adopted to collect evidence across databases of PubMed, CINAHL, and EMBASE. Peer-review publications, reports, news sources, and websites that credibly identified the challenges relating digital health scaled for COVID-19 were scrutinized. Additional supporting materials were obtained through snowball sampling and the authors’ global digital health networks. Studies satisfying the selection criteria were charted based on their study design, primary care focus, and coverage of e-health areas of risk.

Results

Fifty-eight studies were mapped for qualitative synthesis. Five identified digital health risk areas associated with the pandemic were governance, system design and coordination, information access, service provision, and user (professional and public) reception. We observed that rapid digital health responses may embed challenges in health system thinking, the long-term development of digital health ecosystems, and interoperability of health IT infrastructure, with concomitant weaknesses in existing evaluation theories.

Conclusion

Through identifying digital health risks posed during the pandemic, this paper discussed potential directions for next-generation informatics evaluation development, to better prepare for the post-COVID-19 era, a new future epidemic, or other unforeseen global health emergencies. An updated evidence-based approach to health informatics is essential to gain public confidence in digital health across primary and other health sectors.

1. Introduction

During the current pandemic, digital health has been recognized as an essential tool for the coronavirus disease response [1], [2] and moving health care into COVID-safe delivery modes [3], [4]. This involves the full spectrum of digital health users (within, and related to primary care). However, the performance of many of the newly built COVID-response initiatives has been questioned since release [1], [5]. For instance, an EU Commission Recommendation (EU) 2020/518 report has raised concerns that many rapidly rolled out applications were not subject to adequate evidence-based evaluation procedures [6]. Indeed, under an unprecedented and acute emergency situation, collecting evidence for digital health can be exceptionally challenging. Subsequently, there is not only the likelihood of defects being embedded into existing health IT systems (if imperfect systems continue to be used after the emergency), but also inefficiencies and errors caused by the rapid rollout may undermine public confidence toward e-health in general [7]. Since the beginning of the pandemic, many digital health commentaries, reviews, and studies have focused on the e-health benefits in supporting COVID-19 [2], [3], [8], [9], as well as its lessons learned [10], [11], [12]. Our work expands the existing literature body to develop a comprehensive conceptual framework that addresses challenges and risks incurred under such rapid e-health ramp-up in the face of the pandemic. An understanding of the digital health risks posed under the broad concept of health systems thinking [13] would enable subsequent learning and transitioning toward an more agile digital health ecosystem in the post-pandemic era.

This paper aims to identify e-health risks, heightened by the conditions of rapid development and deployment, across the full spectrum of health IT applications created, upscaled, or redeployed for COVID-19 [14]. A special focus on primary care is targeted due to its essential role in improving people’s health and wellbeing. Our goals are to address how existing roadmaps of digital health infrastructure, ecosystems, and interoperability were challenged and deviated. We then offer suggestions for corrective evaluation. Digital health infrastructure refers to the applications and communications technology that connect clinical systems into a wider integrated digital ecosystem. This encompasses the processes and interfaces by which clinicians/care providers mutually interact with each other and the subject population. Advocating a patient-centered approach, digital health ecosystems enable managing population health and wellness in a secure way using digital advancement [15]. The ability to understand digital health deficiencies arising from the pandemic response would highlight the need for evaluation techniques to strengthen public and professional confidence in digital health. This study is not about assigning culpability for taking urgent actions essential in an emergency, but about the need for timely review, consolidation, and transition toward a more robust digital health future.

2. Methods

2.1. Study design

The authors first brainstormed early credible evidence, enabling a sketch to retrieve materials regarding risks associated with the COVID-19 digital health response. We adopted the general guidelines of scoping review methodology to enable a rapid synthesis of existing evidence, recognizing that the COVID-19 pandemic and its responses were still evolving. The preferred reporting items for systematic review and meta-analysis extension for scoping reviews (PRISMA-ScR) were employed.

2.2. Search strategy

We carried out a PubMed, CINAHL and EMBASE search retrieving literature from 1st November 2019 to 24th August 2021 (Date - Publication). The search attempted to retrieve an exhaustive literature base associated with two themes: informatics and COVID-19. The detailed set of master search terms is appended in Appendix A. In addition, related official policies, gray literature, websites, and other dynamically available materials (such as news reports) obtained through non-exhaustive snowball sampling and pragmatic digital health networks were screened and assessed.

2.3. Inclusion and exclusion criteria

English language materials that credibly identified significant challenges, risks, problems, threats, or lessons learned regarding digital health scaled for COVID-19 were considered, and included original research, editorials, perspectives, reports, and news items. We excluded studies that advocated the generic benefits of digital health, addressed how technologies were being used, examined digital health advancement/frameworks, or focused on subpopulation/subdomain issues.

2.4. Evidence synthesis

Two phases of screening were conducted. In the first phase, a reviewer screened all the titles and abstracts identified. In the second phase, full texts were reviewed and assessed against the inclusion and exclusion criteria. Studies fulfilling the assessment criteria were reported in a table of qualitative synthesis. For all the included studies, we included the literature source, study country’s income group, study design, an indication of primary care focus, and specific informatics topics. Subsequently, findings were mapped into five areas of risk, and all authors synthesized the evidence, extracted information, and appraised the study outcomes.

3. Results

After removing duplicates, our initial database search yielded 1,244 records, of which 153 were reviewed for full-text assessment. As illustrated in Fig. 1 , 47 studies were included from these databases. In total, together with 11 studies obtained from other additional sources (also evaluated against the assessment criteria), 58 studies were included for reporting qualitative synthesis. Of these, 24 focused specifically on primary care delivery (including those focused on contact tracing and telehealth). More than 85% (50 of 58 studies) were presented as (expert) opinions and review studies. Appendix B comprises the details of the PRISMA-ScR, and Appendix C summarizes the table of studies and the qualitative synthesis of included studies across identified risk areas. Five risk areas were identified, namely, governance, system design and coordination, information access, service provision, and user (professional and public) reception. Fig. 2 illustrates these risk areas, their sub-divisions, and their potential damage to a digital health development.

Fig. 1.

PRISMA study flow diagram.

Fig. 2.

Risk areas under COVID-19 digital response and their potential impairment to health system development.

3.1. Governance

3.1.1. Risk entailed in commissioning style

Governance and commissioning styles were observed as factors affecting e-health deployment during this pandemic [4], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25]. Bayram et al. [17] addressed the overreaching risk imposed through a “command-style” government approach, and that created health IT concerns, uncertainties, and future unknowns. The reason is that the degree of investment in sustainable technological, public healthcare, and health policy development can all hinder the resultant health system, as well as the effectiveness of disease response [19]. This kind of centralized model, that emphasized political leaders directly assuming command, often downplayed professional advice and bypassed established communication channels [19], [26], [27], implying that health IT initiatives under this model may have lacked comprehensive scientific grounding[26]. Other governments adopted an informed approach to enable trust through investing in existing expertise, with public health-related political decisions often made in close collaboration with disease and health system experts [28], [29], [30], [31].

3.1.2. Inadequately coordinated response

Lack of accessibility, internet infrastructure and inadequately coordinated response across health stakeholders were often reported as potential threats during this pandemic [4], [10], [14], [23], [24], [25], [32], [33], [34], [11], [35], [36], [37], [38], [39], [40]. Studies, such as [35], [40], have identified multiple problems arising from insufficient integration and collaboration across primary healthcare, secondary care and the wider public health sector. Frequently, genuine collaboration and solidarity between informaticians and disease experts often faced insurmountable difficulties [41], explaining the state of global uncoordinated health IT systems posed by the pandemic [42]. In primary care settings, the access to smartphones and a reliable internet connection have proved essential to care-seeking patients [4], [21]. Nevertheless, some collaborative effort, such as virtual hackathons and data science networks, like Observational Health Data Sciences and Informatics (OHDSI) [43] and European Health Data & Evidence Network (EHDEN) [44], have demonstrated some successes in enhancing digital coordination for disease response.

3.1.3. Insufficient digital health strategy and regulation

Another governance challenge stems from the deviation of the existing national and global digital health strategies and insufficient regulatory guidelines addressing visionary e-health development and implementation [4], [10], [16], [18], [25], [32], [34], [11], [37], [39], [45], [46], [47], [48], [49], [33], [50], [51], [52]. Humphreys [46] indicated the urgent need for governments to devise system thinking and coherent national strategies bounded by legislation and regulations, to ensure e-health interoperability and quality. Under the current pandemic, some progressive initiatives, such as the EU Health Data Space [53] proposal, have supported the harmonization of international strategies for the regulation, evaluation and use of digital technologies. There were also other successful examples demonstrating strategic alliances with trusted stakeholders [54]. For instance, in primary care settings, the Royal College of General Practitioners (RCGP) sentinel network has demonstrated success in networking primary care collaboration in data analysis to enhance COVID-19 real world knowledge.

3.2. System design and coordination

3.2.1. Sub-optimal/flaws-embedded design

Under the pandemic response, e-health function, design and implementation can be less than perfect either technically [55] or functionally as part of the clinical process [56]. In the short term, sub-optimal solutions can be a satisfactory fix to make rapid progress. At times, the expected functions and user profiles were still relatively simple, and workarounds could produce rapid usable solutions, for example, the increased use of phone and video-consultations in telehealth during COVID-19 [4] has demonstrated that they were sufficiently resilient to adapt to the pandemic in emergency mode. Yet, many health IT applications implemented were deemed to be sub-optimal when efficacy and utility really matter [57], [58], [59], [60], [61]. The potential risk is that, if there is no rectification process in the long run, the imperfect applications may be embedded permanently into the e-health ecosystem and may fracture the overall digital strategy. For instance, the setback and criticism of COVID-19 tracing apps [55] have demonstrated the consequences of ill-defined specifications [60].

3.2.2. Uncoordinated IT

Indeed, the current pandemic situation demands an agile response. However, overlooking and downplaying the value of coordinated IT and design [34], [35] is imprudent. He et al. [34] observed that many systems and apps scaled for COVID-19 were poorly linked, stemming from the fact that these initiatives were developed by different government agencies, health authorities, and organizations. Under such conditions, integration issues, like how to effectively consolidate disease data, patient records and care pathways, and how to allow them feed into the national digital health blueprint [40], are often overlooked. It is particularly important for primary care decision makers to adopt systematic thinking and consider the holistic aspects of the health system. For instance, they may consider the value added by e-health within the primary care system, drawbacks that may be removed, and which primary care initiatives should be prioritized going forward [37]. Pooling of developments for the common good was possible, but its success in practice was rare [62]. Regarding other successful modeling, system approaches for handling health information complexity [63] may be helpful in shaping a robust e-health support for future emergencies.

3.2.3. Lack of technical standards; and incompatible terminologies and coding

Standardized clinical terminologies (such as SNOMED CT[64] and ICD [65]) and standardized data collection projects (such as [66]) could massively increase data comparability; however, in practice, digital designs often omitted these foundations and data standards were often not utilized to their full potential [4], [14], [18], [24], [32], [34], [11], [35], [38], [40], [45], [46], [48], [49], [33], [52], [66], [67], [68], [69]. The lack of integration between digital health designs also resulted in interoperability problems across primary care, and even the wider public health systems. Liaw et al. [4] explained the urgent need for international agencies to establish common data models to share primary care data across regions and nations.

3.3. Information access

3.3.1. Uncertainty related to disease characteristics

Regarding digital solutions tailored for the pandemic, it is critical to consider the unique disease characteristics, risk factors and transmission patterns [19], [21], [24], [37], [50], [17], [70]. However, amidst the current pandemic, many digital health designs were unable to discern the dynamic and uncertainty arising from the COVID-19 disease epidemiology. Early in the pandemic, when researchers globally were still trying to characterize the origin, biological, epidemiological, and transmission aspects of the disease [71], the partial nature of the information available inevitably created challenges regarding appropriate design [72]. Subsequent reports [73], [74] indicated shortcomings and faults, and furthermore, these uncertainties unavoidably created challenges apropos a systematic evaluation with meaningful population health outcomes.

3.3.2. Compromised specification and function

The above risks also resulted in widespread compromised and imperfect digital health designs, specifications, and functions [4], [18], [19], [20], [21], [24], [25], [32], [50], [56], [17], [70], [75], [76]. For instance, automatic contact tracing apps in general attempt to keep track of both exposed and susceptible individuals in close proximity over a significant period of time. However, it is difficult to ascertain what exactly constitutes a meaningful period of interaction between two individuals or an appropriate physical distance rule [77]. Furthermore, one may detect enormous variabilities across different environmental and cultural settings [77], [78], [79], that have already led to varying compromised designs [60].

3.3.3. Lack of evidence-based approach

Usability, effectiveness, evaluation, and validation should not be compromised during any stage of e-health maturity [80], but are often sacrificed through the overwhelming need for rapid COVID-19 deployment [5], [10], [11], [39], [40], [45], [49], [33], [51], [55], [56], [75], [81], [82]. Accepting ‘good enough’ in the short term means that long-term optimization may well have been threatened, risking sub-optimal solutions becoming embedded into post-COVID-19 health systems. Digital health was not widely popularized in any previous pandemics. Given its novelty, neither new-builds nor modifications had prior blueprints designed and calibrated for the pandemic, multiplying the challenges faced. Primary care informatics, such as remote technologies [39], in particular, by being dispersed but also overloaded, are under-investigated and have exhibited potential safety concerns [39].

3.4. Service provision

3.4.1. Time pressure to new built and scaling up

In contrast to the systematic development approach (with a typical product lifespan of 5–10 years or longer), digital health solutions for the pandemic have been created and rolled out rapidly as urgent responses [14], [18], [32], [33], [70], [83], [84], [85], [86] – this is unusual compared to the typically deliberative e-health development process. Given the extreme pressures, many local inventions of ‘new wheel’ designs were observed. The applied policies and procurements were unprecedented in the face of time, resource, and expertise constraints (for instance, the geneticist project leader of the English symptom tracker app indicated a 5-day timeframe for development [87]). By contrast, some countries used creative multi-stakeholder workshops [31], and the pharmaceutical sector has shown instances of inter-competitor collaboration in the common societal interest during the pandemic crisis [88].

3.4.2. Insufficient patient-centricity, equality, and diversity

Patient-centrality, equality and diversity have obtained wide public recognition in recent decades, and these values have become important core concepts [4], [10], [14], [17], [20], [21], [34], [37], [46], [75], [89], [12], [90] of health informatics design in different high-income [5], [22], [23], [25], [32], [39], [48], [91], [92], [93] and low- and middle-income [16], [51], [52], [70] countries. Integration of care pathways, patients’ individuality and preferences, and understanding of problems in accessing services are all identified as important aspects that should be built into modern data systems [94], [95], [96]. In addition, technology may create inequalities between different social, economic, and political groups and may generate unintended consequences [89], [97]. It is also noteworthy that a one-size-fits-all solution is unlikely to suit diverse populations or global development settings [98]. Urgent technology-led applications of a one-shape-for-all nature, if not subsequently reviewed and rectified, may undermine progress in supporting health access equity.

3.4.3. Compromised privacy, confidentiality, and security

Control of citizen’s confidentiality, privacy and data protection may also be compromised [4], [10], [12], [14], [16], [18], [22], [32], [34], [11], [36], [37], [46], [47], [49], [33], [52], [67], [75], [76], [82], [91], [93], [99], [100], [101], [102], [103]. In particular, contact tracing apps have been widely adopted for tracking COVID-19 infections [104], raising privacy concerns regarding personal data breach [20], [21], [23], [24], [25], [38], [39], [50], [55], [56], [70], [105], [106], [107]. The apps introduced by various countries may opt for different data sharing consent and data sharing policies. Recent studies [38], [104] have expressed concerns about how personal data were being handled and to what degree effective tracing and tracking should outweigh privacy and data protection. As the future of public health will most likely rely more on digital methods, the inherent trade-offs between users’ privacy and e-health effectiveness should be carefully examined [103] in the post-pandemic phase.

3.5. User (professional and public) reception

3.5.1. Undermining trust

The varied expedient digital health responses implemented without the normal design and function checks should be evaluated and validated to restore and strengthen both professional and public trust, and an appreciation of digital health benefits. The risk is that any inbuilt flaws and frictions, if uncorrected, will undermine trust and credibility into the future [4], [12], [14], [20], [23], [25], [34], [36], [37], [38], [50], [55], [70], [100], [101], [105], [107], [108].

3.5.2. Risk of liability

Considering that even in traditional health IT systems, unintended and detrimental effects often occur [109], [110], the lack of objective evaluation, verification and rectification during the pandemic has raised serious questions regarding e-health liability [4], [5], [24], [32], [11], [37], [48], [67]. Pagliari [37] addressed recent primary care experience in handling unintended consequences, and concluded that the wider aspect of health systems, the nature of health care work and culture, patient characteristics, and ethical issues should be considered concurrently.

3.5.3. Digital divide

Recent studies [10], [16], [22], [24], [34], [46], [52], [90], [93], [111] have addressed concerns regarding the gap between those who are perforce de facto excluded from e-health (a concept known as digital divide) in the age of COVID-19. Due to persistent social, economic, and political factors, the disparity between different groups in accessing digital health solutions may increase further. When designing health informatics, social determinants of health should be considered, including built environment, social and community context, education, economic stability and health/healthcare access, as these may all perpetuate digital inequity [90]. From a public reception viewpoint, users’ social status, cultural background, age group, living conditions and location, income group, and moral and religious background [14] may affect their acceptance and their ability to use digital health tools, and therefore impact its effectiveness in patient care and service delivery.

4. Discussion

This review resulted in a conceptual framework that addresses a new group of risks in the face of the rapid e-health development under emergency response – a novel addition to existing literature. The risks associated with digital health urgent pandemic responses covered areas of governance, system design and coordination, information access, service provision and user (professional and public) reception. As addressed in the Results section, the e-health risks heightened by the stressed conditions of rapid development and deployment could potentially result in impairments to health system development on different levels. On the health data level, these risks may cause fragmentations of data/records/care pathways, while some newly built solutions may breach personal user rights. On the patient level, those identified risks may discount patient centricity and increase digital inequality across different sociodemographic groups. On the digital health sector level, these risks would potentially fracture the existing digital strategy and interoperability roadmaps. On a broader disease control and user reception level, these e-health risks could potentially discount efficacy in combating the disease, damage professional and consumer trust in digital health technology and hinder patient care and service improvement long term. Considering these potential impacts regarding the digital health infrastructure, health ecosystems, and interoperability roadmap, this section suggests potential evidence-based evaluation directions to attenuate or even resolve the abovementioned e-health risks.

Systematic evaluation of what has been implemented in an emergency seems to be a long-standing ethical imperative to ensure robust digital ecosystems. As IMIA opinion leaders have highlighted [10], [81], evidence-based health informatics should be the backbone of digital health solution development and implementation [80], [112]. Hence, we believe that digital technologies developed for acute response require a more dynamic mechanism to incorporate evidence-based evaluation. Under the timely pandemic response conditions, a rectification process, that considers newly discovered evidence, implemented through a feedback mechanism would enable agile e-health system improvement. The conflicts between feedback and evaluation during the current pandemic should be carefully reviewed and investigated for future emergency purposes. This also presents an opportunity for much greater scientific and professional collaboration in the digital health sector.

Fig. 3 illustrates a concept of how digital health ecosystems may be reinforced through ameliorating the pandemic-triggered risks. We propose a template of important evidence-based evaluation questions, covering areas of design and specification, strategic context, ecosystem integration, patient and citizen focus, implementation and training, maintenance and enhancement, evaluation (and feedback), embedding and ownership, and stretch and extension. As shown in Table 1 , these questions are applicable not only to primary care, but to all health settings, and serves as a general guide to policy makers and system managers, as well as digital health developers to enable them to systematically assess the health systems following emergency response.

Fig. 3.

An image of how the digital health ecosystem may be reinforced to ameliorate pandemic-triggered risks.

Table 1.

Template of key questions for evaluating digital health under emergency response.

Aspect	Key Questions
Design and Specification	•Have subject leaders / experts confirmed the functional specification? •Have end users agreed on the specification, including input and output? •Have data definitions and functionality algorithms been verified as appropriate in the long term, based on cited evidence? •Has the system been stress tested, with user involvement?
Strategic Context	•Does the system fit (or enhance) the pre-existing digital health strategy? •Does the system fit (or enhance) patient-flow processes and collaborative treatment regimes? •Have urgent priorities been delayed by resource diversion, and is rectification in hand?
Ecosystem Integration	•Is there optimal technical and data harmonization and inter-operability? •Is there optimal semantic interoperability? •Does the system meet organizational security standards? •Does the system, with its user protocols, meet data protection and ethical requirements? •Does the system fit operational support processes, including quality assurance, clinical audit, and financial management?
Patient and Citizen Focus	•Is the system suitably focused on citizen and patient interests? •Are confidentiality and data protection robustly addressed; where new principles arise, are these clearly flagged up for subsequent debate? •Are risks and mitigations for digital inequality clearly addressed; are helplines or other support available for the digitally disadvantaged?
Implementation and Training	•Has the system reached all relevant service areas equitably? •Have all users and technical support been fully trained in the system? •Have technical manuals or online support been provided; and have they been validated in use? •Is there lay material for patients where they are direct users? Has the system been stress tested, with user involvement? •Have adequate training materials (including those for later new entrant users) been provided; and have they been validated in use? •Are there help functions incorporated, and are these updated regularly?
Maintenance and Enhancement	•Following the urgent implementation, has ongoing maintenance been put in place? •Is there a technical capacity and agency to enable improvements to be effected?
Evaluation (and Feedback)	•Has there been an organized evaluation or proof-in-use assessment? •Was this planned and openly announced? •Did it directly seek organizational and professional user, and healthcare and patient end user input and views? •Is there an impartial no-blame incident reporting function in place, open to all users and interested parties; is there independent analysis of these reports and the action taken?
Embedding and Ownership	•Recognizing that many systems were implemented in emergency situations, is there a process for retrospectively approving them after any necessary modification into the established organizational digital health ecosystem? •Is there a budget, and lead responsibility, for post-emergency rectification and rationalization? •Is ongoing responsibility for maintenance, enhancement, and related user notification now in place? •Are there appropriate responsibilities, resources, and retraining to address system modification, data reformatting, or partner system modification? •If a system is found inappropriate or redundant post-pandemic, are there resources for system decommissioning with data transfer or archiving, and any necessary replacement functionality?
Stretch and Extension	•Where pre-existing systems (such as tele-consultation) have been extended during the pandemic emergency to new service areas, new patient groups, and / or new groups of users, has this extension been evaluated as a new service? •Has training, including in digital practice, been provided in the new application areas? •Have unanticipated benefits, and unintended problems such as increased health inequalities, been assessed and acted upon?

Many of these questions can be addressed systematically by the developing or implementing body devising a systematic checklist and applying it via a representative audience of professionals, users, and digital system managers. Furthermore, there is also an opportunity for such checklists to be created and validated by professional or academic bodies. Where a specific situation or deeper innovation requires it, a more comprehensive study on impact and outcomes may be needed, and guidance is available, including [113], [114], [115], [116], [117].

As addressed in the Results section, systematic collaboration among different digital health stakeholders during an emergency is difficult, but can be highly beneficial and impactful. A closer, more rapid and more innovative collaboration in the digital health sector, as exemplified by EHTEL [118] and Linux [62] during this pandemic, should be encouraged for future emergencies. Other selected experience [31], [88] also demonstrated how the public interest can be safeguarded through multiple stakeholder involvement in a pandemic response. This evidence demonstrates that digital health experts and professional bodies should increase coordination and establish strong and effective collaboration to better prepare for the post-COVID-19 era, a new future epidemic, or other unforeseen global health emergencies.

Access to evidence is a limitation of this study. It is understood that available literature accessible from scientific databases may result in delays and suffer from funding, publication, and reporting biases[119]. Therefore, scoping review methods were attempted to underpin an opportunistic study, while a snowball sampling approach was employed to identify ranges of emerging issues. It is also noted that the adoption of single-reviewer screening, the absence of risk of bias assessment and quality appraisal, and the inclusion of all study types (including commentaries, editorials, and news), may inevitably circumscribe the evidence synthesized.

5. Conclusion

Through awareness of urgent informal and gray literature alerts, then substantiated by a scoping review of existing literature, this paper summarizes the digital health risks generated during this pandemic. We advocate the employment and enhancement of health informatics evaluation concepts, theories and methods, and deeper sector collaboration, to optimize systems created in the recent emergency, and create more agile IT initiatives in response to future health emergencies. Primary care, as the pivotal and the most patient-facing sector, is therefore a key stakeholder, requiring effective communication with public health and secondary care over shared health risk concerns. The conclusion is that digital health has played an important but imperfect role in urgent responses to the COVID-19 pandemic. Early no-blame evaluation is now important to consolidate the gains while protecting against perpetuating sub-optimal applications. Digital health stands to strengthen or lose credibility and support according to the action taken. And in this context, the opportunities for case studies of processes, and development of practical and incisive international collaboration for the common global good, should be maximized and acted upon.

Summary table

What was already known on the topic •Digital health has been recognized as an essential tool for the coronavirus disease response and management. •Many rapidly rolled out applications were not subject to adequate evidence-based evaluation procedures. •Many digital health commentaries, reviews, and studies have focused on the e-health benefits in supporting COVID-19.

What this study added to our knowledge •Digital health has played an important but imperfect role in urgent responses to the COVID-19 pandemic. •Five risk areas were identified, namely, governance, system design and coordination, information access, service provision, and user (professional and public) reception. •A template of evidence-based evaluation questions was proposed to ameliorate the pandemic-triggered risks. •Stronger coordination and collaboration are needed to better prepare for the post-COVID-19.

Funding

This research was supported by the Japan Society for the Promotion of Science KAKENHI (Grant No. 18H03336).

CRediT authorship contribution statement

Zoie Shui-Yee Wong and Michael Rigby each made substantial contributions to the conceptualization, data curation, formal analysis, investigation, methodology, project administration, validation, visualization, writing – original draft, writing – review & editing of the paper. Zoie Shui-Yee Wong involved in funding acquisition. All the authors are accountable for the integrity of the work.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary material

The following are the Supplementary data to this article: