Table 1. Mapping the six higher-order principles of adaptive governance by systemic oversight (Blasimme and Vayena, JMLE 2018) to key events or concepts in the three cases of study.
Case of Study | |||
---|---|---|---|
Adaptive Governance Principle | Clinical | Operational | Epidemiological |
Adaptivity / Flexibility How do partners navigate the use of novel technologies broadly, especially in changing organizational contexts? |
- Pandemic motivated the Ontario government to create new physician billing codes for virtual care - Hospital B had previously done some diligence on tech company A and were able to move faster in collaboration since tech company was viewed as a trusted partner - Hospital B had an ongoing roll out of new electronic medical record, which changed workflows - Patients/families in LTCH were able to withdraw consent from virtual assessments |
- Hospital D and tech company C had established partnership and had already completed pilot and began to roll out solution prior to pandemic - Pandemic accelerated scale-up of solution through hospital, key COVID-19 use cases were used to pilot new features/applications of solution - Tech company C provided additional flexibility in contracting to accommodate hospital’s community partners |
- Tech company E beginning to organize around highly agile and cross-functional teams to break down silos between departments (e.g., finance, data science, product, etc.) - Tech company E had been exploring novel data streams to inform public health response for over two years prior to the pandemic, developing key internal mechanisms and safeguards - Tech company E regularly publishes in and seeks inspiration from scientific literature to guide products and consulting outputs with novel data streams, viewing peer review of methodologies to be an important gauge of quality - Public health organization F has been exploring how to modernize data science efforts and industry partnerships prior to pandemic - Public health organization F activated an emergency operations centre to centrally coordinate response related to COVID-19 - Public health organization F exploring novel methods of procurement (e.g., challenge-based procurement) |
Inclusiveness What stakeholders are at the table throughout the partnership? |
- Patient and family advisory committee provided feedback on LTCH solution throughout deployment - Medical advisory committee provided feedback on LTCH solution throughout deployment |
- Hospital D fostered high-quality, longitudinal engagement with a wide range of community partners - Tech company C spent hundreds of hours engaging with (e.g., interviewing, shadowing) hospital administrators, clinicians, etc. to understand workflows |
- Public health organization F consulted privacy, legal, and ethics experts to inform efforts with novel data streams - Tech company E had an established working group as well as open door meetings to discuss how to navigate ethical challenges posed by novel data streams—including around ethical “red lines” |
Responsiveness How do partners ensure that risks do not translate into real-world harms? |
- Hospital B had to navigate confusion from patients around if they would be charged for COVID-19 screening due to partnership with tech company A - Tech company A had to navigate massive increase in demand for services (e.g., onboarding new physicians, server upgrades, replying to customer support tickets) |
- Tech company C was able to provide a high degree of customization in response to user needs (e.g., in product features, training materials, etc.) - Hospital D had to navigate potential liability with external community partners using their “instance” of the solution |
- Tech company E had well established measures to control for privacy (e.g., spatiotemporal aggregation), security (e.g., two-factor authentication), and quality (e.g., review of outputs at multiple stages by managers) when users were working with novel data streams to generate consulting outputs - Tech company E created automated quality assurance bots to continuously ensure integrity of back-end data pipelines - Tech company E created streamlined workflows on Slack to respond to client issues as they appeared (e.g., who discovered the problem, what causes the problem, who is working on a fix, etc.) |
Reflexivity Are partners aware of the risks posed by emerging technologies in their respective domains? |
- Hospital B Leadership recognized that older adults are medically complex, require care from multiple specialists at a given time, have additional cognitive/communication barriers and can rapidly deteriorate if transported to ED–providing “burning platform” for deployment during pandemic - Vulnerable populations (e.g., recent immigrants, refugees) may not have a health insurance card to register with; could leave the field blank - Vulnerable populations may not have devices or internet access to engage with virtual screening; could still arrive to get testing conventionally |
- Hospital D leadership recognized that physicians and other health providers were already relying on non-secure and inefficient solutions for care coordination (e.g., Whatsapp, texting, etc.)–providing “burning platform” for accelerating deployment during pandemic - Hospital D leadership recognized that COVID-19 patients could deteriorate rapidly and that care coordination involves numerous providers/specialists (e.g., infectious diseases) when designing workflows |
- Leadership across the partnership recognized the ethical and legal implications of working with novel data streams to inform public health response; sought to minimize the spatiotemporal granularity of these streams at all phases of analysis in keeping with ethical principles and legal responsibilities (e.g., data sharing agreements, contracts with vendors) |
Monitoring How do partners detect and keep abreast of emerging risks, both in terms of outputs and overall partnership direction? |
- Ongoing meetings and alignment between C-Suite at hospital B and tech company A - Tech company A had standard practice of using a risk registry that would be updated through project life cycle - Patients could directly provide feedback into the virtual care platform, which was integrated into product roadmaps - Clearly measurable KPIs were developed in alignment with strategic objectives (e.g., reduce avoidable hospital transfers) - Tech company A recently completed provincial vendor verification, meeting standards for privacy, security, and interoperability |
- Ongoing meetings and alignment between C-Suite at hospital D and tech company C - Clearly measurable KPIs were developed where possible - Continuously evaluated expansion of features / workflows against core value proposition of solution - Tech company C continuously sought out feedback from diverse user groups to inform product roadmap and features Tech company C recently completed detailed security certification and displays key features on a web-based dashboard |
- Ongoing meetings between C-Suite at tech company and high-level managers at public health organization F - Tech company E continuously sought out feedback from users to inform consulting outputs and product roadmap - Tech company E recently completed security audit and will be bringing on in-house security expertise to guide work with novel data streams |
Acronyms: LTCH–Long-term care home; ED–Emergency department; KPI–Key performance indicators